A tracking before detection(TBD) tracking method for minimal targets tracking in unmanned aerial vehicle (UAV) visible image based on improved Kalman filter is presented. Firstly, detected target obtained by detecting algorithm is used as the measurement value of Kalman filter. Parameters of matching similarity in the detection process is used as an important reference for measurement noise covariance matrix of Kalman filter. Secondly, in the tracking module, tracking framework based on Kalman filter is established to predict the target position in next frame. Finally, targets are searched by detection module in local area accor ding to predictive position. In addition, in order to improve the tracking efficiency, accumulation error between detection position and predictive position is calculated to choose detection mode. Global detection mode is taken if accumulation error is greater than the given threshold and accumulation error is set to be zero, or local detection mode is done. The strategy can effectively reduce computational complexity of the TBD tracking method. Simulation experiment results show that the method can obtain the better performance of detection and tracking than that of classic Kalman filter.