The information of drogue containing position and size are used to establish Kalman filter equations. The model parameters are set considering drogue′s motion features in refueling process and the object occlusion coefficient is introduced to complete the parameters′ adaptive adjustment. The next-time states of moving drogue are estimated by the model and the estimation is corrected by observation. The experimental results show that the algorithm can track the drogue stably in work distances. When non-target moving objects appear in the images or the drogue is obscured, the algorithm still has the ability of continuous tracking.