Gesture recognition based on small samples is one of the main trends in the advanced human-computer interaction research. A novel gesture recognition method based on adaptive K-nearest neighbor (A-KNN) and linear discriminant analysis (LDA) is presented. First, hand-shape images are segmented from the given interaction videos, and scaled to the same size to construct the training set. Then an optimized LDA algorithm is designed to extract gesture features. Finally, an improved KNN algorithm is introduced with adaptive Kvalue to classify the real-time gesture information. Test results show that the correct recognition rate of the proposed approach is higher than most existing methods. 