Aiming at the analysis and the recognization in intelligent surveillance system. Hidden Markov model (HMM) is applied to analyze abnormal events detection in surveillance system. The method extracts motive object by background substraction, encodes shape features, color and changes rate of frames for feature vector. In training, feature vector is applied to HMM to obtain parameters A and B. In detecting, the feature vector is input into the HMM to detect abnormal events. The experiment shows that the method can detect abnormal events quickly and accurately.