For colored lighting and complex background facial video image, this article presents a skin color extraction algorithm based on CbCr-CgCr double Gaussian average degree of likelihood and inter-frame difference of I component in YIQ color space. This algorithm calculates the CbCr-CgCr double Gaussian average likelihood of facial video input image after preprocessing , roughly segments the average likelihood image by using the value of average likelihood and the value of I component, and then extracts the skin color region by combinating the skin coarse segmented image and I component inter-frame difference image. Experiment demonstrates that this algorithm has the good results of skin color extraction.