To evaluate accurately the video quality and make it consistent with the subjective evaluation result, a video quality assessment method based on saliency region and motion characteristics weighting is proposed in this paper. This method is based on traditional structural similarity index measurement (SSIM) and makes some improvement. The spatial and temporal saliency is firstly extracted by spectrum analysis and visual attention model combined with motion characteristics, respectively. Then the frame saliency can be extracted by dynamic fusion of temporal and spatial saliency. Finally, the quality assessment index for entire video frame can be got by frame saliency weighted SSIM. The results of experiment on LIVE VQA standard data set show that this index is more close to objective assessment on video quality from human visual system.