To improve the performance of Compressed Video Sensing (CVS), a multi-hypothesis based hierarchical reconstruction method is proposed. In the presented framework, the key frame in Group of Picture (GOP) is first reconstructed independently. Afterwards, reconstruction level is allocated for each non-key frame, following which the reconstruction is processed from the lowest level to the highest one. When reconstructing a non-key frame, block by block reconstruction is processed. The temporal data set in reference frames and spatial data set in current frame are taken as multi-hypothesis (MH) for current block, followed by solving the Total Variation Minimization (TVmin) problem to reconstruct the prediction residual. The final reconstructed image is formed by adding the prediction residual to the prediction value. Experimental results show that compared with the existing method, the proposed one can get higher Peak Signal to Noise Ratio (PSNR) up to about 3.2dB at the same sampling rate.