In order to improve the visual perception ability of unmanned surface vehicle（USV） in harsh navigation scene， a polarization image fusion method of visible light for water navigation scene is proposed based on hue， saturation， value（HSV） color space. The fusion rules for different regions are formulated in accordance with the polarization characteristics of the water navigation scene. And based on the HSV color space， the color information of the original scene is fused， which is tested that realizes the semantic segmentation of the harsh navigation scene image. The most striking result is that the pixel accuracy（PA） value in the flare scene is 0.768 2. And the experimental results indicate that the proposed method can enhance image contrast， highlight edge contour information， and stably obtain feature information with strong contrast as well as better target characteristics in harsh navigation scene， which improves the USV’s performance in harsh navigation scene to a certain extent.