Microwave photonic signal processing has drawn a lot of attentions in the past decades due to the distinct advantages brought by the photonic technology, such as broad bandwidth, large tunability, immunity to electromagnetic interference and capability to perform parallel processing. As compared with the conventional electro-optical phase modulation and intensity modulation, polarization modulation converts (Ratio frequency) RF signal to be processed to an optical parameter with two degrees of freedom (i.e.polarization). By connecting a polarizer to the polarization modulator, to remove one degree of freedom, the phase modulation and intensity modulation can be easily obtained. As a result, any microwave photonic signal processing based on phase modulation and intensity modulation can be realized based on the polarization modulation. If the output of a polarization modulator is split into several branches and in each branch a polarizer is incorporated, multiple modulation schemes can be implemented simultaneously, which can be used to achieve multichannel or multifunction signal processing. In addition, based on the single sideband polarization modulation, a high performance photonic microwave phase shifter can be realized, which can be further applied to implement phase coding, microwave filtering and beamforming network. This paper will establish a theoretical model of the polarization modulation and describe the principle of the polarization-modulation-based signal processing systems.