The nonlinear characteristics of light emitting diode (LED) is an important factor in visible light communication (VLC) system based on the orthogonal frequency division multiplexing (OFDM). To decrease the effect of LED nonlinear characteristics, a transmission system based on amplitude-phase shift keying (APSK) modulation combined with OFDM technology is introduced in the VLC system. The schemes and the applications of APSK in VLC system based on OFDM are also introduced. The VLC system model based on APSK-OFDM modulation is compared with the system based on quadrature amplitude modulation (QAM). The simulation result shows that in OFDM system, the performance of QAM system is better than that of APSK system in the ideal channel, but the result is reversed when considering the nonlinear characteristics of LED.