In this paper,a frequency-domain mathematical model is deduced for polarization-division multiplexing(PDM)co-herent optical orthogonal frequency division multiplexing(CO-OFDM)system with transceiver in-phase and quadra-ture(IQ)imbalances.A novel training symbol structure is designed in which the mirror-subcarriers of pilot subcarriers were modulated with zero signal so that the channel distortion with transceiver IQ imbalances can be estimated.It proves that the channel distortion and transmitter IQ imbalances cannot be separated using the training symbols;therefore,the channel equalization method was used to recover the signals.Simulation results show that at 2 dB trans-mitter amplitude imbalances,15° transmitter phase imbalances,100 Gbit/s transmission rate,1 040 km standard sig-nal-mode fiber link,and 1×10-3bit error rate,23.5 dB optical signal-to-noise ratio(OSNR)is necessary for the pro-posed method.However,the compared schemes cannot achieve effective transmission.