Coherent detection in optical fiber systems is a key technique for achieving high spectral and power efficiency. This review discusses detection methods, including noncoherent, differentially coherent, and coherent detection, as well as a hybrid method. Polarization-multiplexed quadrature-amplitude modulation (QAM) maximizes spectral and power efficiency by utilizing all four available degrees of freedom (DOF), the two field quadratures in the two polarizations. Dual-polarization homodyne or heterodyne downconversion are linear processes that fully recover the received signal field in these four DOF. When downconverted signals are sampled at the Nyquist rate, transmission impairments can be compensated using digital signal processing (DSP). Linear impairments, such as chromatic dispersion (CD) and polarization-mode dispersion (PMD), can be compensated quasi-exactly using finite impulse response (FIR) filters. Some nonlinear impairments, such as intra-channel four-wave mixing and nonlinear phase noise, can be partially compensated. Carrier phase recovery can be performed using feedforward methods, even when phase-locked loops (PLLs) may fail due to delay constraints. DSP-based compensation enables a receiver to adapt to time-varying impairments and facilitates the use of advanced forward-error-correction codes. Both single- and multi-carrier system implementations are discussed. For a given modulation format, coherent detection offers fundamentally the same spectral and power efficiency as noncoherent detection, but may differ in practice due to different impairments and implementation details. With advances in analog-to-digital converters (ADCs) and integrated circuit technology, DSP-based coherent receivers at bit rates up to 100 Gbit/s should become practical within the next few years.Coherent detection in optical fiber systems is a key technique for achieving high spectral and power efficiency. This review discusses detection methods, including noncoherent, differentially coherent, and coherent detection, as well as a hybrid method. Polarization-multiplexed quadrature-amplitude modulation (QAM) maximizes spectral and power efficiency by utilizing all four available degrees of freedom (DOF), the two field quadratures in the two polarizations. Dual-polarization homodyne or heterodyne downconversion are linear processes that fully recover the received signal field in these four DOF. When downconverted signals are sampled at the Nyquist rate, transmission impairments can be compensated using digital signal processing (DSP). Linear impairments, such as chromatic dispersion (CD) and polarization-mode dispersion (PMD), can be compensated quasi-exactly using finite impulse response (FIR) filters. Some nonlinear impairments, such as intra-channel four-wave mixing and nonlinear phase noise, can be partially compensated. Carrier phase recovery can be performed using feedforward methods, even when phase-locked loops (PLLs) may fail due to delay constraints. DSP-based compensation enables a receiver to adapt to time-varying impairments and facilitates the use of advanced forward-error-correction codes. Both single- and multi-carrier system implementations are discussed. For a given modulation format, coherent detection offers fundamentally the same spectral and power efficiency as noncoherent detection, but may differ in practice due to different impairments and implementation details. With advances in analog-to-digital converters (ADCs) and integrated circuit technology, DSP-based coherent receivers at bit rates up to 100 Gbit/s should become practical within the next few years.