The paper systematically studies topological phases of insulators and superconductors in three spatial dimensions (3D). It finds that there exist topologically non-trivial 3D insulators or superconductors in five out of ten symmetry classes introduced by Altland and Zirnbauer using random matrix theory over a decade ago. One of these is the recently introduced $\mathbb{Z}_2$ topological insulator in the symplectic (or spin-orbit) symmetry class. The authors show that there are exactly four more topological insulators. For these systems, all of which are time-reversal invariant in 3D, the space of insulating ground states satisfying certain discrete symmetry properties is partitioned into topological sectors separated by quantum phase transitions. Three of these five topologically non-trivial phases can be realized as time-reversal invariant superconductors, and in these, the different topological sectors are characterized by an integer winding number defined in momentum space. When such 3D topological insulators are terminated by a two-dimensional surface, they support a number of Dirac fermion (Majorana fermion when spin rotation symmetry is completely broken) surface modes that remain gapless under arbitrary perturbations of the Hamiltonian that preserve the characteristic discrete symmetries, including disorder. These topological phases can be thought of as three-dimensional analogues of well-known paired topological phases in two spatial dimensions, such as the spinless chiral $(p_x \pm i p_y)$-wave superconductor (or Moore-Read Pfaffian state). In the corresponding topologically non-trivial (analogous to "weak pairing") and topologically trivial (analogous to "strong pairing") 3D phases, the wave functions exhibit distinct behaviors. When electromagnetic U(1) gauge fields and fluctuations of the gap functions are included in the dynamics, the superconducting phases with non-vanishing winding numbers possess non-trivial topological ground state degeneracies.The paper systematically studies topological phases of insulators and superconductors in three spatial dimensions (3D). It finds that there exist topologically non-trivial 3D insulators or superconductors in five out of ten symmetry classes introduced by Altland and Zirnbauer using random matrix theory over a decade ago. One of these is the recently introduced $\mathbb{Z}_2$ topological insulator in the symplectic (or spin-orbit) symmetry class. The authors show that there are exactly four more topological insulators. For these systems, all of which are time-reversal invariant in 3D, the space of insulating ground states satisfying certain discrete symmetry properties is partitioned into topological sectors separated by quantum phase transitions. Three of these five topologically non-trivial phases can be realized as time-reversal invariant superconductors, and in these, the different topological sectors are characterized by an integer winding number defined in momentum space. When such 3D topological insulators are terminated by a two-dimensional surface, they support a number of Dirac fermion (Majorana fermion when spin rotation symmetry is completely broken) surface modes that remain gapless under arbitrary perturbations of the Hamiltonian that preserve the characteristic discrete symmetries, including disorder. These topological phases can be thought of as three-dimensional analogues of well-known paired topological phases in two spatial dimensions, such as the spinless chiral $(p_x \pm i p_y)$-wave superconductor (or Moore-Read Pfaffian state). In the corresponding topologically non-trivial (analogous to "weak pairing") and topologically trivial (analogous to "strong pairing") 3D phases, the wave functions exhibit distinct behaviors. When electromagnetic U(1) gauge fields and fluctuations of the gap functions are included in the dynamics, the superconducting phases with non-vanishing winding numbers possess non-trivial topological ground state degeneracies.