This paper presents a novel deep learning framework for the diagnosis and referral of retinal diseases using three-dimensional optical coherence tomography (OCT) scans. The framework consists of two stages: a deep segmentation network and a deep classification network. The segmentation network creates device-independent tissue segmentations, while the classification network analyzes these segmentations to provide diagnoses and referral suggestions. The framework was trained on 14,884 OCT scans from 7621 patients and demonstrated expert-level performance in making referral decisions for a range of sight-threatening retinal diseases. The performance of the framework was compared to that of eight clinical experts, including four retina specialists and four optometrists, and it achieved or exceeded their performance in some cases. The framework also showed high accuracy when using tissue segmentations from a different type of OCT scanner, demonstrating its device independence. The study highlights the potential of this approach to improve the efficiency and accuracy of retinal disease diagnosis and referral in clinical settings.This paper presents a novel deep learning framework for the diagnosis and referral of retinal diseases using three-dimensional optical coherence tomography (OCT) scans. The framework consists of two stages: a deep segmentation network and a deep classification network. The segmentation network creates device-independent tissue segmentations, while the classification network analyzes these segmentations to provide diagnoses and referral suggestions. The framework was trained on 14,884 OCT scans from 7621 patients and demonstrated expert-level performance in making referral decisions for a range of sight-threatening retinal diseases. The performance of the framework was compared to that of eight clinical experts, including four retina specialists and four optometrists, and it achieved or exceeded their performance in some cases. The framework also showed high accuracy when using tissue segmentations from a different type of OCT scanner, demonstrating its device independence. The study highlights the potential of this approach to improve the efficiency and accuracy of retinal disease diagnosis and referral in clinical settings.