CT2Rep is the first method designed to generate radiology reports for 3D medical imaging, specifically for chest CT volumes. The authors address the challenges of computational complexity and data scarcity in 3D imaging by introducing a novel auto-regressive causal transformer for 3D vision feature extraction. They also incorporate relational memory and a multi-modal fusion module to leverage longitudinal data, enhancing the accuracy and context of generated reports. The effectiveness of CT2Rep is demonstrated through a comprehensive evaluation using the CT-RATE dataset, which includes 25,701 non-contrast chest CT volumes from 21,314 unique patients. The method outperforms a state-of-the-art vision encoder, CT-Net, and a baseline model, showcasing its potential for automating radiology report generation in 3D medical imaging. The code and trained models are made publicly available to facilitate further research.CT2Rep is the first method designed to generate radiology reports for 3D medical imaging, specifically for chest CT volumes. The authors address the challenges of computational complexity and data scarcity in 3D imaging by introducing a novel auto-regressive causal transformer for 3D vision feature extraction. They also incorporate relational memory and a multi-modal fusion module to leverage longitudinal data, enhancing the accuracy and context of generated reports. The effectiveness of CT2Rep is demonstrated through a comprehensive evaluation using the CT-RATE dataset, which includes 25,701 non-contrast chest CT volumes from 21,314 unique patients. The method outperforms a state-of-the-art vision encoder, CT-Net, and a baseline model, showcasing its potential for automating radiology report generation in 3D medical imaging. The code and trained models are made publicly available to facilitate further research.