CaloDREAM is a detector response emulation framework that uses Conditional Flow Matching (CFM) combined with transformer elements to simulate calorimeter showers. The framework combines an autoregressive transformer for energy simulation and a vision transformer for high-dimensional voxel distributions. It also incorporates latent diffusion for efficient training and bespoke samplers for faster evaluation. The method is tested on datasets 2 and 3 of the CaloChallenge, demonstrating high-fidelity calorimeter shower generation. The approach uses a factorized architecture, with an energy network and a shape network, and leverages latent diffusion to reduce dimensionality while maintaining performance. The framework shows that diffusion networks, despite being slower in forward generation, can be efficiently evaluated with bespoke solvers. The results indicate that the method achieves high precision in simulating calorimeter showers, with performance comparable to traditional methods. The study highlights the potential of modern generative networks in simulating complex detector responses, particularly in high-dimensional phase spaces. The framework is shown to be effective in handling sparse data and maintaining precision, even with reduced latent spaces. The results also demonstrate the importance of proper sampling techniques and the benefits of using latent diffusion for efficient training and sampling. The study concludes that CaloDREAM provides a promising approach for simulating calorimeter showers with high accuracy and efficiency.CaloDREAM is a detector response emulation framework that uses Conditional Flow Matching (CFM) combined with transformer elements to simulate calorimeter showers. The framework combines an autoregressive transformer for energy simulation and a vision transformer for high-dimensional voxel distributions. It also incorporates latent diffusion for efficient training and bespoke samplers for faster evaluation. The method is tested on datasets 2 and 3 of the CaloChallenge, demonstrating high-fidelity calorimeter shower generation. The approach uses a factorized architecture, with an energy network and a shape network, and leverages latent diffusion to reduce dimensionality while maintaining performance. The framework shows that diffusion networks, despite being slower in forward generation, can be efficiently evaluated with bespoke solvers. The results indicate that the method achieves high precision in simulating calorimeter showers, with performance comparable to traditional methods. The study highlights the potential of modern generative networks in simulating complex detector responses, particularly in high-dimensional phase spaces. The framework is shown to be effective in handling sparse data and maintaining precision, even with reduced latent spaces. The results also demonstrate the importance of proper sampling techniques and the benefits of using latent diffusion for efficient training and sampling. The study concludes that CaloDREAM provides a promising approach for simulating calorimeter showers with high accuracy and efficiency.