3D Gaussian Splatting (3DGS) is a popular technique for 3D novel view synthesis due to its fast training and rendering speeds. However, current 3DGS training is limited to a single GPU, which restricts its ability to handle high-resolution and large-scale 3D reconstruction tasks due to memory constraints. To address this, the authors introduce Grendel, a distributed system designed to partition 3DGS parameters and parallelize computation across multiple GPUs. Grendel employs sparse all-to-all communication to transfer necessary Gaussians to pixel partitions and performs dynamic load balancing. Unlike existing 3DGS systems that train using one camera view image at a time, Grendel supports batched training with multiple views. The authors explore various optimization hyperparameter scaling strategies and find that a simple sqrt(batch_size) scaling rule is highly effective. Evaluations using large-scale, high-resolution scenes show that Grendel enhances rendering quality by scaling up 3DGS parameters across multiple GPUs. On the "Rubble" dataset, Grendel achieves a test PSNR of 27.28 by distributing 40.4 million Gaussians across 16 GPUs, compared to a PSNR of 26.28 using 11.2 million Gaussians on a single GPU. Grendel is an open-source project available at <https://github.com/nyu-systems/Grendel-GS>.3D Gaussian Splatting (3DGS) is a popular technique for 3D novel view synthesis due to its fast training and rendering speeds. However, current 3DGS training is limited to a single GPU, which restricts its ability to handle high-resolution and large-scale 3D reconstruction tasks due to memory constraints. To address this, the authors introduce Grendel, a distributed system designed to partition 3DGS parameters and parallelize computation across multiple GPUs. Grendel employs sparse all-to-all communication to transfer necessary Gaussians to pixel partitions and performs dynamic load balancing. Unlike existing 3DGS systems that train using one camera view image at a time, Grendel supports batched training with multiple views. The authors explore various optimization hyperparameter scaling strategies and find that a simple sqrt(batch_size) scaling rule is highly effective. Evaluations using large-scale, high-resolution scenes show that Grendel enhances rendering quality by scaling up 3DGS parameters across multiple GPUs. On the "Rubble" dataset, Grendel achieves a test PSNR of 27.28 by distributing 40.4 million Gaussians across 16 GPUs, compared to a PSNR of 26.28 using 11.2 million Gaussians on a single GPU. Grendel is an open-source project available at <https://github.com/nyu-systems/Grendel-GS>.