GAZELLE is a low-latency framework for secure neural network inference, designed to allow clients to obtain classification results without revealing their input to the server while ensuring the server's neural network remains private. The system combines homomorphic encryption and two-party computation techniques, including garbled circuits, to achieve efficient and secure inference. GAZELLE makes three key contributions: (1) a fast homomorphic encryption library for basic operations like SIMD addition and multiplication; (2) optimized homomorphic linear algebra kernels for neural network layers; and (3) encryption switching protocols that seamlessly convert between homomorphic and garbled circuit encodings. GAZELLE outperforms existing systems like MiniONN and Chameleon in online runtime, achieving 20× and 30× speedups, respectively. It also surpasses fully homomorphic approaches like CryptoNets by three orders of magnitude in runtime. The system is evaluated on benchmark neural networks trained on MNIST and CIFAR-10 datasets, demonstrating significant improvements in both performance and efficiency. GAZELLE uses a packed additively homomorphic encryption (PAHE) scheme, which supports fast matrix-vector multiplications and convolutions. It also integrates garbled circuits for non-linear layers, allowing for efficient secure computation. The system's design enables a secure and efficient inference process, where the client receives the classification result without revealing their input, and the server learns nothing about the input. The paper describes the technical details of GAZELLE, including its homomorphic encryption library, linear algebra kernels, and encryption switching protocols. It also discusses the trade-offs between homomorphic encryption and garbled circuits, showing that combining both techniques can lead to significant performance gains. The system is evaluated on various neural network layers, with a focus on linear and non-linear operations, and demonstrates efficient performance in both small and large networks. The results show that GAZELLE achieves a 20-80× reduction in online bandwidth per inference, with end-to-end latency improvements of up to 3.6 seconds for a CIFAR-10 network. The system is also compared to other secure neural network inference protocols, showing significant improvements in both computational and communication efficiency.GAZELLE is a low-latency framework for secure neural network inference, designed to allow clients to obtain classification results without revealing their input to the server while ensuring the server's neural network remains private. The system combines homomorphic encryption and two-party computation techniques, including garbled circuits, to achieve efficient and secure inference. GAZELLE makes three key contributions: (1) a fast homomorphic encryption library for basic operations like SIMD addition and multiplication; (2) optimized homomorphic linear algebra kernels for neural network layers; and (3) encryption switching protocols that seamlessly convert between homomorphic and garbled circuit encodings. GAZELLE outperforms existing systems like MiniONN and Chameleon in online runtime, achieving 20× and 30× speedups, respectively. It also surpasses fully homomorphic approaches like CryptoNets by three orders of magnitude in runtime. The system is evaluated on benchmark neural networks trained on MNIST and CIFAR-10 datasets, demonstrating significant improvements in both performance and efficiency. GAZELLE uses a packed additively homomorphic encryption (PAHE) scheme, which supports fast matrix-vector multiplications and convolutions. It also integrates garbled circuits for non-linear layers, allowing for efficient secure computation. The system's design enables a secure and efficient inference process, where the client receives the classification result without revealing their input, and the server learns nothing about the input. The paper describes the technical details of GAZELLE, including its homomorphic encryption library, linear algebra kernels, and encryption switching protocols. It also discusses the trade-offs between homomorphic encryption and garbled circuits, showing that combining both techniques can lead to significant performance gains. The system is evaluated on various neural network layers, with a focus on linear and non-linear operations, and demonstrates efficient performance in both small and large networks. The results show that GAZELLE achieves a 20-80× reduction in online bandwidth per inference, with end-to-end latency improvements of up to 3.6 seconds for a CIFAR-10 network. The system is also compared to other secure neural network inference protocols, showing significant improvements in both computational and communication efficiency.