This review discusses the recent progress in high-speed memory technologies for AI hardware accelerators, focusing on the global buffer memory in digital systolic-array architectures. Traditional static random-access memory (SRAM) is widely used due to its fast access speed and high endurance, but it is expensive and has high standby power consumption. The review highlights alternative high-speed memory candidates, including capacitorless gain cell-based embedded dynamic random-access memories (eDRAMs), ferroelectric memories (FeFETs and FeRAMs), spin-transfer torque magnetic random-access memories (STT-MRAMs), and spin-orbit torque magnetic random-access memories (SOT-MRAMs). These technologies offer faster access speeds and higher endurance compared to SRAM, making them potential replacements for global buffers in AI accelerators. The review covers the operation principles, prospects, and challenges of each technology. For example, eDRAMs use a capacitorless two-transistor gain cell architecture, while FeRAMs employ metal–ferroelectric–metal capacitors. STT-MRAMs and SOT-MRAMs leverage the tunneling magnetoresistance effect in magnetic tunnel junctions. The review also presents a benchmarking analysis of these memory candidates on tensor processing unit (TPU)-like architectures, comparing their energy efficiency and area consumption with SRAM. The industrial development and technological challenges in buffer memory applications are discussed, along with the state-of-the-art prototype chip demonstrations for some of these memory technologies. The review concludes by highlighting the potential of emerging memories in reducing the silicon area and improving energy efficiency in AI hardware accelerators, particularly in low-power edge devices.This review discusses the recent progress in high-speed memory technologies for AI hardware accelerators, focusing on the global buffer memory in digital systolic-array architectures. Traditional static random-access memory (SRAM) is widely used due to its fast access speed and high endurance, but it is expensive and has high standby power consumption. The review highlights alternative high-speed memory candidates, including capacitorless gain cell-based embedded dynamic random-access memories (eDRAMs), ferroelectric memories (FeFETs and FeRAMs), spin-transfer torque magnetic random-access memories (STT-MRAMs), and spin-orbit torque magnetic random-access memories (SOT-MRAMs). These technologies offer faster access speeds and higher endurance compared to SRAM, making them potential replacements for global buffers in AI accelerators. The review covers the operation principles, prospects, and challenges of each technology. For example, eDRAMs use a capacitorless two-transistor gain cell architecture, while FeRAMs employ metal–ferroelectric–metal capacitors. STT-MRAMs and SOT-MRAMs leverage the tunneling magnetoresistance effect in magnetic tunnel junctions. The review also presents a benchmarking analysis of these memory candidates on tensor processing unit (TPU)-like architectures, comparing their energy efficiency and area consumption with SRAM. The industrial development and technological challenges in buffer memory applications are discussed, along with the state-of-the-art prototype chip demonstrations for some of these memory technologies. The review concludes by highlighting the potential of emerging memories in reducing the silicon area and improving energy efficiency in AI hardware accelerators, particularly in low-power edge devices.