VMware ESX Server is a thin software layer that efficiently multiplexes hardware resources among virtual machines (VMs) running unmodified commodity operating systems. This paper introduces several novel mechanisms and policies for managing memory in ESX Server. A ballooning technique reclaims pages considered least valuable by the guest OS. An idle memory tax ensures efficient memory utilization while maintaining performance isolation. Content-based page sharing and hot I/O page remapping exploit transparent page remapping to eliminate redundancy and reduce copying overheads. These techniques are combined to efficiently support VM workloads that overcommit memory. ESX Server virtualizes the Intel IA-32 architecture and is used in production on servers running multiple instances of unmodified operating systems such as Microsoft Windows 2000 Advanced Server and Red Hat Linux 7.2. Unlike VMware Workstation, which uses a hosted virtual machine architecture, ESX Server manages system hardware directly, providing higher I/O performance and complete control over resource management. The need to run existing operating systems without modification presented several challenges. Unlike IBM’s mainframe division, we could not influence the design of the guest operating systems. Even Disco prototypes, designed to run unmodified operating systems, required minor modifications in the IRIX kernel sources. This paper introduces several novel mechanisms and policies that ESX Server 1.5 uses to manage memory. High-level resource management policies compute a target memory allocation for each VM based on specified parameters and system load. These allocations are achieved by invoking lower-level mechanisms to reclaim memory from VMs. In addition, a background activity exploits opportunities to share identical pages between VMs, reducing overall memory pressure on the system. Section 2 describes low-level memory virtualization. Section 3 discusses mechanisms for reclaiming memory to support dynamic resizing of virtual machines. Section 4 presents a general technique for conserving memory by sharing identical pages between VMs. Section 5 discusses the integration of working-set estimates into a proportional-share allocation algorithm. Section 6 describes the high-level allocation policy that coordinates these techniques. Section 7 presents a remapping optimization that reduces I/O copying overheads in large-memory systems. Section 8 examines related work. Finally, we summarize our conclusions and highlight opportunities for future work in Section 9. ESX Server uses a ballooning technique to reclaim memory by coaxing the guest OS into cooperating. A small balloon module is loaded into the guest OS as a pseudo-device driver or kernel service. It communicates with ESX Server via a private channel. When the server wants to reclaim memory, it instructs the driver to "inflate" by allocating pinned physical pages within the VM. Similarly, the server may "deflate" the balloon by instructing it to deallocate previously-allocated pages. Ballooning increases memory pressure in the guest OS, causing it to invoke its own native memory management algorithms. When memory is plentiful, the guest OS will return memory from its free list. When memory is scarce, it mustVMware ESX Server is a thin software layer that efficiently multiplexes hardware resources among virtual machines (VMs) running unmodified commodity operating systems. This paper introduces several novel mechanisms and policies for managing memory in ESX Server. A ballooning technique reclaims pages considered least valuable by the guest OS. An idle memory tax ensures efficient memory utilization while maintaining performance isolation. Content-based page sharing and hot I/O page remapping exploit transparent page remapping to eliminate redundancy and reduce copying overheads. These techniques are combined to efficiently support VM workloads that overcommit memory. ESX Server virtualizes the Intel IA-32 architecture and is used in production on servers running multiple instances of unmodified operating systems such as Microsoft Windows 2000 Advanced Server and Red Hat Linux 7.2. Unlike VMware Workstation, which uses a hosted virtual machine architecture, ESX Server manages system hardware directly, providing higher I/O performance and complete control over resource management. The need to run existing operating systems without modification presented several challenges. Unlike IBM’s mainframe division, we could not influence the design of the guest operating systems. Even Disco prototypes, designed to run unmodified operating systems, required minor modifications in the IRIX kernel sources. This paper introduces several novel mechanisms and policies that ESX Server 1.5 uses to manage memory. High-level resource management policies compute a target memory allocation for each VM based on specified parameters and system load. These allocations are achieved by invoking lower-level mechanisms to reclaim memory from VMs. In addition, a background activity exploits opportunities to share identical pages between VMs, reducing overall memory pressure on the system. Section 2 describes low-level memory virtualization. Section 3 discusses mechanisms for reclaiming memory to support dynamic resizing of virtual machines. Section 4 presents a general technique for conserving memory by sharing identical pages between VMs. Section 5 discusses the integration of working-set estimates into a proportional-share allocation algorithm. Section 6 describes the high-level allocation policy that coordinates these techniques. Section 7 presents a remapping optimization that reduces I/O copying overheads in large-memory systems. Section 8 examines related work. Finally, we summarize our conclusions and highlight opportunities for future work in Section 9. ESX Server uses a ballooning technique to reclaim memory by coaxing the guest OS into cooperating. A small balloon module is loaded into the guest OS as a pseudo-device driver or kernel service. It communicates with ESX Server via a private channel. When the server wants to reclaim memory, it instructs the driver to "inflate" by allocating pinned physical pages within the VM. Similarly, the server may "deflate" the balloon by instructing it to deallocate previously-allocated pages. Ballooning increases memory pressure in the guest OS, causing it to invoke its own native memory management algorithms. When memory is plentiful, the guest OS will return memory from its free list. When memory is scarce, it must