This paper presents the aggregate power usage characteristics of large-scale server workloads over six months, analyzing how power consumption varies across different levels of aggregation (racks, clusters, and datacenters). The study finds that there is a significant gap between the actual and theoretical peak power usage of large-scale systems, with the gap growing to nearly 40% at the datacenter level. This headroom can be used to deploy additional computing equipment within the same power budget with minimal risk of exceeding it. The paper also evaluates the potential of power management schemes to reduce peak power and energy usage, finding that opportunities for savings are significant, especially at the cluster level. It argues that systems should be power efficient across the entire activity range, not just at peak performance levels. The study highlights the importance of accurate power modeling and monitoring for effective power provisioning. It shows that power capping techniques can help prevent over-subscription and provide additional power savings. CPU voltage/frequency scaling is found to be moderately effective at reducing peak power consumption when applied to large groups of machines. The paper also discusses the benefits of mixing diverse workloads at the datacenter level, which can help smooth out power peaks and reduce the overall power budget. The study concludes that power provisioning strategies should consider the actual power draw of machines, not just nameplate ratings. It emphasizes the need for dynamic power management to ensure that power is used efficiently across the entire activity range. The paper also highlights the importance of power efficiency in computing systems, arguing that systems should be designed to consume less power at lower activity levels, thereby reducing overall energy consumption. The findings suggest that power provisioning strategies should be based on real-world power usage patterns and not just theoretical maximums.This paper presents the aggregate power usage characteristics of large-scale server workloads over six months, analyzing how power consumption varies across different levels of aggregation (racks, clusters, and datacenters). The study finds that there is a significant gap between the actual and theoretical peak power usage of large-scale systems, with the gap growing to nearly 40% at the datacenter level. This headroom can be used to deploy additional computing equipment within the same power budget with minimal risk of exceeding it. The paper also evaluates the potential of power management schemes to reduce peak power and energy usage, finding that opportunities for savings are significant, especially at the cluster level. It argues that systems should be power efficient across the entire activity range, not just at peak performance levels. The study highlights the importance of accurate power modeling and monitoring for effective power provisioning. It shows that power capping techniques can help prevent over-subscription and provide additional power savings. CPU voltage/frequency scaling is found to be moderately effective at reducing peak power consumption when applied to large groups of machines. The paper also discusses the benefits of mixing diverse workloads at the datacenter level, which can help smooth out power peaks and reduce the overall power budget. The study concludes that power provisioning strategies should consider the actual power draw of machines, not just nameplate ratings. It emphasizes the need for dynamic power management to ensure that power is used efficiently across the entire activity range. The paper also highlights the importance of power efficiency in computing systems, arguing that systems should be designed to consume less power at lower activity levels, thereby reducing overall energy consumption. The findings suggest that power provisioning strategies should be based on real-world power usage patterns and not just theoretical maximums.