RainbowCake: Mitigating Cold-starts in Serverless with Layer-wise Container Caching and Sharing

RainbowCake: Mitigating Cold-starts in Serverless with Layer-wise Container Caching and Sharing

April 27-May 1, 2024 | Hanfei Yu, Rohan Basu Roy, Christian Fontenot, Devesh Tiwari, Jian Li, Hong Zhang, Hao Wang, Seung-Jong Park
RainbowCake is a serverless computing technique that mitigates cold-start problems by using layer-wise container caching and sharing. Cold-starts occur when containers are started, leading to significant latency. Full container caching has been widely used but is outperformed by partial caching and container sharing. However, both approaches have drawbacks: partial caching reduces memory waste but only partially mitigates cold-starts, while container sharing reduces cold-starts but causes excessive memory waste due to over-packed containers. RainbowCake introduces a layer-wise pre-warming and keep-alive technique that effectively mitigates cold-starts with minimal memory waste. It uses structured container layers and sharing-aware modeling to be robust and tolerant to invocation bursts. RainbowCake breaks the container startup process into three stages and manages different container layers individually. It develops a sharing-aware algorithm that makes event-driven layer-wise caching decisions in real-time. Experiments on OpenWhisk clusters with real-world workloads show that RainbowCake reduces 68% function startup latency and 77% memory waste compared to state-of-the-art solutions. RainbowCake is designed to be lightweight and scalable, with a focus on minimizing resource wasting while maintaining low function end-to-end latency. It uses historical invocation information to customize pre-warm and keep-alive strategies tailored to every function. RainbowCake's key contributions include a careful examination of the layer-wise container initialization process, a layer-wise sharing-aware algorithm, and an implementation in OpenWhisk with extensive experiments showing significant improvements in performance.RainbowCake is a serverless computing technique that mitigates cold-start problems by using layer-wise container caching and sharing. Cold-starts occur when containers are started, leading to significant latency. Full container caching has been widely used but is outperformed by partial caching and container sharing. However, both approaches have drawbacks: partial caching reduces memory waste but only partially mitigates cold-starts, while container sharing reduces cold-starts but causes excessive memory waste due to over-packed containers. RainbowCake introduces a layer-wise pre-warming and keep-alive technique that effectively mitigates cold-starts with minimal memory waste. It uses structured container layers and sharing-aware modeling to be robust and tolerant to invocation bursts. RainbowCake breaks the container startup process into three stages and manages different container layers individually. It develops a sharing-aware algorithm that makes event-driven layer-wise caching decisions in real-time. Experiments on OpenWhisk clusters with real-world workloads show that RainbowCake reduces 68% function startup latency and 77% memory waste compared to state-of-the-art solutions. RainbowCake is designed to be lightweight and scalable, with a focus on minimizing resource wasting while maintaining low function end-to-end latency. It uses historical invocation information to customize pre-warm and keep-alive strategies tailored to every function. RainbowCake's key contributions include a careful examination of the layer-wise container initialization process, a layer-wise sharing-aware algorithm, and an implementation in OpenWhisk with extensive experiments showing significant improvements in performance.
Reach us at info@study.space