11 May 2025 | Kai Mei, Xi Zhu, Wujiang Xu, Mingyu Jin, Wenyue Hua, Zelong Li, Shuyuan Xu, Ruosong Ye, Yingqiang Ge, Yongfeng Zhang
AIOS is an LLM-based agent operating system designed to address challenges in deploying intelligent agents, particularly in resource management. Traditional agent frameworks allow unrestricted access to LLM and tool resources, leading to inefficient or harmful resource allocation. AIOS introduces a novel architecture that isolates agent applications and resources, such as LLMs and tools, into an AIOS kernel, providing fundamental services like scheduling, context management, memory management, storage management, and access control. The AIOS SDK offers a comprehensive set of APIs for developers to utilize AIOS kernel functionalities without dealing with complex kernel details. Experimental results show that AIOS can achieve up to 2.1× faster execution for agents built with various frameworks.
The AIOS architecture consists of three layers: application, kernel, and hardware. The application layer uses the AIOS SDK to request system resources, while the kernel layer manages core functionalities and resources for agent applications. The hardware layer controls physical computing resources and devices. The AIOS kernel includes modules such as the LLM core, scheduler, context manager, memory manager, storage manager, tool manager, and access manager, each playing a critical role in resource management, scheduling, and security. The LLM core treats each LLM instance as a dedicated processing unit, while the scheduler manages system calls using algorithms like FIFO and RR. The context manager handles context interruptions and restoration, the memory manager manages agent interaction histories, and the storage manager handles persistent data storage. The tool manager resolves tool call conflicts, and the access manager enforces access control.
AIOS has been evaluated on various benchmarks, showing that it maintains or enhances agent performance while significantly improving execution speed and system throughput. It demonstrates scalability under increasing concurrent agent loads, making it efficient for handling high-demand scenarios. The architecture is designed to support diverse agent frameworks and provides a flexible environment for developing and deploying LLM-based agents. AIOS aims to refine and expand the architecture to meet evolving requirements for agent development and deployment.AIOS is an LLM-based agent operating system designed to address challenges in deploying intelligent agents, particularly in resource management. Traditional agent frameworks allow unrestricted access to LLM and tool resources, leading to inefficient or harmful resource allocation. AIOS introduces a novel architecture that isolates agent applications and resources, such as LLMs and tools, into an AIOS kernel, providing fundamental services like scheduling, context management, memory management, storage management, and access control. The AIOS SDK offers a comprehensive set of APIs for developers to utilize AIOS kernel functionalities without dealing with complex kernel details. Experimental results show that AIOS can achieve up to 2.1× faster execution for agents built with various frameworks.
The AIOS architecture consists of three layers: application, kernel, and hardware. The application layer uses the AIOS SDK to request system resources, while the kernel layer manages core functionalities and resources for agent applications. The hardware layer controls physical computing resources and devices. The AIOS kernel includes modules such as the LLM core, scheduler, context manager, memory manager, storage manager, tool manager, and access manager, each playing a critical role in resource management, scheduling, and security. The LLM core treats each LLM instance as a dedicated processing unit, while the scheduler manages system calls using algorithms like FIFO and RR. The context manager handles context interruptions and restoration, the memory manager manages agent interaction histories, and the storage manager handles persistent data storage. The tool manager resolves tool call conflicts, and the access manager enforces access control.
AIOS has been evaluated on various benchmarks, showing that it maintains or enhances agent performance while significantly improving execution speed and system throughput. It demonstrates scalability under increasing concurrent agent loads, making it efficient for handling high-demand scenarios. The architecture is designed to support diverse agent frameworks and provides a flexible environment for developing and deploying LLM-based agents. AIOS aims to refine and expand the architecture to meet evolving requirements for agent development and deployment.