The paper introduces Memory of Amortized Contexts (MAC), an efficient and effective online adaptation framework for large language models (LLMs). MAC addresses the challenge of keeping LLMs up-to-date with new information by compressing and extracting relevant knowledge from new documents into compact modulations stored in a memory bank. During question answering tasks, the model attends to and extracts relevant knowledge from this memory bank. MAC utilizes amortization-based meta-learning to learn informative modulations efficiently, without requiring gradient updates during test time. The model can adapt to new documents without retraining, making it suitable for real-world applications. Experiments demonstrate MAC's superior performance in online adaptation, memory efficiency, and knowledge retention compared to other methods. MAC also shows promise when combined with retrieval augmented generation (RAG) techniques, further improving the selection quality of retrieved documents. The paper provides detailed experimental results and discusses future directions, including extending MAC to handle out-of-distribution datasets and exploring privacy concerns in memory bank usage.The paper introduces Memory of Amortized Contexts (MAC), an efficient and effective online adaptation framework for large language models (LLMs). MAC addresses the challenge of keeping LLMs up-to-date with new information by compressing and extracting relevant knowledge from new documents into compact modulations stored in a memory bank. During question answering tasks, the model attends to and extracts relevant knowledge from this memory bank. MAC utilizes amortization-based meta-learning to learn informative modulations efficiently, without requiring gradient updates during test time. The model can adapt to new documents without retraining, making it suitable for real-world applications. Experiments demonstrate MAC's superior performance in online adaptation, memory efficiency, and knowledge retention compared to other methods. MAC also shows promise when combined with retrieval augmented generation (RAG) techniques, further improving the selection quality of retrieved documents. The paper provides detailed experimental results and discusses future directions, including extending MAC to handle out-of-distribution datasets and exploring privacy concerns in memory bank usage.