This paper introduces MAC, an efficient and effective online adaptation framework for large language models (LLMs) with strong knowledge retention. MAC compresses new documents into compact modulations stored in a memory bank, allowing the model to retain knowledge from previous documents. The framework uses amortization-based meta-learning to learn how to extract and aggregate relevant information from the memory bank when answering questions. This approach enables the model to adapt without requiring further gradient updates, making it highly efficient in terms of both time and memory. MAC outperforms existing online adaptation methods in multiple aspects, including online adaptation performance, time, and memory efficiency. It also shows significant effectiveness in retaining learned knowledge compared to other online finetuning baselines. MAC can be combined with retrieval augmented generation (RAG) to further improve the performance of retrieved documents. The framework is tested on multiple datasets and architectures, demonstrating strong results. For instance, MAC improves performance on StreamingQA and SQuAD-Seq datasets compared to prior work. Additionally, MAC is efficient in terms of adaptation time, training, and inference memory usage. The paper also discusses the efficiency of MAC in terms of memory usage, showing that it can significantly reduce memory consumption during training and inference. Techniques such as backpropagation dropout and hierarchical modulation aggregation are proposed to achieve this efficiency. These techniques allow MAC to handle large memory banks without excessive memory usage, making it suitable for real-world applications. MAC is evaluated on various datasets, including StreamingQA, SQuAD, and ArchivalQA. The results show that MAC outperforms other online learning techniques in terms of performance, adaptation time, and memory efficiency. The framework is also effective in retaining knowledge from previously learned documents, which is crucial for online learning. The paper concludes that MAC is a promising approach for online adaptation of LLMs, with potential applications in various real-world scenarios.This paper introduces MAC, an efficient and effective online adaptation framework for large language models (LLMs) with strong knowledge retention. MAC compresses new documents into compact modulations stored in a memory bank, allowing the model to retain knowledge from previous documents. The framework uses amortization-based meta-learning to learn how to extract and aggregate relevant information from the memory bank when answering questions. This approach enables the model to adapt without requiring further gradient updates, making it highly efficient in terms of both time and memory. MAC outperforms existing online adaptation methods in multiple aspects, including online adaptation performance, time, and memory efficiency. It also shows significant effectiveness in retaining learned knowledge compared to other online finetuning baselines. MAC can be combined with retrieval augmented generation (RAG) to further improve the performance of retrieved documents. The framework is tested on multiple datasets and architectures, demonstrating strong results. For instance, MAC improves performance on StreamingQA and SQuAD-Seq datasets compared to prior work. Additionally, MAC is efficient in terms of adaptation time, training, and inference memory usage. The paper also discusses the efficiency of MAC in terms of memory usage, showing that it can significantly reduce memory consumption during training and inference. Techniques such as backpropagation dropout and hierarchical modulation aggregation are proposed to achieve this efficiency. These techniques allow MAC to handle large memory banks without excessive memory usage, making it suitable for real-world applications. MAC is evaluated on various datasets, including StreamingQA, SQuAD, and ArchivalQA. The results show that MAC outperforms other online learning techniques in terms of performance, adaptation time, and memory efficiency. The framework is also effective in retaining knowledge from previously learned documents, which is crucial for online learning. The paper concludes that MAC is a promising approach for online adaptation of LLMs, with potential applications in various real-world scenarios.