This paper introduces new information-theoretic tools to analyze the error in meta-learning from sequences, particularly focusing on in-context learning (ICL) with transformers. The tools decompose the error into three components: irreducible error, meta-learning error, and intra-task error. This decomposition unifies previous analyses across various meta-learning challenges and avoids contrived assumptions such as mixing times. The main result characterizes how error decays with the number of training sequences and sequence lengths, providing a linear decay without explicit reliance on stability or mixing assumptions. The paper applies these tools to establish new results about ICL, demonstrating that error can decay linearly in both the number of sequences and sequence lengths. The analysis is extended to a sparse mixture of transformer models, providing a theoretical explanation for how ICL can be achieved with limited data. The results suggest that ICL is possible due to the ability of transformers to learn from a mixture of models, with the error decaying linearly in both the number of documents and document lengths.This paper introduces new information-theoretic tools to analyze the error in meta-learning from sequences, particularly focusing on in-context learning (ICL) with transformers. The tools decompose the error into three components: irreducible error, meta-learning error, and intra-task error. This decomposition unifies previous analyses across various meta-learning challenges and avoids contrived assumptions such as mixing times. The main result characterizes how error decays with the number of training sequences and sequence lengths, providing a linear decay without explicit reliance on stability or mixing assumptions. The paper applies these tools to establish new results about ICL, demonstrating that error can decay linearly in both the number of sequences and sequence lengths. The analysis is extended to a sparse mixture of transformer models, providing a theoretical explanation for how ICL can be achieved with limited data. The results suggest that ICL is possible due to the ability of transformers to learn from a mixture of models, with the error decaying linearly in both the number of documents and document lengths.