MAGDI is a structured distillation method that improves reasoning in smaller language models by distilling knowledge from multi-agent interactions. The method represents multi-agent interactions as graphs, augments a base student model with a graph encoder, and distills knowledge using three objective functions: next-token prediction, a contrastive loss between correct and incorrect reasoning, and a graph-based objective to model the interaction structure. Experiments on seven benchmarks show that MAGDI improves the reasoning capabilities of smaller models, outperforming several methods that distill from a single teacher and multiple teachers. MAGDI also demonstrates an order of magnitude higher efficiency over its teachers. The method enhances generalizability to out-of-domain tasks, scales positively with the size and strength of the base student model, and achieves larger improvements when applying self-consistency. MAGDI is effective in transferring multi-agent capabilities into a single student model and improves inference efficiency compared to RECONCILE. The method also boosts self-consistency and outperforms alternative graph modeling approaches. MAGDI's structured distillation method is effective in improving reasoning in smaller models and has potential for further research in modeling multi-agent interactions.MAGDI is a structured distillation method that improves reasoning in smaller language models by distilling knowledge from multi-agent interactions. The method represents multi-agent interactions as graphs, augments a base student model with a graph encoder, and distills knowledge using three objective functions: next-token prediction, a contrastive loss between correct and incorrect reasoning, and a graph-based objective to model the interaction structure. Experiments on seven benchmarks show that MAGDI improves the reasoning capabilities of smaller models, outperforming several methods that distill from a single teacher and multiple teachers. MAGDI also demonstrates an order of magnitude higher efficiency over its teachers. The method enhances generalizability to out-of-domain tasks, scales positively with the size and strength of the base student model, and achieves larger improvements when applying self-consistency. MAGDI is effective in transferring multi-agent capabilities into a single student model and improves inference efficiency compared to RECONCILE. The method also boosts self-consistency and outperforms alternative graph modeling approaches. MAGDI's structured distillation method is effective in improving reasoning in smaller models and has potential for further research in modeling multi-agent interactions.