This paper introduces a novel approach called CommFormer, which addresses the challenge of learning multi-agent communication from a graph modeling perspective. CommFormer conceptualizes the communication architecture among agents as a learnable graph and formulates the problem as determining the communication graph while enabling the architecture parameters to update normally, necessitating a bi-level optimization process. By leveraging continuous relaxation of graph representation and incorporating attention mechanisms, CommFormer enables the concurrent optimization of the communication graph and architectural parameters in an end-to-end manner. Extensive experiments on various cooperative tasks demonstrate the robustness and effectiveness of CommFormer, showing that it consistently outperforms strong baselines and achieves comparable performance to methods allowing unrestricted information sharing among all agents. The approach is particularly effective in scenarios with limited communication capabilities, reducing communication costs and overhead while maintaining high coordination and efficiency.This paper introduces a novel approach called CommFormer, which addresses the challenge of learning multi-agent communication from a graph modeling perspective. CommFormer conceptualizes the communication architecture among agents as a learnable graph and formulates the problem as determining the communication graph while enabling the architecture parameters to update normally, necessitating a bi-level optimization process. By leveraging continuous relaxation of graph representation and incorporating attention mechanisms, CommFormer enables the concurrent optimization of the communication graph and architectural parameters in an end-to-end manner. Extensive experiments on various cooperative tasks demonstrate the robustness and effectiveness of CommFormer, showing that it consistently outperforms strong baselines and achieves comparable performance to methods allowing unrestricted information sharing among all agents. The approach is particularly effective in scenarios with limited communication capabilities, reducing communication costs and overhead while maintaining high coordination and efficiency.