MARG: Multi-Agent Review Generation for Scientific Papers

MARG: Multi-Agent Review Generation for Scientific Papers

8 Jan 2024 | Mike D'Arcy, Tom Hope, Larry Birnbaum, Doug Downey
MARG is a multi-agent approach for generating peer-review feedback for scientific papers. It uses multiple LLM instances (agents) that engage in internal discussion to generate feedback across the full text of papers, overcoming the input length limitations of base LLMs. By specializing agents and incorporating sub-tasks tailored to different comment types (experiments, clarity, impact), MARG improves the helpfulness and specificity of feedback. In a user study, MARG-S, a specialized variant of MARG, generated 3.7 "good" comments per paper, significantly outperforming baseline methods. MARG-S also produced more specific comments than baselines, with 71% of its comments rated as specific. However, MARG-S has high costs and internal communication errors. The method outperforms baselines in automated evaluation and user studies, generating more specific and helpful comments. It is effective in generating actionable feedback for scientific papers, but requires careful prompting and refinement. MARG-S is more expensive than other methods but produces more specific comments. The method is effective in generating actionable feedback for scientific papers, but requires careful prompting and refinement. MARG-S is more expensive than other methods but produces more specific comments. The method is effective in generating actionable feedback for scientific papers, but requires careful prompting and refinement.MARG is a multi-agent approach for generating peer-review feedback for scientific papers. It uses multiple LLM instances (agents) that engage in internal discussion to generate feedback across the full text of papers, overcoming the input length limitations of base LLMs. By specializing agents and incorporating sub-tasks tailored to different comment types (experiments, clarity, impact), MARG improves the helpfulness and specificity of feedback. In a user study, MARG-S, a specialized variant of MARG, generated 3.7 "good" comments per paper, significantly outperforming baseline methods. MARG-S also produced more specific comments than baselines, with 71% of its comments rated as specific. However, MARG-S has high costs and internal communication errors. The method outperforms baselines in automated evaluation and user studies, generating more specific and helpful comments. It is effective in generating actionable feedback for scientific papers, but requires careful prompting and refinement. MARG-S is more expensive than other methods but produces more specific comments. The method is effective in generating actionable feedback for scientific papers, but requires careful prompting and refinement. MARG-S is more expensive than other methods but produces more specific comments. The method is effective in generating actionable feedback for scientific papers, but requires careful prompting and refinement.
Reach us at info@study.space
[slides and audio] MARG%3A Multi-Agent Review Generation for Scientific Papers