**Abstract:** Large language model (LLM) agents have shown significant potential in leveraging external tools and knowledge to enhance accuracy and reduce hallucinations. However, developing effective prompting techniques to guide LLM agents in using these tools is a challenging and labor-intensive task. To address this, the authors introduce AVaTAR, an automated framework that optimizes LLM agents to effectively use provided tools and improve performance on given tasks. During optimization, a comparator module is designed to iteratively provide insightful and holistic prompts to the LLM agent by reasoning between positive and negative examples sampled from training data. AVaTAR is evaluated on four complex multimodal retrieval datasets, demonstrating consistent outperformance over state-of-the-art approaches across all tasks and strong generalization ability, achieving an average relative improvement of 14% on the Hit@1 metric. **Introduction:** LLM agents have demonstrated remarkable capabilities in reasoning and planning, but effective prompting techniques to guide them through multi-stage problem-solving procedures remain challenging. Current approaches often rely on complex human-designed "mega-prompts," which can be brittle and suboptimal. AVaTAR addresses these challenges by using a comparator module to generate holistic instructions, optimizing the actor LLM for better tool utilization and task performance. **Problem Formulation:** The problem is formulated as a multi-step procedure involving problem decomposition, tool-assisted subproblem solving, and synthesis and response formulation. AVaTAR aims to improve the agent's ability to handle complex tasks and generalize to novel scenarios. **Our Method:** AVaTAR consists of two main components: an actor LLM and a comparator LLM. The comparator module generates holistic instructions by contrasting positive and negative examples, identifying systematic flaws, and providing general improvements. The actor LLM then updates its actions based on these instructions, improving its performance over time. **Experiments:** Experiments on four retrieval datasets (STARK, FLICKR30K-ENTITIES, AMAZON, MAG) show that AVaTAR significantly outperforms state-of-the-art methods, achieving substantial improvements in metrics such as Hit@1 and MRR. The comparator module is crucial for identifying and addressing systemic flaws, and the memory bank helps the actor learn from past experiences, enhancing its performance and generalization. **Conclusion:** AVaTAR is a novel framework that automates the optimization of LLM agents for enhanced tool utilization in complex retrieval tasks. The comparator module's ability to identify and address systemic flaws through contrastive reasoning is key to its success. Future work could explore extending this methodology to other agent tasks and more dynamic environments.**Abstract:** Large language model (LLM) agents have shown significant potential in leveraging external tools and knowledge to enhance accuracy and reduce hallucinations. However, developing effective prompting techniques to guide LLM agents in using these tools is a challenging and labor-intensive task. To address this, the authors introduce AVaTAR, an automated framework that optimizes LLM agents to effectively use provided tools and improve performance on given tasks. During optimization, a comparator module is designed to iteratively provide insightful and holistic prompts to the LLM agent by reasoning between positive and negative examples sampled from training data. AVaTAR is evaluated on four complex multimodal retrieval datasets, demonstrating consistent outperformance over state-of-the-art approaches across all tasks and strong generalization ability, achieving an average relative improvement of 14% on the Hit@1 metric. **Introduction:** LLM agents have demonstrated remarkable capabilities in reasoning and planning, but effective prompting techniques to guide them through multi-stage problem-solving procedures remain challenging. Current approaches often rely on complex human-designed "mega-prompts," which can be brittle and suboptimal. AVaTAR addresses these challenges by using a comparator module to generate holistic instructions, optimizing the actor LLM for better tool utilization and task performance. **Problem Formulation:** The problem is formulated as a multi-step procedure involving problem decomposition, tool-assisted subproblem solving, and synthesis and response formulation. AVaTAR aims to improve the agent's ability to handle complex tasks and generalize to novel scenarios. **Our Method:** AVaTAR consists of two main components: an actor LLM and a comparator LLM. The comparator module generates holistic instructions by contrasting positive and negative examples, identifying systematic flaws, and providing general improvements. The actor LLM then updates its actions based on these instructions, improving its performance over time. **Experiments:** Experiments on four retrieval datasets (STARK, FLICKR30K-ENTITIES, AMAZON, MAG) show that AVaTAR significantly outperforms state-of-the-art methods, achieving substantial improvements in metrics such as Hit@1 and MRR. The comparator module is crucial for identifying and addressing systemic flaws, and the memory bank helps the actor learn from past experiences, enhancing its performance and generalization. **Conclusion:** AVaTAR is a novel framework that automates the optimization of LLM agents for enhanced tool utilization in complex retrieval tasks. The comparator module's ability to identify and address systemic flaws through contrastive reasoning is key to its success. Future work could explore extending this methodology to other agent tasks and more dynamic environments.