This paper introduces AgentHPO, a novel approach for hyperparameter optimization (HPO) that leverages large language models (LLMs) to automate the process across diverse machine learning tasks. Traditional HPO methods are labor-intensive, require expert knowledge, and often lack interpretability. AgentHPO addresses these challenges by utilizing LLM-powered autonomous agents, specifically a Creator and Executor agent, to streamline the optimization process. The Creator agent interprets user-provided task details and generates initial hyperparameter configurations, while the Executor agent conducts experiments, analyzes results, and iteratively refines the hyperparameters based on historical data. This human-like optimization process significantly reduces the number of required trials, simplifies setup, and enhances interpretability and user trust. Extensive experiments on 12 representative machine learning tasks demonstrate that AgentHPO not only matches but often surpasses the performance of human experts while providing explainable results. The framework's ability to adapt to various scenarios and its efficient trial process make it a promising solution for automating HPO. The study also highlights the effectiveness of LLMs in optimizing tasks through strategic exploration and iterative refinement. AgentHPO's approach offers a more efficient and interpretable alternative to traditional AutoML methods, reducing the reliance on expert knowledge and improving the accessibility of HPO for non-experts. The results show that AgentHPO achieves high performance with minimal trials, demonstrating its potential to significantly reduce the time and computational resources required for HPO while increasing the likelihood of achieving near-optimal solutions.This paper introduces AgentHPO, a novel approach for hyperparameter optimization (HPO) that leverages large language models (LLMs) to automate the process across diverse machine learning tasks. Traditional HPO methods are labor-intensive, require expert knowledge, and often lack interpretability. AgentHPO addresses these challenges by utilizing LLM-powered autonomous agents, specifically a Creator and Executor agent, to streamline the optimization process. The Creator agent interprets user-provided task details and generates initial hyperparameter configurations, while the Executor agent conducts experiments, analyzes results, and iteratively refines the hyperparameters based on historical data. This human-like optimization process significantly reduces the number of required trials, simplifies setup, and enhances interpretability and user trust. Extensive experiments on 12 representative machine learning tasks demonstrate that AgentHPO not only matches but often surpasses the performance of human experts while providing explainable results. The framework's ability to adapt to various scenarios and its efficient trial process make it a promising solution for automating HPO. The study also highlights the effectiveness of LLMs in optimizing tasks through strategic exploration and iterative refinement. AgentHPO's approach offers a more efficient and interpretable alternative to traditional AutoML methods, reducing the reliance on expert knowledge and improving the accessibility of HPO for non-experts. The results show that AgentHPO achieves high performance with minimal trials, demonstrating its potential to significantly reduce the time and computational resources required for HPO while increasing the likelihood of achieving near-optimal solutions.