The paper introduces AgentHPO, a novel paradigm that leverages Large Language Models (LLMs) to automate hyperparameter optimization (HPO) across diverse machine learning tasks. AgentHPO consists of two specialized agents: Creator and Executor. The Creator agent processes task information and generates initial hyperparameters (HPs), while the Executor agent trains models, records experimental data, and analyzes outcomes. This human-like optimization process reduces the number of trials, simplifies setup, and enhances interpretability and user trust compared to traditional AutoML methods. Extensive experiments on 12 representative machine learning tasks show that AgentHPO matches or surpasses human performance in terms of accuracy while providing explainable results. The study highlights the effectiveness and adaptability of LLMs in various scenarios, making AgentHPO a promising approach for HPO in modern machine learning.The paper introduces AgentHPO, a novel paradigm that leverages Large Language Models (LLMs) to automate hyperparameter optimization (HPO) across diverse machine learning tasks. AgentHPO consists of two specialized agents: Creator and Executor. The Creator agent processes task information and generates initial hyperparameters (HPs), while the Executor agent trains models, records experimental data, and analyzes outcomes. This human-like optimization process reduces the number of trials, simplifies setup, and enhances interpretability and user trust compared to traditional AutoML methods. Extensive experiments on 12 representative machine learning tasks show that AgentHPO matches or surpasses human performance in terms of accuracy while providing explainable results. The study highlights the effectiveness and adaptability of LLMs in various scenarios, making AgentHPO a promising approach for HPO in modern machine learning.