The paper introduces HAIChart, a reinforcement learning-based framework designed to generate and refine visualizations for datasets by incorporating user feedback. HAIChart aims to balance the efficiency of human-powered tools with the accuracy of AI-powered tools. It uses a Monte Carlo Graph Search (MCGS) algorithm and a composite reward function to efficiently explore the visualization space and generate high-quality visualizations. The system also includes a visualization hints mechanism to actively incorporate user feedback, refining the visualization generation process iteratively. The top-$k$ visualization hints selection problem is proven to be NP-hard, and an efficient algorithm is designed to address this. Experiments show that HAIChart outperforms both human-powered and AI-powered tools in terms of recall, speed, and effectiveness. The source code and artifacts are available at https://github.com/yxpkent/HAIChart.The paper introduces HAIChart, a reinforcement learning-based framework designed to generate and refine visualizations for datasets by incorporating user feedback. HAIChart aims to balance the efficiency of human-powered tools with the accuracy of AI-powered tools. It uses a Monte Carlo Graph Search (MCGS) algorithm and a composite reward function to efficiently explore the visualization space and generate high-quality visualizations. The system also includes a visualization hints mechanism to actively incorporate user feedback, refining the visualization generation process iteratively. The top-$k$ visualization hints selection problem is proven to be NP-hard, and an efficient algorithm is designed to address this. Experiments show that HAIChart outperforms both human-powered and AI-powered tools in terms of recall, speed, and effectiveness. The source code and artifacts are available at https://github.com/yxpkent/HAIChart.