Make Every Move Count: LLM-based High-Quality RTL Code Generation Using MCTS

Make Every Move Count: LLM-based High-Quality RTL Code Generation Using MCTS

5 Feb 2024 | Matthew DeLorenzo, Animesh Basak Chowdhury, Vasudev Gohil, Shailja Thakur, Ramesh Karri, Siddharth Garg, Jeyavijayan Rajendran
This paper presents a novel approach to generate high-quality register transfer level (RTL) Verilog code using large language models (LLMs) combined with Monte Carlo tree search (MCTS). Existing LLMs for RTL code generation face challenges such as compilation failures and suboptimal power, performance, and area (PPA) efficiency due to a lack of PPA awareness in conventional transformer decoding algorithms. To address these issues, the authors propose an automated transformer decoding algorithm that integrates MCTS for lookahead, guiding the transformer to produce compilable, functionally correct, and PPA-optimized code. The approach formulates the problem of generating optimal Verilog codes using LLMs as a Markov decision process (MDP) and develops a tree-search-based algorithm to solve the MDP. The MCTS-based approach explores the vast search space of potential token combinations, aiming to find an optimal RTL code based on user requirements while ensuring functional correctness. The algorithm uses a reward function that considers syntactical correctness, functional correctness, and PPA metrics, with the goal of minimizing the area-delay product (ADP). The authors address challenges such as search efficiency and scalability to practical designs by reducing the search space through pruning unnecessary paths and leveraging modularity to reuse optimized codes for smaller modules. The MCTS algorithm is implemented using Python 3.8 on an NVIDIA RTX A5000 GPU machine, and experiments show that the proposed technique consistently generates functionally correct code compared to prompting-only methods and effectively addresses the PPA-unawareness drawback of naive LLMs. For the largest design generated by the state-of-the-art LLM (16-bit adder), the technique achieves a 31.8% improvement in the area-delay product. The main contributions of this work include: (1) the first technique to enhance LLMs for Verilog generation using MCTS, (2) overcoming challenges with search space and scalability to practical designs using domain-specific optimizations, (3) generating functionally correct Verilog codes for various designs such as adders, multipliers, and MAC units, and (4) leveraging MCTS formulation to produce PPA-optimized codes to meet user objectives using LLMs. The results demonstrate that the proposed technique outperforms prior works in terms of accuracy and quality, showing significant improvements in PPA metrics for various designs.This paper presents a novel approach to generate high-quality register transfer level (RTL) Verilog code using large language models (LLMs) combined with Monte Carlo tree search (MCTS). Existing LLMs for RTL code generation face challenges such as compilation failures and suboptimal power, performance, and area (PPA) efficiency due to a lack of PPA awareness in conventional transformer decoding algorithms. To address these issues, the authors propose an automated transformer decoding algorithm that integrates MCTS for lookahead, guiding the transformer to produce compilable, functionally correct, and PPA-optimized code. The approach formulates the problem of generating optimal Verilog codes using LLMs as a Markov decision process (MDP) and develops a tree-search-based algorithm to solve the MDP. The MCTS-based approach explores the vast search space of potential token combinations, aiming to find an optimal RTL code based on user requirements while ensuring functional correctness. The algorithm uses a reward function that considers syntactical correctness, functional correctness, and PPA metrics, with the goal of minimizing the area-delay product (ADP). The authors address challenges such as search efficiency and scalability to practical designs by reducing the search space through pruning unnecessary paths and leveraging modularity to reuse optimized codes for smaller modules. The MCTS algorithm is implemented using Python 3.8 on an NVIDIA RTX A5000 GPU machine, and experiments show that the proposed technique consistently generates functionally correct code compared to prompting-only methods and effectively addresses the PPA-unawareness drawback of naive LLMs. For the largest design generated by the state-of-the-art LLM (16-bit adder), the technique achieves a 31.8% improvement in the area-delay product. The main contributions of this work include: (1) the first technique to enhance LLMs for Verilog generation using MCTS, (2) overcoming challenges with search space and scalability to practical designs using domain-specific optimizations, (3) generating functionally correct Verilog codes for various designs such as adders, multipliers, and MAC units, and (4) leveraging MCTS formulation to produce PPA-optimized codes to meet user objectives using LLMs. The results demonstrate that the proposed technique outperforms prior works in terms of accuracy and quality, showing significant improvements in PPA metrics for various designs.
Reach us at info@study.space
Understanding Make Every Move Count%3A LLM-based High-Quality RTL Code Generation Using MCTS