QUERY2CAD: GENERATING CAD MODELS USING NATURAL LANGUAGE QUERIES

QUERY2CAD: GENERATING CAD MODELS USING NATURAL LANGUAGE QUERIES

June 5, 2024 | Akshay Badagabettu, Sai Sravan Yarlagadda, Amir Barati Farimani
Query2CAD is a novel framework designed to generate 3D CAD models using natural language queries. The system leverages large language models (LLMs) to generate executable Python macros that can be executed in FreeCAD software to create CAD models. The framework includes a self-refinement loop that iteratively refines the generated models based on feedback from a BLIP2 caption model or human-in-the-loop feedback. The system operates without supervised data or additional training, using the LLM as both a generator and a refiner. The authors evaluated the framework using a dataset of 57 user queries of varying difficulty levels (easy, medium, and hard). Using GPT-4 Turbo as the language model, the system achieved a success rate of 53.6% on the first attempt, which increased by 23.1% with subsequent refinements. The most significant improvement was observed in the first iteration of refinement. The study also found that human feedback was more effective than the BLIP2 caption model in refining the models. The authors open-sourced their data, model, and code.Query2CAD is a novel framework designed to generate 3D CAD models using natural language queries. The system leverages large language models (LLMs) to generate executable Python macros that can be executed in FreeCAD software to create CAD models. The framework includes a self-refinement loop that iteratively refines the generated models based on feedback from a BLIP2 caption model or human-in-the-loop feedback. The system operates without supervised data or additional training, using the LLM as both a generator and a refiner. The authors evaluated the framework using a dataset of 57 user queries of varying difficulty levels (easy, medium, and hard). Using GPT-4 Turbo as the language model, the system achieved a success rate of 53.6% on the first attempt, which increased by 23.1% with subsequent refinements. The most significant improvement was observed in the first iteration of refinement. The study also found that human feedback was more effective than the BLIP2 caption model in refining the models. The authors open-sourced their data, model, and code.
Reach us at info@study.space
[slides] Query2CAD%3A Generating CAD models using natural language queries | StudySpace