Sculpt3D is a novel framework designed to enhance multi-view consistent text-to-3D generation by explicitly incorporating 3D priors from retrieved reference objects without retraining the 2D diffusion model. The method addresses the common issues of inaccurate shapes and inconsistent appearances in 3D generation by using a sparse ray sampling approach to supervise key points that describe the object's overall structure. This allows the 2D diffusion model to maintain its generative quality while being guided by 3D information. Additionally, the method uses a unified image adapter to modulate the output of the 2D diffusion model to align with the correct patterns of the template views, ensuring accurate appearances across different viewpoints. Extensive experiments demonstrate that Sculpt3D significantly improves multi-view consistency while retaining fidelity and diversity in the generated objects. The project page is available at: https://stellarcheng.github.io/Sculpt3D/.Sculpt3D is a novel framework designed to enhance multi-view consistent text-to-3D generation by explicitly incorporating 3D priors from retrieved reference objects without retraining the 2D diffusion model. The method addresses the common issues of inaccurate shapes and inconsistent appearances in 3D generation by using a sparse ray sampling approach to supervise key points that describe the object's overall structure. This allows the 2D diffusion model to maintain its generative quality while being guided by 3D information. Additionally, the method uses a unified image adapter to modulate the output of the 2D diffusion model to align with the correct patterns of the template views, ensuring accurate appearances across different viewpoints. Extensive experiments demonstrate that Sculpt3D significantly improves multi-view consistency while retaining fidelity and diversity in the generated objects. The project page is available at: https://stellarcheng.github.io/Sculpt3D/.