CAD-SIGNet: CAD Language Inference from Point Clouds using Layer-wise Sketch Instance Guided Attention

CAD-SIGNet: CAD Language Inference from Point Clouds using Layer-wise Sketch Instance Guided Attention

27 Feb 2024 | Mohammad Sadil Khan, Elona Dupont, Sk Aziz Ali, Kseniya Cherenkova, Anis Kacem, Djamila Aouada
CAD-SIGNet is an end-to-end trainable and auto-regressive architecture designed to recover the design history of a CAD model from a point cloud. The design history is represented as a sequence of sketch-and-extrusion steps. The model learns CAD visual-language representations through layer-wise cross-attention between point cloud and CAD language embeddings. A new Sketch Instance Guided Attention (SGA) module is introduced to reconstruct fine-grained sketch details. CAD-SIGNet not only reconstructs the full design history of a CAD model from a point cloud but also provides multiple plausible design choices, enabling interactive reverse engineering. Extensive experiments on publicly available CAD datasets show that CAD-SIGNet outperforms existing baseline models in two settings: full design history recovery and conditional auto-completion from point clouds. The model uses multi-modal transformer blocks with layer-wise cross-attention between point cloud and CAD language embeddings. The SGA module focuses attention on relevant regions of the point cloud for sketch parameterization. CAD-SIGNet is trained on the DeepCAD dataset and tested on cross-dataset scenarios, including the Fusion360 and CC3D datasets. The model achieves better 3D reconstruction quality and lower Chamfer Distance compared to existing methods. It also supports user-controlled reverse engineering by providing multiple design choices at each step. The auto-regressive nature of CAD-SIGNet allows for generating multiple plausible CAD sequences from a single point cloud. The model is evaluated using metrics such as mean and median Chamfer Distance, Invalidity Ratio, and F1 scores for different primitive types. The results show that CAD-SIGNet outperforms existing methods in terms of reconstruction quality and design choice generation.CAD-SIGNet is an end-to-end trainable and auto-regressive architecture designed to recover the design history of a CAD model from a point cloud. The design history is represented as a sequence of sketch-and-extrusion steps. The model learns CAD visual-language representations through layer-wise cross-attention between point cloud and CAD language embeddings. A new Sketch Instance Guided Attention (SGA) module is introduced to reconstruct fine-grained sketch details. CAD-SIGNet not only reconstructs the full design history of a CAD model from a point cloud but also provides multiple plausible design choices, enabling interactive reverse engineering. Extensive experiments on publicly available CAD datasets show that CAD-SIGNet outperforms existing baseline models in two settings: full design history recovery and conditional auto-completion from point clouds. The model uses multi-modal transformer blocks with layer-wise cross-attention between point cloud and CAD language embeddings. The SGA module focuses attention on relevant regions of the point cloud for sketch parameterization. CAD-SIGNet is trained on the DeepCAD dataset and tested on cross-dataset scenarios, including the Fusion360 and CC3D datasets. The model achieves better 3D reconstruction quality and lower Chamfer Distance compared to existing methods. It also supports user-controlled reverse engineering by providing multiple design choices at each step. The auto-regressive nature of CAD-SIGNet allows for generating multiple plausible CAD sequences from a single point cloud. The model is evaluated using metrics such as mean and median Chamfer Distance, Invalidity Ratio, and F1 scores for different primitive types. The results show that CAD-SIGNet outperforms existing methods in terms of reconstruction quality and design choice generation.
Reach us at info@study.space