This dissertation, titled "Semantic 3D Object Maps for Everyday Manipulation in Human Living Environments," by Radu Bogdan Rusu, focuses on developing advanced perceptual capabilities for personal assistive robots. The primary goal is to enable robots to understand and manipulate objects in human living environments more effectively. The thesis makes two key contributions: 1. **Semantic 3D Object Model Acquisition**: It proposes a novel framework for acquiring Semantic 3D Object Models from Point Cloud Data. This framework includes robust alignment and integration mechanisms, fast segmentation based on local surface characteristics, and reliable object detection, categorization, and reconstruction. The models are semantic in nature, inferring structures meaningful to the robot's tasks, such as doors, handles, supporting planes, and movable objects. 2. **Point Cloud Representations**: It introduces 3D Point Feature Histograms (3D-PFHs) to model the local surface geometry of each point. 3D-PFHs are designed to be fast to compute, robust against pose and sampling density variations, and effective with noisy sensor data. They significantly improve the quality and speed of Semantic 3D Object Model acquisition. The thesis is divided into three parts: - **Part I** covers the Semantic 3D Object Mapping Kernel, including data acquisition techniques, point cloud representations, and the framework for acquiring Semantic 3D Object Models. - **Part II** addresses semantic scene interpretation of indoor environments using machine learning classifiers and parametric shape model fitting. - **Part III** presents applications of comprehensive 3D perception systems on complete robotic platforms, including table cleaning, door and handle identification, and real-time semantic mapping for navigation. The contributions have been fully implemented and evaluated on different robots performing various tasks in different environments, demonstrating robustness and effectiveness in dynamic and cluttered settings.This dissertation, titled "Semantic 3D Object Maps for Everyday Manipulation in Human Living Environments," by Radu Bogdan Rusu, focuses on developing advanced perceptual capabilities for personal assistive robots. The primary goal is to enable robots to understand and manipulate objects in human living environments more effectively. The thesis makes two key contributions: 1. **Semantic 3D Object Model Acquisition**: It proposes a novel framework for acquiring Semantic 3D Object Models from Point Cloud Data. This framework includes robust alignment and integration mechanisms, fast segmentation based on local surface characteristics, and reliable object detection, categorization, and reconstruction. The models are semantic in nature, inferring structures meaningful to the robot's tasks, such as doors, handles, supporting planes, and movable objects. 2. **Point Cloud Representations**: It introduces 3D Point Feature Histograms (3D-PFHs) to model the local surface geometry of each point. 3D-PFHs are designed to be fast to compute, robust against pose and sampling density variations, and effective with noisy sensor data. They significantly improve the quality and speed of Semantic 3D Object Model acquisition. The thesis is divided into three parts: - **Part I** covers the Semantic 3D Object Mapping Kernel, including data acquisition techniques, point cloud representations, and the framework for acquiring Semantic 3D Object Models. - **Part II** addresses semantic scene interpretation of indoor environments using machine learning classifiers and parametric shape model fitting. - **Part III** presents applications of comprehensive 3D perception systems on complete robotic platforms, including table cleaning, door and handle identification, and real-time semantic mapping for navigation. The contributions have been fully implemented and evaluated on different robots performing various tasks in different environments, demonstrating robustness and effectiveness in dynamic and cluttered settings.