The paper "Clio: Real-time Task-Driven Open-Set 3D Scene Graphs" addresses the challenge of creating task-driven 3D scene graphs for robots, focusing on real-time construction and open-set semantic understanding. The authors propose a novel approach, Clio, which integrates task-driven clustering into a real-time system. Key contributions include: 1. **Task-Driven 3D Scene Understanding Problem**: The paper formulates the problem of task-driven 3D scene understanding, where a robot is given a list of natural language tasks and must select the appropriate granularity and subset of objects and scene structure to include in its map. 2. **Information Bottleneck (IB) Framework**: The problem is formulated using the Information Bottleneck (IB) framework, which aims to compress the original signal while preserving task-relevant information. The Agglomerative IB algorithm is applied to cluster 3D objects into task-relevant objects and regions. 3. **Real-Time System Clio**: Clio is a real-time system that constructs a hierarchical 3D scene graph of the environment as the robot explores it. It integrates the task-driven clustering algorithm and can run on a laptop carried by a robot like Spot. 4. **Experimental Evaluation**: Extensive experiments on various datasets ( Replica, Office, Apartment, Cubicle, and a large-scale building) demonstrate that Clio constructs more parsimonious and useful map representations, performs well in closed-set settings, and supports task execution on real robots. 5. **Limitations**: The approach has limitations, such as vulnerability to prompt tuning and the need for more grounded ways to combine semantic descriptions. The paper highlights the importance of task-driven mapping for robots, showing that it can improve the accuracy of task execution by limiting the map to relevant semantic concepts.The paper "Clio: Real-time Task-Driven Open-Set 3D Scene Graphs" addresses the challenge of creating task-driven 3D scene graphs for robots, focusing on real-time construction and open-set semantic understanding. The authors propose a novel approach, Clio, which integrates task-driven clustering into a real-time system. Key contributions include: 1. **Task-Driven 3D Scene Understanding Problem**: The paper formulates the problem of task-driven 3D scene understanding, where a robot is given a list of natural language tasks and must select the appropriate granularity and subset of objects and scene structure to include in its map. 2. **Information Bottleneck (IB) Framework**: The problem is formulated using the Information Bottleneck (IB) framework, which aims to compress the original signal while preserving task-relevant information. The Agglomerative IB algorithm is applied to cluster 3D objects into task-relevant objects and regions. 3. **Real-Time System Clio**: Clio is a real-time system that constructs a hierarchical 3D scene graph of the environment as the robot explores it. It integrates the task-driven clustering algorithm and can run on a laptop carried by a robot like Spot. 4. **Experimental Evaluation**: Extensive experiments on various datasets ( Replica, Office, Apartment, Cubicle, and a large-scale building) demonstrate that Clio constructs more parsimonious and useful map representations, performs well in closed-set settings, and supports task execution on real robots. 5. **Limitations**: The approach has limitations, such as vulnerability to prompt tuning and the need for more grounded ways to combine semantic descriptions. The paper highlights the importance of task-driven mapping for robots, showing that it can improve the accuracy of task execution by limiting the map to relevant semantic concepts.