Test-Time Adaptation for Depth Completion

Test-Time Adaptation for Depth Completion

27 May 2024 | Hyoungseob Park, Anjali Gupta, Alex Wong
This paper proposes ProxyTTA, an online test-time adaptation method for depth completion, which aims to reduce the performance gap between models trained on source data and tested on target data. The method leverages the fact that sparse depth modality is less sensitive to domain shift than image modality. It designs an embedding module trained in the source domain that maps features encoding sparse depth to those encoding both image and sparse depth. During test time, sparse depth features are projected using this map as a proxy for source domain features and are used to train auxiliary parameters (i.e., adaptation layer) to align image and sparse depth features from the target domain to that of the source domain. The method is evaluated on indoor and outdoor scenarios and shows an average improvement of 21.1% over baselines. The method is effective in scenarios where the domain shift is significant, and it can be applied to both supervised and unsupervised settings. The method is implemented with an adaptation layer that is updated while freezing the rest of the network, allowing for low-cost adaptation. The method is also effective in scenarios where the source dataset is not available, and it can be used in more general scenarios than previous methods. The method is the first to introduce test-time adaptation for depth completion and is released with code, models, and dataset benchmarking setup to make development accessible for the research community.This paper proposes ProxyTTA, an online test-time adaptation method for depth completion, which aims to reduce the performance gap between models trained on source data and tested on target data. The method leverages the fact that sparse depth modality is less sensitive to domain shift than image modality. It designs an embedding module trained in the source domain that maps features encoding sparse depth to those encoding both image and sparse depth. During test time, sparse depth features are projected using this map as a proxy for source domain features and are used to train auxiliary parameters (i.e., adaptation layer) to align image and sparse depth features from the target domain to that of the source domain. The method is evaluated on indoor and outdoor scenarios and shows an average improvement of 21.1% over baselines. The method is effective in scenarios where the domain shift is significant, and it can be applied to both supervised and unsupervised settings. The method is implemented with an adaptation layer that is updated while freezing the rest of the network, allowing for low-cost adaptation. The method is also effective in scenarios where the source dataset is not available, and it can be used in more general scenarios than previous methods. The method is the first to introduce test-time adaptation for depth completion and is released with code, models, and dataset benchmarking setup to make development accessible for the research community.
Reach us at info@study.space
[slides and audio] Test- Time Adaptation for Depth Completion