29 Dec 2017 | Judy Hoffman, Eric Tzeng, Taesung Park, Jun-Yan Zhu, Phillip Isola, Kate Saenko, Alexei A. Efros, Trevor Darrell
CyCADA is a novel cycle-consistent adversarial domain adaptation model that adapts representations at both pixel and feature levels, enforces cycle consistency, and leverages a task loss without requiring aligned image pairs. The model is applicable across various visual recognition and prediction tasks and achieves state-of-the-art results in digit classification and semantic segmentation across domains, including challenging synthetic-to-real scenarios. CyCADA combines feature-level and image-level adversarial domain adaptation methods with cycle-consistent image-to-image translation techniques. It uses a reconstruction (cycle-consistency) loss to preserve local structural information and a semantic loss to enforce semantic consistency. The model is evaluated on multiple adaptation tasks, including digit adaptation, cross-season adaptation in synthetic data, and synthetic-to-real scenarios. Results show that CyCADA significantly improves performance, achieving 82% per-pixel accuracy in the synthetic-to-real scenario, nearly closing the gap to the target-trained model. The model also demonstrates complementary improvements when adapting at both pixel and representation levels, leading to the highest performing model for digit classification tasks. CyCADA is effective in pixel-level semantic segmentation with contemporary FCN architectures and is particularly beneficial for tasks where domain shifts are significant. The model is implemented using a pixel-to-pixel convnet for G, a convnet classifier or FCN for f, and a convnet with binary outputs for D. Experiments show that CyCADA achieves state-of-the-art results in multiple evaluation settings, including cross-season adaptation and synthetic-to-real adaptation. The model is also effective in semantic segmentation tasks, achieving high performance on the Cityscapes dataset and demonstrating the ability to produce realistic domain conversions. The results indicate that CyCADA is highly effective at correcting the most common classes in the dataset and is particularly useful in unsupervised settings where domain shifts are significant.CyCADA is a novel cycle-consistent adversarial domain adaptation model that adapts representations at both pixel and feature levels, enforces cycle consistency, and leverages a task loss without requiring aligned image pairs. The model is applicable across various visual recognition and prediction tasks and achieves state-of-the-art results in digit classification and semantic segmentation across domains, including challenging synthetic-to-real scenarios. CyCADA combines feature-level and image-level adversarial domain adaptation methods with cycle-consistent image-to-image translation techniques. It uses a reconstruction (cycle-consistency) loss to preserve local structural information and a semantic loss to enforce semantic consistency. The model is evaluated on multiple adaptation tasks, including digit adaptation, cross-season adaptation in synthetic data, and synthetic-to-real scenarios. Results show that CyCADA significantly improves performance, achieving 82% per-pixel accuracy in the synthetic-to-real scenario, nearly closing the gap to the target-trained model. The model also demonstrates complementary improvements when adapting at both pixel and representation levels, leading to the highest performing model for digit classification tasks. CyCADA is effective in pixel-level semantic segmentation with contemporary FCN architectures and is particularly beneficial for tasks where domain shifts are significant. The model is implemented using a pixel-to-pixel convnet for G, a convnet classifier or FCN for f, and a convnet with binary outputs for D. Experiments show that CyCADA achieves state-of-the-art results in multiple evaluation settings, including cross-season adaptation and synthetic-to-real adaptation. The model is also effective in semantic segmentation tasks, achieving high performance on the Cityscapes dataset and demonstrating the ability to produce realistic domain conversions. The results indicate that CyCADA is highly effective at correcting the most common classes in the dataset and is particularly useful in unsupervised settings where domain shifts are significant.