MetaCoCo is a new few-shot classification benchmark designed to evaluate the impact of spurious correlation shifts in real-world scenarios. The benchmark includes 175,637 images, 155 contexts, and 100 classes, with spurious correlations arising from various contexts. The benchmark is constructed by labeling images with main concepts and contexts, allowing for the creation of spurious-correlation-shift settings by training models in some contexts and testing them in others. A metric based on CLIP is used to quantify and compare the extent of spurious correlations on MetaCoCo and other FSC benchmarks. Extensive experiments on the benchmark evaluate the performance of state-of-the-art methods in few-shot classification, cross-domain shifts, and self-supervised learning. The results show that existing methods degrade significantly in the presence of spurious-correlation shifts. The benchmark is open-sourced to facilitate future research on spurious-correlation shifts in few-shot classification. The code is available at: https://github.com/remiMZ/MetaCoCo-ICLR24.MetaCoCo is a new few-shot classification benchmark designed to evaluate the impact of spurious correlation shifts in real-world scenarios. The benchmark includes 175,637 images, 155 contexts, and 100 classes, with spurious correlations arising from various contexts. The benchmark is constructed by labeling images with main concepts and contexts, allowing for the creation of spurious-correlation-shift settings by training models in some contexts and testing them in others. A metric based on CLIP is used to quantify and compare the extent of spurious correlations on MetaCoCo and other FSC benchmarks. Extensive experiments on the benchmark evaluate the performance of state-of-the-art methods in few-shot classification, cross-domain shifts, and self-supervised learning. The results show that existing methods degrade significantly in the presence of spurious-correlation shifts. The benchmark is open-sourced to facilitate future research on spurious-correlation shifts in few-shot classification. The code is available at: https://github.com/remiMZ/MetaCoCo-ICLR24.