A sanity check for AI-generated image detection is conducted to assess whether the task has been solved. The paper introduces the Chameleon dataset, which contains AI-generated images that are challenging for human perception. Existing detectors, when evaluated on this dataset, often misclassify AI-generated images as real. To address this, the authors propose AIDE, a detector that combines low-level pixel statistics and high-level semantic features. AIDE uses DCT scoring to capture low-level artifacts and CLIP to extract high-level semantics. It achieves improvements over state-of-the-art methods on benchmarks like AIGCDetectBenchmark and GenImage. However, the results on Chameleon show that detecting AI-generated images remains challenging. The paper highlights the need for more realistic benchmarks and emphasizes the importance of considering both low-level and high-level features for effective detection. The dataset, code, and pre-trained models are available for further research.A sanity check for AI-generated image detection is conducted to assess whether the task has been solved. The paper introduces the Chameleon dataset, which contains AI-generated images that are challenging for human perception. Existing detectors, when evaluated on this dataset, often misclassify AI-generated images as real. To address this, the authors propose AIDE, a detector that combines low-level pixel statistics and high-level semantic features. AIDE uses DCT scoring to capture low-level artifacts and CLIP to extract high-level semantics. It achieves improvements over state-of-the-art methods on benchmarks like AIGCDetectBenchmark and GenImage. However, the results on Chameleon show that detecting AI-generated images remains challenging. The paper highlights the need for more realistic benchmarks and emphasizes the importance of considering both low-level and high-level features for effective detection. The dataset, code, and pre-trained models are available for further research.