This paper investigates the impact of data contamination on code language models (CLMs) and evaluates countermeasures to mitigate this issue. Data contamination occurs when evaluation datasets have already been used to train the CLMs, potentially leading to unreliable performance assessments. The study systematically examines three countermeasures: using recent data, curating new data, and refactoring existing data. The research collects 2.49 million Python functions from 2018 to 2023, categorizing them as contaminated or cleansed based on their creation time relative to the CLM's training cutoff date. The study finds that CLMs often perform better on recent data, refactored data, and curated datasets compared to contaminated data. However, existing metrics like perplexity are ineffective in distinguishing contaminated and cleansed data. The results suggest that refactoring code structures can improve CLM performance, while semantic refactoring operators like identifier renaming and adding special parameters are more effective in mitigating data contamination. The study also highlights that the popularity of AI programming assistants like Copilot may exacerbate data contamination. Overall, the findings indicate that current countermeasures may not fully address data contamination, and further research is needed to develop more effective solutions.This paper investigates the impact of data contamination on code language models (CLMs) and evaluates countermeasures to mitigate this issue. Data contamination occurs when evaluation datasets have already been used to train the CLMs, potentially leading to unreliable performance assessments. The study systematically examines three countermeasures: using recent data, curating new data, and refactoring existing data. The research collects 2.49 million Python functions from 2018 to 2023, categorizing them as contaminated or cleansed based on their creation time relative to the CLM's training cutoff date. The study finds that CLMs often perform better on recent data, refactored data, and curated datasets compared to contaminated data. However, existing metrics like perplexity are ineffective in distinguishing contaminated and cleansed data. The results suggest that refactoring code structures can improve CLM performance, while semantic refactoring operators like identifier renaming and adding special parameters are more effective in mitigating data contamination. The study also highlights that the popularity of AI programming assistants like Copilot may exacerbate data contamination. Overall, the findings indicate that current countermeasures may not fully address data contamination, and further research is needed to develop more effective solutions.