The CodeSearchNet Challenge evaluates the state of semantic code search. Semantic code search involves retrieving relevant code based on natural language queries. This task is challenging because it requires bridging the gap between the technical language used in code and natural language. To evaluate progress in code search, the CodeSearchNet Corpus and Challenge are introduced. The Corpus contains 6 million functions from open-source code in six programming languages, along with 2 million automatically generated query-like natural language annotations. The Challenge includes 99 natural language queries with expert annotations for likely results. The CodeSearchNet Corpus was created by scraping open-source repositories and pairing functions with their documentation. It includes 2 million function-documentation pairs and 4 million functions without documentation. The Corpus was preprocessed to make it more realistic for code search tasks, including truncating documentation, removing short functions, and filtering out functions with names containing "test". The CodeSearchNet Challenge provides realistic queries and expert annotations for likely results. It includes 99 natural language queries paired with likely results for six programming languages. Each query/result pair was labeled by a human expert. Baseline models were created using various neural sequence processing techniques, including bag of words, RNNs, CNNs, and attentional models. These models were evaluated on the dataset. The Challenge aims to encourage further research in semantic code search and will host a competition and leaderboard to track progress. The dataset and challenge are available for researchers to use and extend. The results show that the self-attention-based model performs best, while the neural bag of words model performs well in keyword matching. The challenge highlights the importance of semantic understanding in code search and identifies open challenges, such as improving performance on rare terms and leveraging code semantics.The CodeSearchNet Challenge evaluates the state of semantic code search. Semantic code search involves retrieving relevant code based on natural language queries. This task is challenging because it requires bridging the gap between the technical language used in code and natural language. To evaluate progress in code search, the CodeSearchNet Corpus and Challenge are introduced. The Corpus contains 6 million functions from open-source code in six programming languages, along with 2 million automatically generated query-like natural language annotations. The Challenge includes 99 natural language queries with expert annotations for likely results. The CodeSearchNet Corpus was created by scraping open-source repositories and pairing functions with their documentation. It includes 2 million function-documentation pairs and 4 million functions without documentation. The Corpus was preprocessed to make it more realistic for code search tasks, including truncating documentation, removing short functions, and filtering out functions with names containing "test". The CodeSearchNet Challenge provides realistic queries and expert annotations for likely results. It includes 99 natural language queries paired with likely results for six programming languages. Each query/result pair was labeled by a human expert. Baseline models were created using various neural sequence processing techniques, including bag of words, RNNs, CNNs, and attentional models. These models were evaluated on the dataset. The Challenge aims to encourage further research in semantic code search and will host a competition and leaderboard to track progress. The dataset and challenge are available for researchers to use and extend. The results show that the self-attention-based model performs best, while the neural bag of words model performs well in keyword matching. The challenge highlights the importance of semantic understanding in code search and identifies open challenges, such as improving performance on rare terms and leveraging code semantics.