This paper investigates how to select effective in-context examples for GPT-3 to improve its few-shot learning performance. The authors propose a retrieval-based method that selects examples semantically similar to the test sample, which outperforms random sampling. They evaluate their approach on several natural language understanding and generation tasks, including sentiment analysis, table-to-text generation, and open-domain question answering. The results show that the retrieval-based method significantly improves performance, especially on tasks like table-to-text generation (41.9% improvement on the ToTTo dataset) and open-domain question answering (45.5% improvement on the NQ dataset). The authors also find that fine-tuning sentence encoders on task-related datasets enhances the retrieval effectiveness. They further analyze the impact of the number of in-context examples, the size of the training set, and the order of examples on the performance of their method. The results indicate that the retrieval module complements GPT-3's few-shot learning capabilities, and that the choice of sentence encoder and the number of examples can significantly affect performance. The study provides insights into how to better select in-context examples for GPT-3 and other large pre-trained language models.This paper investigates how to select effective in-context examples for GPT-3 to improve its few-shot learning performance. The authors propose a retrieval-based method that selects examples semantically similar to the test sample, which outperforms random sampling. They evaluate their approach on several natural language understanding and generation tasks, including sentiment analysis, table-to-text generation, and open-domain question answering. The results show that the retrieval-based method significantly improves performance, especially on tasks like table-to-text generation (41.9% improvement on the ToTTo dataset) and open-domain question answering (45.5% improvement on the NQ dataset). The authors also find that fine-tuning sentence encoders on task-related datasets enhances the retrieval effectiveness. They further analyze the impact of the number of in-context examples, the size of the training set, and the order of examples on the performance of their method. The results indicate that the retrieval module complements GPT-3's few-shot learning capabilities, and that the choice of sentence encoder and the number of examples can significantly affect performance. The study provides insights into how to better select in-context examples for GPT-3 and other large pre-trained language models.