This paper investigates the impact of in-context examples on the performance of GPT-3, a powerful language model known for its few-shot learning capabilities. The authors find that the choice of in-context examples significantly affects GPT-3's performance, with semantically similar examples leading to better results. To address this, they propose KATE (KNN-Augmented in-Context Example selection), a method that retrieves the nearest neighbors of a test sample from the training set to form the context for GPT-3. This approach consistently outperforms random sampling on various natural language understanding and generation tasks, including sentiment analysis, table-to-text generation, and open-domain question answering. The study also highlights the importance of fine-tuning sentence encoders on task-related datasets, which further enhances the retrieval accuracy and, consequently, the performance of GPT-3. The paper provides insights into how GPT-3's few-shot learning capabilities can be optimized by selecting appropriate in-context examples.This paper investigates the impact of in-context examples on the performance of GPT-3, a powerful language model known for its few-shot learning capabilities. The authors find that the choice of in-context examples significantly affects GPT-3's performance, with semantically similar examples leading to better results. To address this, they propose KATE (KNN-Augmented in-Context Example selection), a method that retrieves the nearest neighbors of a test sample from the training set to form the context for GPT-3. This approach consistently outperforms random sampling on various natural language understanding and generation tasks, including sentiment analysis, table-to-text generation, and open-domain question answering. The study also highlights the importance of fine-tuning sentence encoders on task-related datasets, which further enhances the retrieval accuracy and, consequently, the performance of GPT-3. The paper provides insights into how GPT-3's few-shot learning capabilities can be optimized by selecting appropriate in-context examples.