The paper introduces Codex, a GPT language model fine-tuned on publicly available code from GitHub, and evaluates its Python code-writing capabilities. The model is used to power GitHub Copilot and is evaluated on the HumanEval dataset, which measures functional correctness for synthesizing programs from docstrings. Codex solves 28.8% of the problems, significantly outperforming GPT-3 (0%) and GPT-J (11.4%). The authors find that repeated sampling from the model is an effective strategy for producing working solutions, solving 70.2% of problems with 100 samples per problem. The paper also discusses the limitations of Codex, including difficulty with long chains of operations and binding operations to variables. Finally, the authors discuss the broader impacts of deploying powerful code generation technologies, covering safety, security, and economic implications.The paper introduces Codex, a GPT language model fine-tuned on publicly available code from GitHub, and evaluates its Python code-writing capabilities. The model is used to power GitHub Copilot and is evaluated on the HumanEval dataset, which measures functional correctness for synthesizing programs from docstrings. Codex solves 28.8% of the problems, significantly outperforming GPT-3 (0%) and GPT-J (11.4%). The authors find that repeated sampling from the model is an effective strategy for producing working solutions, solving 70.2% of problems with 100 samples per problem. The paper also discusses the limitations of Codex, including difficulty with long chains of operations and binding operations to variables. Finally, the authors discuss the broader impacts of deploying powerful code generation technologies, covering safety, security, and economic implications.