BioMedLM is a 2.7 billion parameter language model trained exclusively on PubMed abstracts and full articles. It is a GPT-style autoregressive model with a biomedical domain-specific tokenizer. BioMedLM can produce strong results on multiple-choice biomedical question-answering tasks, achieving 57.3% on MedMCQA (dev) and 69.0% on the MMLU Medical Genetics exam. It can also be fine-tuned to answer medical questions, demonstrating that smaller models can serve as transparent, privacy-preserving, economical, and environmentally friendly foundations for biomedical NLP applications. BioMedLM is available on the Hugging Face Hub. The model is designed to address the drawbacks of large, closed models, such as high costs, data privacy issues, and limited flexibility. It is smaller and more accessible, allowing for fine-tuning on a single GPU and inference on a laptop. Its training data is fully documented, and the model is open-source, enabling researchers to fine-tune it as needed. BioMedLM outperforms larger models on several biomedical tasks, including MedMCQA, MedQA, MMLU, PubMedQA, and BioASQ. It also performs well on free-response question-answering tasks, generating multi-sentence answers to medical questions. BioMedLM is a promising solution for biomedical NLP tasks, offering a balance between performance and accessibility.BioMedLM is a 2.7 billion parameter language model trained exclusively on PubMed abstracts and full articles. It is a GPT-style autoregressive model with a biomedical domain-specific tokenizer. BioMedLM can produce strong results on multiple-choice biomedical question-answering tasks, achieving 57.3% on MedMCQA (dev) and 69.0% on the MMLU Medical Genetics exam. It can also be fine-tuned to answer medical questions, demonstrating that smaller models can serve as transparent, privacy-preserving, economical, and environmentally friendly foundations for biomedical NLP applications. BioMedLM is available on the Hugging Face Hub. The model is designed to address the drawbacks of large, closed models, such as high costs, data privacy issues, and limited flexibility. It is smaller and more accessible, allowing for fine-tuning on a single GPU and inference on a laptop. Its training data is fully documented, and the model is open-source, enabling researchers to fine-tune it as needed. BioMedLM outperforms larger models on several biomedical tasks, including MedMCQA, MedQA, MMLU, PubMedQA, and BioASQ. It also performs well on free-response question-answering tasks, generating multi-sentence answers to medical questions. BioMedLM is a promising solution for biomedical NLP tasks, offering a balance between performance and accessibility.