Tx-LLM: A Large Language Model for Therapeutics

Tx-LLM: A Large Language Model for Therapeutics

10 Jun 2024 | Juan Manuel Zambrano Chaves*,1, Eric Wang*,2, Tao Tu2, Eshit Dhaval Vaishnav, Byron Lee, S. Sara Mahdavi2, Christopher Semturs1, David Fleet2, Vivek Natarajan†,1 and Shekoofeh Azizi†,1,2
Tx-LLM is a large language model (LLM) designed to accelerate the therapeutics development process by encoding knowledge about diverse therapeutic modalities. Trained on 709 datasets from the Therapeutics Data Commons (TDC), Tx-LLM processes a wide range of chemical and biological entities (small molecules, proteins, nucleic acids, cell lines, diseases) alongside free-text, achieving competitive or superior performance on 43 out of 66 tasks. The model excels in tasks combining molecular representations with text, likely due to context learned during pretraining. Positive transfer between tasks involving different drug types is observed, and ablation studies highlight the impact of model size, domain fine-tuning, and prompting strategies. Tx-LLM shows promise as an end-to-end tool for therapeutic development, covering tasks from target discovery to clinical trial approval. The work demonstrates the potential of LLMs in encoding biochemical knowledge and their role in enhancing various aspects of therapeutic development.Tx-LLM is a large language model (LLM) designed to accelerate the therapeutics development process by encoding knowledge about diverse therapeutic modalities. Trained on 709 datasets from the Therapeutics Data Commons (TDC), Tx-LLM processes a wide range of chemical and biological entities (small molecules, proteins, nucleic acids, cell lines, diseases) alongside free-text, achieving competitive or superior performance on 43 out of 66 tasks. The model excels in tasks combining molecular representations with text, likely due to context learned during pretraining. Positive transfer between tasks involving different drug types is observed, and ablation studies highlight the impact of model size, domain fine-tuning, and prompting strategies. Tx-LLM shows promise as an end-to-end tool for therapeutic development, covering tasks from target discovery to clinical trial approval. The work demonstrates the potential of LLMs in encoding biochemical knowledge and their role in enhancing various aspects of therapeutic development.
Reach us at info@study.space
[slides] Tx-LLM%3A A Large Language Model for Therapeutics | StudySpace