OmniPred is a framework that trains language models as universal end-to-end regressors for numerical prediction using textual representations of mathematical parameters and values. The framework leverages data from Google Vizier, a large proprietary blackbox optimization database, to demonstrate that language models can achieve high precision in numerical regression, even when trained on diverse tasks. The model uses a 200M parameter T5 encoder-decoder and represents inputs and outputs in text-based formats, allowing for flexible and scalable regression across various input spaces and objectives. The model is trained on multi-task data and can be fine-tuned online for new tasks, improving performance on unseen studies. The framework's ability to handle diverse input spaces and objectives makes it a promising tool for general-purpose regression. The research highlights the potential of language models in experimental design and other domains requiring precise numerical predictions. The study also addresses challenges in representing numerical data and the importance of transfer learning in improving model performance. The results show that language models can outperform traditional regression models in many cases, especially when trained on diverse tasks. The framework's design allows for efficient training and inference, making it suitable for a wide range of applications. The research contributes to the field of experimental design by demonstrating the effectiveness of language models in regression tasks and opens up new possibilities for future research in this area.OmniPred is a framework that trains language models as universal end-to-end regressors for numerical prediction using textual representations of mathematical parameters and values. The framework leverages data from Google Vizier, a large proprietary blackbox optimization database, to demonstrate that language models can achieve high precision in numerical regression, even when trained on diverse tasks. The model uses a 200M parameter T5 encoder-decoder and represents inputs and outputs in text-based formats, allowing for flexible and scalable regression across various input spaces and objectives. The model is trained on multi-task data and can be fine-tuned online for new tasks, improving performance on unseen studies. The framework's ability to handle diverse input spaces and objectives makes it a promising tool for general-purpose regression. The research highlights the potential of language models in experimental design and other domains requiring precise numerical predictions. The study also addresses challenges in representing numerical data and the importance of transfer learning in improving model performance. The results show that language models can outperform traditional regression models in many cases, especially when trained on diverse tasks. The framework's design allows for efficient training and inference, making it suitable for a wide range of applications. The research contributes to the field of experimental design by demonstrating the effectiveness of language models in regression tasks and opens up new possibilities for future research in this area.