OmniJet-α: The first cross-task foundation model for particle physics

OmniJet-α: The first cross-task foundation model for particle physics

8 Mar 2024 | Joschka Birk, Anna Hallin, Gregor Kasieczka
The paper introduces OMNIJET-α, the first cross-task foundation model for particle physics, which aims to improve physics performance and reduce training time and data requirements. The authors report significant progress in several areas, including the development of comprehensive evaluation methods to assess the quality of encoding physics data into representations suitable for autoregressive generation of particle jets using transformer architectures. They demonstrate transfer learning between an unsupervised jet generation task and a supervised jet tagging task, marking a major step in building foundation models for particle physics. The model uses a conditional tokenization strategy with a higher-fidelity tokenization approach, achieving better resolution and fidelity compared to previous methods. The results show that OMNIJET-α can generate jets with good agreement to ground truth and achieve high classification accuracy, even with limited training data. This work paves the way for more efficient and versatile foundation models in particle physics.The paper introduces OMNIJET-α, the first cross-task foundation model for particle physics, which aims to improve physics performance and reduce training time and data requirements. The authors report significant progress in several areas, including the development of comprehensive evaluation methods to assess the quality of encoding physics data into representations suitable for autoregressive generation of particle jets using transformer architectures. They demonstrate transfer learning between an unsupervised jet generation task and a supervised jet tagging task, marking a major step in building foundation models for particle physics. The model uses a conditional tokenization strategy with a higher-fidelity tokenization approach, achieving better resolution and fidelity compared to previous methods. The results show that OMNIJET-α can generate jets with good agreement to ground truth and achieve high classification accuracy, even with limited training data. This work paves the way for more efficient and versatile foundation models in particle physics.
Reach us at info@study.space