**Psychometry: An Omnifit Model for Image Reconstruction from Human Brain Activity** **Authors:** Ruijie Quan, Wenguang Wang, Zhibo Tian, Fan Ma, Yi Yang **Institution:** ReLER, CCAI, Zhejiang University; Lanzhou University **GitHub:** https://github.com/QUANRJ/Psychometry **Abstract:** Reconstructing images from human brain activity using functional Magnetic Resonance Imaging (fMRI) is a challenging task in Brain-Computer Interface research. Existing methods often focus on training separate models for each individual, ignoring commonalities among data. This paper introduces Psychometry, an omnifit model that captures both inter-subject commonalities and individual differences. Psychometry incorporates an Omni Mixture-of-Experts (Omni MoE) module, where all experts work together to capture commonalities, while subject-specific parameters address individual differences. Additionally, Psychometry uses a retrieval-enhanced inference strategy, Ephory, which enhances fMRI representations by retrieving relevant memories. These features make Psychometry efficient and effective, enabling high-quality image reconstructions from fMRI data. **Introduction:** The paper discusses the challenges of interpreting brain activity through fMRI, highlighting the need for a more generalized model. Current methods suffer from performance degradation when using data from multiple subjects to train a unified model. Psychometry addresses this by training a single, omnifit model on aggregated fMRI data, capturing both commonalities and individual variations. **Methodology:** - **Omni MoE Layer:** This layer uses multiple experts to capture commonalities and subject-specific parameters to address individual differences. It employs a split-then-lump mechanism to maintain efficiency. - **Ephory Inference Strategy:** This strategy enhances fMRI representations by retrieving relevant memories, improving the quality of conditional signals for image reconstruction. **Experiments:** - **Datasets and Evaluation:** The Natural Scenes Dataset (NSD) is used, with fMRI data from 8 participants. Psychometry is compared to state-of-the-art methods, showing significant improvements in both qualitative and quantitative evaluations. **Conclusion:** Psychometry is a significant advancement in fMRI-to-image reconstruction, offering high-quality reconstructions and capturing both inter-subject commonality and individual specificity. Future work includes extending the model to handle more complex brain activity signals and ensuring privacy protection when using aggregated fMRI data.**Psychometry: An Omnifit Model for Image Reconstruction from Human Brain Activity** **Authors:** Ruijie Quan, Wenguang Wang, Zhibo Tian, Fan Ma, Yi Yang **Institution:** ReLER, CCAI, Zhejiang University; Lanzhou University **GitHub:** https://github.com/QUANRJ/Psychometry **Abstract:** Reconstructing images from human brain activity using functional Magnetic Resonance Imaging (fMRI) is a challenging task in Brain-Computer Interface research. Existing methods often focus on training separate models for each individual, ignoring commonalities among data. This paper introduces Psychometry, an omnifit model that captures both inter-subject commonalities and individual differences. Psychometry incorporates an Omni Mixture-of-Experts (Omni MoE) module, where all experts work together to capture commonalities, while subject-specific parameters address individual differences. Additionally, Psychometry uses a retrieval-enhanced inference strategy, Ephory, which enhances fMRI representations by retrieving relevant memories. These features make Psychometry efficient and effective, enabling high-quality image reconstructions from fMRI data. **Introduction:** The paper discusses the challenges of interpreting brain activity through fMRI, highlighting the need for a more generalized model. Current methods suffer from performance degradation when using data from multiple subjects to train a unified model. Psychometry addresses this by training a single, omnifit model on aggregated fMRI data, capturing both commonalities and individual variations. **Methodology:** - **Omni MoE Layer:** This layer uses multiple experts to capture commonalities and subject-specific parameters to address individual differences. It employs a split-then-lump mechanism to maintain efficiency. - **Ephory Inference Strategy:** This strategy enhances fMRI representations by retrieving relevant memories, improving the quality of conditional signals for image reconstruction. **Experiments:** - **Datasets and Evaluation:** The Natural Scenes Dataset (NSD) is used, with fMRI data from 8 participants. Psychometry is compared to state-of-the-art methods, showing significant improvements in both qualitative and quantitative evaluations. **Conclusion:** Psychometry is a significant advancement in fMRI-to-image reconstruction, offering high-quality reconstructions and capturing both inter-subject commonality and individual specificity. Future work includes extending the model to handle more complex brain activity signals and ensuring privacy protection when using aggregated fMRI data.