BrainSegFounder is a novel 3D foundation model for multimodal neuroimage segmentation, developed through a two-stage self-supervised pretraining approach using vision transformers. The first stage involves pretraining on a large-scale unlabeled neuroimage dataset from 41,400 participants, focusing on encoding anatomical structures in generally healthy brains. The second stage identifies disease-specific attributes, such as geometric shapes of tumors and lesions. This dual-phase methodology significantly reduces the data requirements for AI model training in neuroimage segmentation and allows adaptation to various imaging modalities. The model was evaluated on the BraTS and ATLAS v2.0 datasets, demonstrating superior performance compared to previous winning solutions using fully supervised learning. BrainSegFounder's performance gains are attributed to scaling up model complexity and the volume of unlabeled training data from generally healthy brains. The model's pretrained versions and code are available at https://github.com/lab-smile/BrainSegFounder. The framework is designed for various neurological tasks, including brain tumor segmentation, stroke localization, and Alzheimer's diagnosis. It uses a large dataset of brain imaging from a generally healthy population to improve clinical workflows and enhance diagnostic speed and accuracy. The model's architecture includes a vision transformer-based encoder and an up-sampling decoder, adapted from the SwinUNETR architecture. The two-stage pretraining process involves self-supervised learning and fine-tuning for downstream tasks. The model's performance on the BraTS and ATLAS datasets shows significant improvements, with BrainSegFounder-Small achieving the best average Dice coefficient. The model's ability to adapt to different data modalities and its effectiveness in few-shot learning scenarios highlight its robustness and versatility. The study also demonstrates the model's performance on the ATLAS dataset, achieving a high Dice score and lesion-wise F1-score. Overall, BrainSegFounder represents a significant advancement in 3D foundation models for neuroimage segmentation, offering improved accuracy and efficiency in medical imaging tasks.BrainSegFounder is a novel 3D foundation model for multimodal neuroimage segmentation, developed through a two-stage self-supervised pretraining approach using vision transformers. The first stage involves pretraining on a large-scale unlabeled neuroimage dataset from 41,400 participants, focusing on encoding anatomical structures in generally healthy brains. The second stage identifies disease-specific attributes, such as geometric shapes of tumors and lesions. This dual-phase methodology significantly reduces the data requirements for AI model training in neuroimage segmentation and allows adaptation to various imaging modalities. The model was evaluated on the BraTS and ATLAS v2.0 datasets, demonstrating superior performance compared to previous winning solutions using fully supervised learning. BrainSegFounder's performance gains are attributed to scaling up model complexity and the volume of unlabeled training data from generally healthy brains. The model's pretrained versions and code are available at https://github.com/lab-smile/BrainSegFounder. The framework is designed for various neurological tasks, including brain tumor segmentation, stroke localization, and Alzheimer's diagnosis. It uses a large dataset of brain imaging from a generally healthy population to improve clinical workflows and enhance diagnostic speed and accuracy. The model's architecture includes a vision transformer-based encoder and an up-sampling decoder, adapted from the SwinUNETR architecture. The two-stage pretraining process involves self-supervised learning and fine-tuning for downstream tasks. The model's performance on the BraTS and ATLAS datasets shows significant improvements, with BrainSegFounder-Small achieving the best average Dice coefficient. The model's ability to adapt to different data modalities and its effectiveness in few-shot learning scenarios highlight its robustness and versatility. The study also demonstrates the model's performance on the ATLAS dataset, achieving a high Dice score and lesion-wise F1-score. Overall, BrainSegFounder represents a significant advancement in 3D foundation models for neuroimage segmentation, offering improved accuracy and efficiency in medical imaging tasks.