The paper "EMR-MERGING: Tuning-Free High-Performance Model Merging" addresses the challenge of merging pre-trained and finetuned models to enable multi-task capabilities without significant performance degradation or additional tuning. The authors propose ELECT, MASK & RESCALE-MERGING (EMR-MERGING), a novel method that combines a unified model with lightweight task-specific modulators ( masks and rescalers) to align the direction and magnitude of the unified model with individual models. EMR-MERGING is designed to be tuning-free, requiring no additional data or training. The method is evaluated on various benchmarks, including vision, NLP, PEFT, and multi-modal settings, showing significant performance improvements compared to existing methods. The effectiveness of EMR-MERGING is demonstrated through theoretical analysis and empirical experiments, which validate its ability to reduce performance gaps and improve alignment with individual models. The paper also discusses the limitations and future directions for further improvements.The paper "EMR-MERGING: Tuning-Free High-Performance Model Merging" addresses the challenge of merging pre-trained and finetuned models to enable multi-task capabilities without significant performance degradation or additional tuning. The authors propose ELECT, MASK & RESCALE-MERGING (EMR-MERGING), a novel method that combines a unified model with lightweight task-specific modulators ( masks and rescalers) to align the direction and magnitude of the unified model with individual models. EMR-MERGING is designed to be tuning-free, requiring no additional data or training. The method is evaluated on various benchmarks, including vision, NLP, PEFT, and multi-modal settings, showing significant performance improvements compared to existing methods. The effectiveness of EMR-MERGING is demonstrated through theoretical analysis and empirical experiments, which validate its ability to reduce performance gaps and improve alignment with individual models. The paper also discusses the limitations and future directions for further improvements.