SyMerge merges AI models for higher shared performance

SKKU and NAVER team up on SyMerge

Sungkyunkwan University (SKKU) has announced SyMerge, a model-merging framework developed by an artificial intelligence research team from the College of Computing and Informatics. The work was led by Professor Sung-Eun Hong with researchers Ae-cheon Jeong and Seung-hwan Lee, in collaboration with NAVER AI Lab (Dr. Dong-yoon Han).

The framework is designed to let independently trained AI models trade capabilities and boost overall performance when merged into a single system. The research has been accepted for presentation at the 43rd International Conference on Machine Learning (ICML 2026).

From interference to synergy

Traditional model-merging methods hit a wall when used to build multitask systems. When models specializing in different tasks are combined, their knowledge often collides, causing “task interference”—a sharp performance drop compared with the original models.

Previous academic work has largely tried to minimize or prevent this interference. Hong’s team instead targeted active synergy, aiming for merged models that complement each other rather than just avoid conflict.

Recommended reading

Mira Murati’s lab unveils Inkling, a 975B open model

Single-layer adaptation as the key

The researchers report that one specific internal layer plays an outsized role: the task-specific layer. By coordinating and optimizing the merging ratio of just this single layer among the many inside each model, they found they could maximize compatibility between different models.

The team describes the performance effect as being achievable by adapting just a single core layer, rather than performing broad, expensive retraining across the entire network. This approach directly targets the point where task-specific knowledge resides, which is where interference tends to manifest.

Expert-Guided Self-Labeling on unlabeled data

SyMerge introduces a method called “Expert-Guided Self-Labeling”. When the system encounters new, unlabeled data, it trains itself by using the predictions of existing models as experts.

This guidance allows the merged model to handle corrupted or altered data while maintaining stable performance under adverse conditions. The method effectively recycles the knowledge of expert models into the merged system without requiring labeled datasets.

Merging models from different pretraining origins

A major technical constraint of earlier merging approaches was that they generally required models to be derived from the same pretrained backbone. According to the SKKU team, SyMerge breaks this restriction.

They report that SyMerge can successfully integrate architectures with entirely different pretrained origins, something that had been previously deemed impossible. This opens the door to combining specialist models that were trained in isolation on different foundations.

State-of-the-art across vision and NLP

Experimental results cited by the researchers show that SyMerge achieves state-of-the-art (SOTA) performance across three core pillars of AI:

• Image classification
• Computer vision-based dense prediction
• Natural language processing (NLP)

These results are presented as evidence that the framework is versatile and not limited to a single modality or task type.

Cutting compute costs for multitask systems

Hong frames the work as a step-change in how model merging is viewed:

“This study represents a major milestone that shifts the paradigm of AI model merging from 'interference prevention' to 'mutual synergy creation,'” Hong said.

Hong further emphasized the potential efficiency gains:

“By drastically reducing the massive computing costs associated with retraining AI, this technology will greatly contribute to building lightweight yet highly versatile multitasking AI efficiently in the future.”

The research is detailed in Aecheon Jung et al, “SyMerge: From Non-Interference to Synergistic Merging via Single-Layer Adaptation,” arXiv (2024), DOI: 10.48550/arxiv.2412.19098.