KAIST says this fine-tuning method makes personalized AI safer

Personalized AI is supposed to make models more useful by adapting them to a person’s or company’s own documents and data. The problem is that this kind of fine-tuning can weaken a model’s built-in safety rules. Researchers at KAIST say they have found a way around that.

A team led by Changick Kim, a professor in KAIST’s School of Electrical Engineering, developed a safe fine-tuning framework called Buffer-and-Reinforce. The work was led by Seokil Ham, a doctoral student and first author on the paper, which was selected for a Spotlight presentation at ICML 2026.

The core idea is unusual. The researchers built on earlier findings that fine-tuning a large language model while it is in a temporarily jailbroken state does not significantly damage safety, even though a jailbroken model would normally be expected to answer dangerous requests it should refuse.

KAIST’s system uses that state only during training, not in a deployed service. It adds a temporary buffering module called BufferLoRA while the model is fine-tuned on user data, then removes it after training. According to the team, that buffering layer lets the model learn the new task while reducing the influence of harmful information on the base model.

‹›

1 / 2

×

A second stage then adds ReinforceLoRA, a safety reinforcement module designed to restore and strengthen safeguards. The team said it used QR decomposition to separate different kinds of information and keep the components needed for new capabilities while selectively reinforcing safety.

Recommended reading

Mira Murati’s lab unveils Inkling, a 975B open model

In experiments, the researchers tested the system in an extreme case where all user data consisted of harmful questions and answers. After fine-tuning, the model’s harmful response rate was about 8%, compared with roughly 18% for the original model that had not been fine-tuned.

The team said the framework also delivered strong customized performance and state-of-the-art safety without requiring extra safety data during user-data fine-tuning or significantly raising computational cost.

“This research provides a key foundational technology that allows anyone to build customized AI with their own data while using it more safely,” Kim said.

The paper, “Jailbreak to Protect: Buffering and Reinforcing via Temporary Jailbreaking for Safe Fine-Tuning in Large Language Models,” is available on arXiv with DOI 10.48550/arxiv.2605.24550.