• 2 min read
Weak AI safety rules can raise product risks
A Cornell and Carnegie Mellon model finds that weak, one-sided AI safety rules can create riskier products than no regulation.
Image: iXBT
A mathematical study by researchers at Cornell University and Carnegie Mellon University finds that weak AI safety rules can produce higher-risk products than having no regulation at all—if those rules apply only to companies building end-user services.
The researchers modeled a two-level AI supply chain:
- Foundation-model developers that build general-purpose systems
- Third-party firms that adapt those models for applications such as medical diagnosis and customer service
The model evaluates the resulting products' safety and performance when regulators impose minimum requirements at different stages of development.
How one-sided rules undermine safety
The risk rises when a low safety threshold is imposed exclusively on end-service providers. Under that arrangement, foundation-model suppliers lose incentives to invest in their own safeguards and independent audits. Specialized application developers, meanwhile, inherit much of the financial and technical burden of compliance.
Recommended reading
South Korea targets autonomous port cranes by 2032
Benjamin Laufer of Cornell Tech, the study’s lead author, said:
“Laws become a tool that allows giants to shift the financial and technical burden of safety onto specialized application developers, creating a 'free-rider' effect.”
The researchers define safety broadly, including risks to users such as toxic content generated by chatbots. They argue that mathematical modeling is particularly useful while many AI regulations remain proposals and policymakers are still assessing how rules could distort companies' commercial incentives.
The model highlights how responsibility is split across foundation-model training, fine-tuning, interface design, and operational monitoring. Rules that place legal obligations on only one layer can separate the organizations best positioned to remove a risk from the companies required to document compliance.
The work is a theoretical model, not a measurement of specific AI companies. Its formal framework nevertheless offers regulators and risk professionals guidance for allocating responsibility and designing audit standards that prevent major suppliers from avoiding scrutiny.
AI Editor
Ava covers the rapidly evolving world of artificial intelligence, from foundational models and research labs to the real-world economics of intelligence. With a background in computational linguistics, she cuts through the hype to find out what actually works. She firmly believes that benchmarks are just marketing until reproduced in the wild.
via iXBT
