Sber opens SIRIN to catch errors in AI answers

Image source: Grok

Sber has opened access to SIRIN, a library designed to identify fabricated and unverified facts in language-model responses. The tool targets developers building AI services and has already been tested inside Sber on customer-facing scenarios.

SIRIN can analyze either an entire response or individual passages. It highlights sections where errors may be present and checks whether the model has enough information to answer confidently. If the available data is insufficient, an AI agent can retrieve more information—or decline to answer rather than invent details.

Sber says the metamodels at the core of SIRIN improved error-detection accuracy by almost 30%, despite being trained on just 250 examples. The source does not provide a baseline accuracy figure or explain how the improvement was measured.

How SIRIN checks AI responses

The system addresses a familiar failure mode in generative models: a language model predicts likely continuations of text, so it can present a correct fact, a fabricated quotation, or a nonexistent number with similar confidence.

Recommended reading

Intel targets 240 mm packages for giant AI chips

SIRIN is intended to add a verification layer around that generation process. Rather than checking only whether an answer sounds plausible, it examines the claims in the response and whether the underlying information is sufficient to support them. That gives downstream systems an explicit choice: supplement the missing data or avoid producing an unsupported answer.

The approach is aimed at practical deployments, where inaccurate responses can affect customer support, internal knowledge bases, and agent-based workflows. In banking, insurance, and service scenarios, a false answer may send a customer down the wrong path, creating operational and reputational costs that are less tolerable than mistakes in entertainment chatbots.

SIRIN’s intended users

Sber presents SIRIN as a library for developers, not simply as a feature of a consumer chatbot. The company says it has already tested the system in internal services and customer scenarios, moving it beyond a purely laboratory-stage prototype.

The next step is integration into external products that rely on generative models. Sber has opened access to the library, but the source does not specify its license, distribution terms, or a release schedule for future versions. It also does not identify the customer scenarios used in testing or provide independent verification of the nearly 30% accuracy gain.