MIT researchers say they have found a practical way to make AI much better at reading charts: give it a far bigger, cleaner diet of graphs than the web has ever offered. Their new ChartNet dataset, with more than a million synthetic and labeled charts, helped compact open models beat much larger commercial systems at extracting data, answering questions, and summarizing visualizations.
That result is awkward for the industry, and probably a little embarrassing for the expensive frontier models that keep tripping over bar charts in annual reports. The underlying problem is simple enough: charts mix images, numbers, and text, so a model has to understand all three at once. Most existing training sets are too small and too shallow to teach that properly.
What ChartNet adds to chart training
ChartNet was created by researchers from MIT and the MIT-IBM Computing Research Lab. Each example includes the chart image, the code used to generate it, a text description, a numeric table, and question-and-answer pairs with correct answers. In other words, the model does not just see a picture; it gets the full scaffolding behind it.
The dataset was built in two stages. First, existing charts were converted into code. Then the system automatically produced hundreds of variations by changing chart type, data values, styling, colors, and topic. That approach yielded more than a million diverse samples, while an automatic quality-control layer checked that the code and the rendered images still matched the source data.
Smaller open models beat larger commercial systems
The team trained several open models on ChartNet, including IBM Granite Vision. The improved models performed better in four areas: recovering data from charts, extracting numeric information, generating text summaries, and answering questions about diagrams. The most interesting part is not that they improved, but that relatively small open models consistently outperformed much larger commercial systems after training.
- Data recovery from charts
- Number extraction
- Automatic summarization
- Question answering on diagrams
Why chart reading is harder than it looks
Charts sit at the heart of finance, science, and business reporting, which is why this problem has been so stubborn. A human can glance at a line graph and spot the trend immediately; a model has to decode axes, labels, marks, and hidden numbers all at once. That is also why the field has leaned so heavily on closed systems with brute-force scale, even when the results are flaky.
The more interesting twist is economic, not technical. If smaller open models can handle chart interpretation well enough for corporate work, companies may have a cheaper alternative to premium closed products for analytics tasks that do not need giant general-purpose models. MIT says the next step is to make the dataset harder, add more visualization types, and expand the training tasks, which sounds sensible because charts in the real world rarely stay simple for long.