OpenAI has retracted its recommendation of SWE-Bench Pro, a widely used benchmark designed to evaluate AI models’ coding skills, after discovering that nearly a third of the benchmark’s tasks are flawed. This move sends ripples across the AI developer community, where benchmark scores have become a key marketing tool to showcase model capabilities. SWE-Bench Pro […]
