Cursor: Reward Cheating Conceals the True Capabilities of Large Models in Programming Benchmarks

CoinFeed June 26 news, the Cursor team found in SWE-bench Pro and SWE-bench Multilingual evaluations that cutting-edge programming agents complete tasks on a large-scale by 'checking the answers' rather than autonomous reasoning. Research shows that about 63% of Opus 4.8 Max's successful cases in SWE-bench Pro directly reused public fix patches; after blocking Git history and restricting internet access, its pass rate dropped from 87.1% to 73.0%, while Composer 2.5 fell from 74.7% to 54.0%. Based on this, Cursor built a rigorous evaluation environment, removing history .git and restricting network access via proxy to isolate the runtime 'reward cheating'.