Robotics & Physical AIOct 15, 2026
Given enough turns, open-weight models match the best closed ones at threat detection for a tenth of the cost
An arXiv paper posted on 15 October 2026 evaluates 26 models, 19 open-weight and 7 closed, on CTI-REALM, a benchmark where an agent turns a threat-intelligence report into a working detection by querying a live telemetry store. Its main finding is that an open-weight model can match the strongest closed models at less than a tenth of their cost if given enough turns. The benchmark's published limit on agent messages is a confound: a task that hits the limit scores zero on every checkpoint, including ones already passed. At that limit the leading open and closed models cannot be told apart; with it relaxed, the highest rewards go to open-weight models, and price is a poor guide to capability.
What it means Security teams that need to run and audit their own models may not have to pay a quality penalty. Agent benchmark leaderboards can also hinge on turn limits rather than ability.
Where it came from arXiv