The cybersecurity benchmark question keeps surfacing in different forms this week. Yesterday it was about which models were too locked down to help your security team. Today, the UK government has actual numbers on how fast open-weight models are closing the gap with proprietary ones. The policy implications and the procurement implications point in opposite directions, and most companies haven't noticed yet.
01
The UK government measured how fast open AI models are catching closed ones on cyber. The gap is shrinking fast.
The UK's AI Security Institute published its first public analysis of cybersecurity capability gaps between open-weight models (the kind anyone can download and run) and proprietary ones (like Claude or GPT-5). The headline finding: open models that used to trail the frontier by 6-10 months are now trailing by only 4-7 months on narrow tasks. DeepSeek V4-Pro and GLM-5.2 are the specific models tested, and they're performing comparably to Claude Opus versions released just months earlier. The gap widens on more complex tasks that require chaining multiple steps together, which is where proprietary models still hold a meaningful edge. ---
Why it matters: If you're at a company where "we use open-source models for security reasons" is the policy, that decision just got harder to defend on capability grounds. The cost and control arguments for open models now come with a much smaller capability penalty than they did a year ago. AISI says it plans to test Kimi K3 on the same benchmarks once its weights are public, and given what we saw from Kimi last week, that result will be worth watching.
Box CEO Aaron Levie: cheaper AI means more AI spending, not less
Box CEO Aaron Levie made a point that sounds counterintuitive but holds up: when token costs drop, total AI spending goes up. The logic is that lower prices expand the range of tasks companies can afford to run through AI. You weren't going to process that entire customer database with agents at last year's prices. At this year's prices, you might. ---
Why it matters: Every CFO who approved an AI budget based on current usage patterns is probably underestimating next year's bill. The companies that benefit most from falling AI costs aren't the ones that spend the same amount on fewer tokens. They're the ones that find new things to do with the savings, and then spend more overall. If your team isn't actively looking for what becomes newly affordable each quarter, someone else will find it first.
Vercel CEO Guillermo Rauch: cybersecurity tasks are the real IQ test for AI models
Rauch posted his reasoning for why he considers cybersecurity one of the best benchmarks for measuring genuine AI capability. His argument: cloning an app or writing boilerplate code looks impressive but is actually easy for current models. Finding vulnerabilities, patching them, and understanding how exploits work requires a kind of generalized reasoning that transcends any specific framework or language. The best engineers he's worked with, he notes, have almost always had deep security backgrounds. ---
Why it matters: This connects directly to the UK government's findings above. If security tasks are where the real capability gap between models shows up, then the AISI benchmarks aren't just a policy document. They're a practical guide to which models are actually smarter, not just faster at pattern matching. The next time a vendor shows you a benchmark, ask if it includes security evals.
Thibault Sottiaux is collecting stories about ChatGPT's positive impact
Thibault Sottiaux at OpenAI posted asking users to share moments when ChatGPT had a meaningful positive impact on their life. The post got nearly 800 replies. This is an engagement post, not a product announcement, but 800 replies is a lot of signal. The responses OpenAI collects here will almost certainly show up in future marketing, investor decks, or policy arguments. Worth knowing it's being gathered. ---
Swyx, who runs Latent Space, shared a link to a custom keyboard he found impressive. No AI content here worth covering. Dropping it. *(Item dropped: no substantive AI content to report.)*