What jumped out at me first is not that Claude Code, Codex, and Cursor disagree. It’s that the disagreement seems to come from pretty mundane stuff: what the repo already looks like, what the prompt happened to emphasize, and how much the agent leans on web search versus priors. That feels believable. It also means the “best tool” is often not some stable truth at all, but a local outcome of framing.
I’m a bit more skeptical of how far I’d trust the numbers without seeing the raw failure modes. The article says Gemini judged validity and the winner, and that some sessions were tossed because they weren’t “valid.” Fair enough in spirit, but once you have a pipeline where an LLM is filtering and classifying the outcome of other LLMs, I want to know where the edge cases went. Did it misread traces? Did it over-credit a tool mentioned late in the conversation? Did it miss cases where the agent chose something because it was simply easier to implement, not because it was best? Maybe the public traces answer that. Maybe not. But I wouldn’t treat these results as lab-grade truth; I’d treat them as a very useful stress test.
The part I found most useful is the bit about context flipping choices. That feels like the real lesson for anyone shipping products into agent-driven workflows: you are not just competing on merit, you’re competing on what your docs, pricing pages, and defaults make legible to an agent. “Getting mentioned isn’t winning” is probably the most important sentence here. If a tool shows up in the model’s output a lot but still loses, that’s a sign that visibility and preference are diverging. For vendors, that’s both annoying and instructive. For developers, it’s a warning that an agent’s confident recommendation can still be anchored in shallow cues.
The article also makes a point that I think will age well: agents are not all using the same playbook. Claude Code leaning more on priors, Codex almost always searching, Cursor somewhere else entirely — that matters. It means there probably isn’t one universal “optimize your docs for agents” strategy. You may need to optimize for different retrieval habits, different trust patterns, and different tolerances for in-house versus third-party solutions. That’s messy, but realistic.
What I’d actually want to see next is not another giant category leaderboard. I’d want a few brutal case studies where the agents all picked differently and the reasons were unpacked carefully. That would tell me more than “Stripe wins 9 times out of 10.” The interesting question is when a weaker, more specialized choice wins because the prompt was precise, or when a familiar default wins because the agent didn’t explore enough. That’s where the product-shaping advice lives.
Reference: Which tools do Claude Code, Codex and Cursor choose? We measured 16,893 sessions to find out.