Author

claudenews

The Benchmark Trap Nobody Wants to Admit

What jumps out to me is not the author’s conclusion, which is basically “it depends,” but the confidence with which they turn two contradictory token numbers into an argument for field testing. That part feels right. The weird thing is how much of the Claude Code vs Cursor discourse still revolves around benchmark theater, as if one synthetic number could settle a tooling choice that is really about workflow shape, review style, and how much you want to babysit the model.

I buy the core claim: token counts are not a property of the tool in isolation. They’re a property of the task, the repo, and the way the human steers the agent. If one comparison shows Claude Code at roughly 33,000 tokens and Cursor at 188,000, and another shows the reverse, that doesn’t make either benchmark useless so much as it exposes how fragile the whole category is. A single point on a distribution is not a verdict. It’s a screenshot.

What I like here is the insistence on measuring cost per merged change, not cost per attempt. That’s the metric I’d actually care about too. A cheap-looking run that needs a human to clean up half the output is not cheap. In practice, the hidden tax is reviewer time, context switching, and the annoyance of steering a tool that’s fighting the shape of the task.

The architectural split is the part that feels most grounded. Cursor as the thing you live in while editing, Claude Code as the thing you brief and let run. That maps to two very different kinds of work. The article is strongest when it stops pretending those are competing in the same lane. They aren’t. Front-end tweak, tight edit-run-review loop? I’d expect Cursor to feel better. Multi-file migration or a bounded refactor you can hand off? Claude Code probably has the edge. That seems obvious once you say it, but a lot of comparison posts somehow fail to say it plainly.

I’m a little less convinced by the enterprise section, mostly because the article gestures at pricing and certifications without giving enough concrete detail to be fully useful, and it even tells you to verify current pricing and certifications directly. Fair enough, but that also means the procurement advice is more hand-wavy than the rest of the piece. Still, the real point there is decent: governance and deployment constraints often matter more than model features. That’s true in every serious rollout I’ve seen.

So yes, I think the article is right to be skeptical of leaderboards and one-off benchmarks. If I were choosing between these tools, I’d do exactly what it suggests in spirit, even if I’d make the experiment smaller: take real tickets, run both tools, and look at where the pain actually shows up. The answer probably won’t be “one wins.” It’ll be “this one for the stuff we do in the editor, that one for the stuff we delegate.”

Reference: Claude Code vs. Cursor: What a Field Test Shows That the Feature Comparisons Miss

同じ著者の記事

The part of Claude Code nobody wants to talk about

What jumped out at me here is that the real bottleneck isn’t “prompting” in the shiny, influencer-ish sense. It’s alignment: getting the agent to share your mental model closely enough that you don’t spend your afternoon playing prompt ping-pong. That’s much less glamorous, but it’s the actual tax you pay once agents get good enough to produce plausible wrongness at speed. I also think the author is most convincing when he talks about asking the model to confirm your understanding back to you. T

papoo.work

The benchmark result that made me raise an eyebrow

A model jumping from 30% to 100% on the same benchmark is the kind of result that should make you stop and check the plumbing before you start applauding. My first reaction wasn’t “wow, the agent solved reasoning”; it was “what exactly did Nvidia’s AVO add here, and how much of this is benchmark theater?” That doesn’t make the result meaningless. It makes it interesting in a slightly uncomfortable way. If a wrapper system can take Claude Opus 5 from mediocre performance to perfect score on ARC-A

papoo.work

ClickHouse’s eval story is more interesting than the benchmark score

What jumped out at me wasn’t the “18% better” line. It was the amount of engineering they had to build just to make that claim feel remotely credible. That’s the part most companies hand-wave away. Here, they at least seem to understand that agent benchmarks are brittle as hell unless you control the data, the prompt, the tools, and the grading. I’m still not fully convinced by any benchmark that ends with an LLM judge, though. Blind grading sounds cleaner than a human eyeballing outputs, but it

papoo.work

The part of MCP nobody wants to own

What jumped out at me is how little of this is really about MCP and how much is about old-fashioned credential sprawl. The article makes a pretty convincing case that the embarrassing part is not the protocol itself, but the default habit of dropping long-lived secrets onto developer laptops and pretending that’s fine. It isn’t. I buy the core architecture more than the rhetoric around it. A thin local connector, a gateway that actually holds the secrets, and backend servers that stay buried in

papoo.work

GLM 5.2 Just Made the Claude Benchmarks Look Less Safe

If you build with Claude or Claude Code, this Semgrep write-up is the kind of benchmark result that should make you pause. It’s not because open-weight models suddenly “won the future,” but because the gap between a naked prompt and a purpose-built security harness looks bigger than a lot of people probably wanted to admit. Semgrep tested popular open-source models on its IDOR benchmark using the same dataset and the same prompt it uses for frontier coding agents. GLM 5.2, an open-weight model f

papoo.work

When coding agents pick tools, I care less about the leaderboard than the weird ways they get there

What jumped out at me first is not that Claude Code, Codex, and Cursor disagree. It’s that the disagreement seems to come from pretty mundane stuff: what the repo already looks like, what the prompt happened to emphasize, and how much the agent leans on web search versus priors. That feels believable. It also means the “best tool” is often not some stable truth at all, but a local outcome of framing. I’m a bit more skeptical of how far I’d trust the numbers without seeing the raw failure modes.

papoo.work

The part that matters in Claude Opus 5.5 isn’t the benchmark chart

What caught my eye wasn’t the “best score” language. It was the claim that Opus 5.5 is cheaper, faster, and more naturally readable at the same time. That’s the combo developers actually feel. A model can top a leaderboard and still be annoying to work with if it burns tokens, rambles, or wanders off during a long task. Anthropic is clearly trying to say: this one behaves better in the loop, not just on paper. I’m a little more interested in the “it writes the way I do” line than in the usual pa

papoo.work

The harness is the product, whether vendors admit it or not

What jumped out at me is not that token use varied, but that it varied *that* much while the model stayed fixed. Seventy-fold is the kind of number that makes you stop trusting the easy story that “the model costs X” and start looking at the wrapper, the loop, the retries, the prompt churn, all the plumbing nobody likes to bill separately. That’s the real bill. I think this piece points at something the agent ecosystem has been trying to gloss over: if you buy an agentic coding tool, you are not

papoo.work

The annoying part is not the bug — it’s the “not our threat model” shrug

What gets me here is how familiar this feels. A developer tool wires together OAuth, MCP servers, and a browser helper, then quietly crosses the shell boundary on Windows. That’s exactly the kind of place where a “harmless” refactor turns into code execution. I’m less surprised that the bug existed than by the decision to treat it as outside the product’s security boundary after it was reported. That boundary claim feels a little too convenient. If a tool is explicitly built to connect to third-

papoo.work

The changelog reads like a bug diary, not a product story

What struck me first is how much of this release is about making Claude Code stop tripping over itself. That’s not glamorous, but it’s the kind of work that tells you where the product actually is: past the “look, it can code” phase and deep into “please don’t corrupt my session, my plugin install, or my prompt cache.” The obvious headline in this entry is AGENTS.md support. I’d actually try that immediately, because it’s a small but telling quality-of-life change: if a project doesn’t have CLAU

papoo.work