What really burns tokens is sub agents. I once gave Claude Code a pretty big task, and it immediately launched 7 sub agents which burned through my budget before even one of them was finished. Tried again 5 hours later: same result.
If I let the main agent do the same task sequentially, it was no problem at all. I don't know if it's really just communication and orchestration that makes sub agents so inefficient, or if Anthropic figured that most people using sub agents pay per token on a big corporate account, so this is an easy way to make more money from tokenmaxxers.
As a counterpoint: in a complex project, Fable's "curiosity" may be exactly what you want for an exploration and planning stage - not just for the orchestrator that turns your prompt into different angles with which to explore, but for each subagent whose task is to search the codebase for one of those "angles." If you truly want no stone unturned, letting those subagents spawn their own discoveries, and recursively grow the surface area of the inquiry, then it's quite reasonable to want Fable throughout.
That said, if your project is "do this well-planned thing on a bunch of things in parallel" then you should absolutely be instructing to have subagents "step down" to less curious models. Their output may well be more cohesive as a result!
The curiosity is inefficient though. So many times I have to stop the agent and tell it to just fucking write the code and try compiling it. Otherwise it will fill its entire context tracing through the program logic to derive from the code itself whether the thing it is about to do would work. It completely fails to notice it can just… try.
Everything about LLMs is inefficient. They have their benefits but watching them reason over things that are painfully obvious, that they've literally investigated before (before a memory compaction), never take a step back aand be like 'this is going too slow let me look for a better way', etc. is painful.
It’s got worse though right? Older models from before everyone went off the deep end with CoT don’t do this and just write the code with 1/10 the token usage.
The downside is the code isn’t as good but it is produced a lot faster and more cheaply and often it’s actually fine.
CoT has made LLMs better (say 50% improvement or something) but increases cost by an order of magnitude. That graph is going in the wrong direction and has been for a while now
This is how I worked with LLMs originally, and I much preferred it. This gave me a much better understanding of the code that I was adding. But, there's no way to keep up with my team like this anymore. It's just too slow when everyone else is working directly in Claude Code.
If the entire team is vibe coding and there is no human audit then there is no way but to vibe code, for sure.
I would also just vibe it if there is no responsibility, but if I do it that way I don't even care what happens with the project.
I get so detached from it that I stop caring and if it has huge critical bugs..I just don't care anymore because it's not my responsibility or my code at all at that point. I'm just there to nudge things along.
Just hook it up to Jira and let the managers add the features then pass it off to QA.
Real engineering is fully automated at that point.
> I just don't care anymore because it's not my responsibility or my code at all at that point
Yep, 100%.
Business has made it clear they don’t care, so there’s no point in burning one’s energy. Throw the whole thing on auto, check out, and go do something else during the day.
I agree, that work in these places is likely short lived, if for no other reason than working in them is awful and demoralising. My point is, that these places exist, and have such a hard-on for “oh my god, AI!!!1!1!” that putting in extra effort there, is a waste of your own energy.
I'm just thinking: The result is net-net the same. Either they terminate your contract as a salaried employee or they do it as a consultant. The fee is higher as a consultant and you never have to say you were fired. You're making a really good argument for getting into consulting.
You can just answer the same rote questions to different companies for max salary, then when the going gets tough you can essentially 'fire' the company and say they've moved past your core consulting paradigm. Maybe suggest a new consultant and move one. Rinse; repeat.
Parent comment said they end up "...pass[ing] it off to QA."
Edit: suppose that very well could have been tongue-in-cheek.
Side note: as someone who has been interested in programming for a while, but didn't end up in a software dev track in life, it's been pretty wild watching the ride you all have been going on lately. I used to be pretty bummed I didn't get to do that kind of work for a living, but lately I've been feeling more and more like I dodged a bullet.
Not that I don't have my own AI-related junk I have to deal with where I did end up, of course. I think most have.
I like it but how much context does it need for a complex program? If you're giving instructions and using its code, I imagine context is being passed back up in an exponential way. If not, and you give it a very thin context every time, how do you manage to prompt it enough?
It's tuned for the kinds of tasks where "just try" doesn't get good results.
A major complaint with AI code was that AIs struggle with complex codebases, don't respect existing conventions, reinvent functionality multiple times over, etc. So, newer high end AIs are tuned with the "explore/exploit" dial turned towards "explore".
You could probably get it to do things "quick and dirty" with prompting, but that, of course, requires prompting for it.
I get what you're saying, but these instances are of a different type. It is along the lines of "if I pass None for this parameter, will it default to X or return an error?" and it looks, but finds that the actual logic is distributed across multiple files. And so it quickly falls down that rabbit hole.
All the while, it could have just done a two-line probe test and see what happens when it calls the API with "None" for that parameter. Or just assum it would act as expected and wire in debug logs in case it doesn't.
That is the usual work of high end programmers, right? Growing codebases as consistent, dependable ontologies?
I feel like most mainstream programming languages do this sort of work for their standard libraries and their official docs. Go and Python come to mind, but plenty others do this reasonably well to the point where one mostly doesn’t need to read the implementation code to effectively use the standard library itself.
I use the human-in-the-loop for managing the context.
Give it only what it needs and do things usually 1 file at a time.
Feels like I'm a sort of manual tape editor, if the context was a tape fed into machine, I assemble that and then watch the machine output the results I need.
that's why i mostly use it for asynchronous work, the inefficiency is something i can bear with because the subscription costs are dirt cheap. if it's token-based, it wouldn't make financial sense.
Next time it does that expensive scan, run order it to keep or update an index on the codebase. It really helps prevent these expansive scans if you have additional markdown files for LLM navigation.
Fable and sub agents are two different things. There are many situations for which Fable is great, but Fable doesn't have to run in a sub agent. You can use it for your main agent and that works fine.
Or are you saying my sub agents burned so many tokens because they were all using Fable, whereas my main agent could do the same job with a lesser model?
I think the commenter (who is not me) is saying to use Fable as the main agent but then use lesser models for your subagents so that you get the advantages of Fable to plan but then the subagents don’t cost as much, and may be more focused because they lack Fable’s thinking modes.
I’ve heard the proper pattern is to have Fable write a software design doc and then tell Opus to follow that doc strictly in implementation and testing.
I hear that too, but I'm much more ad-hoc about what model I use for what. Opus can be good at planning too, and Fable is remarkably good at figuring out obscure complexities in the code.
How do I get it to spend fable tokens on “curiosity” then switch to cheaper models? Preferably based on its own judgment of what model is truly needed.
Just ask it to. If you want, you can also give it pointers to how to read .claude jsonl/metadata so it keeps track of usage and self-adjusts. It's not perfect, but it's pretty dang good if you just say 'This project is allotted X% of my 5h limit'.
If the assumption is that they can be searched in parallel and it takes the same amount of tokens as doing it sequentially. What you would gain is a speed up.
I vaguely understand you argument with the context, however is that not solved by sum agents handing their results in to the planner (or a third agent) to run on them again? I'd assume that's what is happening anyway. Let me know if that's wrong
doesnt intelligence involve knowing where to start and what to read and not just throwing everything in the bag.
im on local only AI and subagents are only valuable when they avoid polluting the context with extraneous file reads and parallel exploration when fixes are linear.
as OP is on about, subagents burn tokens because they arnt a deterministic intelligent gatherer but like pooluring water into a maze hoping the exit will illuminate.
Sub agents each have to read part of your code base again to get enough context for the task. And if they take too long, your orchestrator's context is no longer in cache so you pay full price for that again once the subagents finish
If you do it sequentially you only read those files approximately once, and everything hits the same prefix cache
Yes but one of the key things about subagents is they keep all of their tool calls and exploration out of the parent context.
If you plan on continuing on in the parent, and aren't going to necessarily be touching the systems the other agents are exploring, it can be worth it.
It's useful in certain situations where the parent context may need the "10,000 foot" view of something without going back in there. But subsystem-specific AGENTS.md/CLAUDE.md files are still superior and accomplish the same thing. The problem with those is they can become stale.
It seems like there could be a useful strategy of writing a plan with a main agent, and then instead of spawning subagents to implement, fork the main context to write each part. Then use one last fork to verify the work. That way you keep reusing the same context without polluting your main context for when you are ready to continue.
I've started doing this by hand in OpenCode and it works pretty well. But there's no UI support for maintaining a tree of related session forks so there's a little bit of manual fussing involved with session naming to keep organized. I also like to end a session with an "AI-friendly terse but detailed summary" (or some equivalent prompt) that I can then dump out to a Markdown file and then the mainline session can still get info back from the branch session. I don't know how much of this is automatable with OpenCode plugins, or in another hackable harness like Pi.
Right, so it’s a trade off between contexts. There are two reasons to use subagents, parallelism and tailoring of context. For the second, there is the “personality” of the subagents as well as how much context is injected from the main agent. Ignoring the personality, you ideally want the injected context to be small and focused on a single task so the subagent doesn’t get distracted. You want the main agent to be orchestrating all the subagents, but not reading all the same files they are reading, otherwise you’ll be paying for the same tokens in multiple contexts. IMO, this is where prompt engineering comes in, to be able to guide the main agent as to where subagents are desired and where not.
For the sake of accuracy Claude Code has a "fork" feature which does what you'd expect and based on traces I skimmed this morning it seems under some circumstances you get fork behavior for subagents as well.
Every subagent send the same ~30k system prompts. If you are using fable/opus, that's easily 30% of a 5-hour window for 7 subagent, before doing any work
Compared to the primary agent, maybe. But it's highly unlikely that all the agents have different tools and system prompts than each other, and those account for the bulk of the context per the post.
The system prompt and available tools would likely only change for different agent types. So how they're launched probably doesn't matter.
I say this as I don't actually know how Clade Code does this, since it's not open source, but I fail to see why two agents doing the same thing launched at different times would have different tools and system prompts.
I'm pretty sure the system instructions are a function of your environment and not the same universally. That said, there should be a finite number of branches so still cacheable.
Same for me. I never use them. I use Fable on highest effort to plan things and then record the plan in tickets. I use Kata, which is CLI and agent oriented, but I suppose Jira or other systems would work too. I tell it to put enough context in each ticket to on-board a fresh coding agent to implement it. Then I just do /goal, telling to to run `kata ready` to get new tickets to work and continue until they're all closed according to acceptance criteria or until they're blocked on actions from me. I need to play around with getting it to switch to smaller models (or spawning 1 subagent) to do ticket implementation and then auto compact after each. Either way, it results in really easy workflows and uses very few tokens compared to the built in subagent flows that doing this completely avoids.
Subagents with a fat tailed latency distribution completely masks the trough filling that puts the most downwards pressure on per-token COGS.
This is why the subscription plans are forced through the harness (the "OpenClaw Wars"): it creates a false equivalence in the minds of many customers between API tokens (latency sensitive, easy to measure) and Claude Code tokens (remnant backfill to stay to the right of the roofline, marginal cost often zero).
Selling sausage as sirloin is a great business if people go for it. And there's nothing inherently wrong with spot pricing, as long as you're honest about it...
I had learnt that trick, so now I explicitly disallow Fable subagents.
Yesterday, I wanted to review a complex piece after a large refactoring, and requested a review plan beforehand. The first step was 8 agents + one more to verify the findings (all Fable). Looks good, approved.
The verification step turned into an attempt to throw a party with 41 Fable verifiers.
Agreed. The issue is that when working 1:1 you get a feel for how many tokens are being burned but the subagent spawn could be 3 or in one cases it spawned 171 to verify something. The latter was unexpected and burned through my token budget.
I find it useful for code reviews (spawn a subagent with minimal/no context to review X commit). Of course, this is more or less a shortcut that could be done with a seperate agent. Another use is multiple reviews at once if tokens are not an issue, with seperate "personas" or focuses. As far as implementation goes I have not seen any major usecase.
Yeah, my personal workflow has different reviewers for codebase(patterns, code cleanliness, etc), frontend, security, product fit, etc. So they spawn as separate subagents. Both so that they stay limited to their role, and so they don't have preconceived notions about the implementations. It's a bit heavy-handed but works for me.
> CRITICAL: Do NOT spawn sub-agents for any reason. Perform all work in the main session. If a task is too large, ask me to break it down manually.
> This is a big task, and can easily get too large. However, sub-agents make the situation worse, and eat through our token budget way too fast. Do not use them.
> Take on manageable tasks. Don't try to do everything at once. When you start on a big task, break it down into smaller tasks, and make sure you finish each task before starting on the next one.
Or actually Claude put it there for me. Maybe it's a bit much, but it seems to work.
I think there's some setting to restrict the number of them, or maybe turn them off. Doesn't happen for me ~ever and it's not my $$ (work) so I haven't really looked at it much.
> What really burns tokens is sub agents. I once gave Claude Code a pretty big task, and it immediately launched 7 sub agents which burned through my budget before even one of them was finished. Tried again 5 hours later: same result.
Probably because the general purpose subagents inherit the parent model.
I tell Claude explicitly to use Explore subagents, which use Haiku only, now.
Yeah, I was surprised. I had Claude make a skill to extract the explore agent from Claude Code, but set the model back to haiku. Here it is if it's helpful:
for subagents to be cheap/effective, you have to specify the size of those subagents; i.e. right now by default 5.6-sol spawns many 5.6-sol subagents. 5.4-mini as subagent saves me tons of tokens. 5.6-sol audits the work before accepting it, so there's not really a quality issue.
It’s funny too because I’ll ask fairly simple things and it’s fine, similarly simple question might spin up a bunch of sub agents and I don’t know why….
I feel like maybe it could have asked for clarification or something rather than go and try to calculate all the digits of pi all of a sudden.
Probably both. The default subagent orchestration is designed for infinite pockets.
Maybe when they realize there is need to change this they come up with a more configurable interface for us mere mortals who can't afford to gamble their house on a pay as you go subscription.
They did recently change it so the default explorer agent inherits the session agent (capped at Opus). Before Explore was always haiku. I had Claude write a skill that extracts the built in Explorer agent skill, and then writes an identical Explore agent that uses Haiku
There is a negative incentive to fix problems that result in customers picking a more expensive plan to work around it. There are probably several engineers who have ideas about fixing this and they get apathy from many people and obstruction from a few, and sometimes active hostility by a manager somewhere in the chain.
The best you can do in such an environment is seek to introduce new features at the top tier, and then pull old features down the stack as the cost of those features has been amortized out, or to hurt your competitors by raising the ladder.
I’ve had similar experiences. I now have an explicit line in AGENTS.md to not use subagents unless explicitly requested. It also helps that for the tasks that are big enough to benefit from subagents are also the ones with high chances of going off-rails and/or a poor review phase. I’d rather do the orchestrator role and that way I can split up the review phase in a much more manageable chunk.
They optimized it to burn more token in the recent months I feel. I made a small ~100 line change to a codebase by hand and threw claude at it to review. It spawned several sub-agents and burnt a ton of tokens. I guess the word 'review' now triggers some sort of in-built skill or something. It's absurd how rapid enshittification is taking over.
> I still have PTSD from how ungodly terrible it was
Please, for the sake of everyone suffering from actual PTSD: Don't. It's hard enough already for victims to communicate what difficulties they are facing without people watering down terminology like that.
I am asking them to reconsider and reflect on what that kind of language use does. You're the one reading it as "calling them out".
How else are we supposed to learn from each other, voice our opinions, point out our mistakes to each other? For me, this is communication. And currently 8 upvotes seem to agree with me and my request. Feel free to ignore it, or consider it, for your own use of language. But, sorry, to me, you're the one acting like a jerk and trying to "police", not me.
PTSD sucks on its own. Trying to blame other people for its symptoms is a deflection. It would be like someone with ADHD blaming his inability to concentrate on everyone else in the room making noise or moving around. People with OCD tried this, acting like it's a Monopoly property that you have to pay rent for if you land on it / say its name.
If anything, it's a net positive people are talking about mental health and recognize different ailments such as OCD, PTSD, and others.
I agree with what you say? I think you're reading something into my writing that I didn't mean to imply. A headache is not a migraine is not cluster headache is not multiple sclerosis. We have vocabulary so we can express ourselves in an attempt to communicate. I am inviting people to reconsider their use of language when it comes to severe challenges such as PTSD. We are here to exchange opinions after all. If you find that offensive, so be it. You decide for yourself how you express your disapproval; two opinions can stand next to each other without introducing "blame" or "deflection".
I agree with you that it's a low effort, repetitive drone when people say "that cable job irritates my OCD" or "I still have PTSD from dealing with that annoying person". I don't like hearing it because it makes me think that person is a parrot that repeats common phrases instead of developing their own voice. So at least we agree partially.
I disagree that it's insensitive to those or have the illness. If it hurts you when people use PTSD as a literary special effect, I invite you to articulate why exactly it makes you feel (however it makes you feel - I don't want to put words in your mouth). I get the sense that some people take offense the same way a religious fundamentalist doesn't want to hear their Prophet or Messiah used disrespectfully. The source of the anger is built on faith and dogma.
This has been my experience as well. Something happened 2-3 months ago with Claude Code. It got slower, starting spinning and getting stuck more and more. I gave codex another shot out of my Claude frustrations, and have never looked back again.
Just tried Claude Code yesterday, and nope, it's the same old bad.
For a while everyone was saying sub agents is how you save tokens, use lower quality models with limited context to do simple parts of the job after a smart planning agent has put it all in place. Is that no longer true or is this just the result of sub agent being used at the wrong time?
No, you can definitely configure low cost search and apply subagents. CC and Codex do not. Not sure if this is to improve the reliability of their subagents, or just a play to increase user consumption.
I like to use subagents a lot, but I find them to be most useful when explicitly specified. E.g. "assign these tasks to 2 Sonnet, 2 Opus and 1 Fable subagent". Helps keep allocation consumption under control.
Not only a Claude Code issue. Started using OMP with GPT 5.6, and gave up, it loves to use subagents, and it's basically unusable subagents with GPT 5.6 Sol there with Plus limits.
lol I asked fable to help me estimate my TAM and it launched 102 agents and blew my $120 quota in 6 minutes. I do realize I can limit the agent count , hah
It's in the best interest for AI companies to gobble up tokens. I feel like every new release - Fable, etc - is just a way to extract more tokens/money.
(If) something like the current LLM/agent paradigm remains in a few years, and companies settle down into their respective niches, I imagine more user-friendly tools will be built, with more control over subagent spawning, context, caching, etc.
What's happening this year, with secrecy and all, is saddening, but expected.
it's a very handwavey way to "explain" anything. Yes, they make money. But they have competition. And if someone runs out of tokens and switches to deepseek or just goes for a friggin hike in the woods, that does not benefit them. If they get a public image of a ripoff that burns all shit on trivial tasks, that does not do them good either. So there is a limit to this "companies make money" thing.
Sure, fair enough. Clearly, if they increase costs by too much, people will go to their competitors, but those competitors also make money selling tokens, so the whole industry is incentivized to inflate token consumption up to the point of driving people to the competition. And nobody is incentivized to reduce token count.
In fact, the one model with great price/performance is Deepseek v4 Flash and I suspect that they are subsidizing it deeply to get access to everyone’s prompts for training. We may find that they raise prices on the next version (v5) after they’ve mined the user data.
Any AI service that people (and to some extent companies) can afford to pay for today is being heavily subsidized. Will that last forever? I really don't know how those economics work, but I know that bubbles do burst having lived through the dot com burst in 2000. And I know this current one is going to hurt if/when it bursts.
On the issue of the bubble, I’m right there with you, 100%. I’m not sure that “subsidized” is the right word for Anthropic’s or OpenAI’s pricing, though. I’d say it’s forward-priced. Supposedly, they have claimed that inference by itself is profitable; it’s the ongoing training that is not. I don’t know what nuances apply to that, however.
Subagents are quite inefficient and the lossy context transfer between them does lead to more cost and more waiting. However I have found it to produce more reliable output, whether that is worth it for a given task has been a consideration.
My opinion is that claude code uses more tokens simply because Anthropic makes more money that way and forces people into their subscriptions. This is supported by the fact that they won't let you use your sub on a different coding agent. I use pi btw.
Once I realized that Anthropic is a token merchant, I start to understand Anthropic’s decision more. They are always finding reasons for you to use more tokens through them unless the users revolt or demand some guardrails.
I've done a couple side by sides on web chat with the same prompt on Opus 4.6, 4.7, and 4.8 and the output gets longer/more verbose on version increment. The enerr variants are definitely much wordier.
On the other hand, the newer variants also tend to benchmark higher so it's not quite a clean argument of "hey the new version eats more tokens"
From my experience new models are slower and use more tokens even on questions which gpt 4 answered correctly. It is mostly because newer models tend to be more verbose (even with prompt requesting short answers).
I've done a couple side by sides on web chat with the same prompt on local 4b, 14b, 32b open models and the output gets longer/more verbose on version increment.
Its rather frustrating, slower tokens and more tokens.
If I were anthropic I’d force that too. They offer the harness and if they control the entire pipeline then they can optimize the entire experience. It doesn’t have to be nefarious.
This is kind of a strange comment as it implies a false dichotomy.
Its not 'nefarious' in that its in their best business interests.
But it'd be difficult to take anyone serious who thinks Anthropic's motivation was to improve the UX, and the other effect were by accident. At the time they specifically started blocking based on openclaw prompt text. Its a walled-garden tactic.
A walled garden is nefarious to people who do not want to be inside one.
I don't think it's in the same ballpark at all. I checked the `/usage` in my session which uses a Max x5 plan. One day I had used $400 of tokens and 20% of my Fable allocation. Anthropic is effectively giving us more tokens per $ on the monthly plans but it comes at the cost of Anthropic being the prompt-writers and managers of the agents pretty much entirely. I don't think this is a bad deal.
Whether or not it's a bad deal depends on what you're comparing it to. Compared to API pricing, of course a subscription through CC is a good deal. But when OAI offers their super-subsidized plan and allows you to use your own harness which is 75% more token-efficient, then the CC deal starts looking like a bad one in comparison.
Sounds like they're modeling their PR on the classic Apple playbook: "choice is bad, and you should appreciate the constraints we've generously imposed"
The nefarious part is because it's non optional. They could give you an option and compete by being better, instead you're given the finger as the option is taken from you. Competition is hard and banning people to create more FUD serves business need better.
You've obviously been gaslit so badly you're desperate to find a way to defend a shitty move and pretend it's the only way to increase usability. But you don't have to deny really! You're allowed to admit control is easier for a company than competition, and that they didn't have to, but did because it increases their control of the ecosystem.
If you want to defend someone, good? But at least save it for someone who actually deserves it. They don't; and you insult you and your readers intelligence by trying.
But they gave us double the tokens! Then a limited time more usage! Then even more tokens "off peak" times! Then some new model released but apparently it inherently used 1.69x tokens! Then Fable is here but "it uses much more usage". But only until ~~the US banned it~~ ~~7th July~~ ~~19th July~~ who even knows.
At this point I think Dario is just in his wellness retreat adjusting a revenue/profit dial.
now reealize that LLMs are trained to produce tokens and like the halting problem, cant be trained not to produce tokens and youll realiE the AI labs are the perfect essential capitalist and like cancer, will keep growing useless tokens until it kills its host.
no amount of alignment will stop aomeone drom just shutting up.
LLMs might be trained to produce tokens, but Anthropic don’t have to price by tokens.
If an organization is a ‘non-profit’ and they decided to design their pricing to be tokens-based, I get it.
If a for-profit design their pricing to be tokens-based, I don’t know where are they drawing the line between profit vs benefit. That doubts makes it hard for me to be a customer. Disclaimer, I still use Claude…
Seems unlikely they'd be this dumb. The way to get us to use more tokens is to make those tokens more useful, not less. Anthropic is full of people (including higher-ups) who know this.
> What stops any business from over-charging? Competition.
I fully agree.
> It's the same conflict of interest quite literally any business has.
I know that you know what I meant ;)
In the long term it is just as you say - overcharging (eventually corrected by competition forces), but in the short term it can be additional revenue, blamed on a bug, but making some manager look good.
The vast majority of my company's enterprise plan use is through Claude Code even though we have access to the API and could be using OpenCode instead.
I don't fully agree with the premise that they intentionally increase system prompts, but the enterprise plan usage is going to make that a huge income for Anthropic.
The fact that individuals are more likely to use the alternatives than businesses is telling.
Anthropic is fine, as long as someone else (a clueless employer drinking Dario Koolaid) is paying for it. But the moment you have to pay for it, people just bail and go for DeepSeek, Kimmi, OpenRouter, OpenCode Go and other alternatives that give more bang for the buck than Anthropic.
You're making the opposite argument. Anthropic is incentivized to use less tokens in Claude Code because people are paying a fixed monthly fee for subscriptions.
That strategy only makes sense if there's an abundance of tokens, but that's not the case. AI companies are spending a ton of resources on improving token efficiency because they are all severely GPU constrained. Anthropic instead nudges you to move to a higher tier by setting rate limits.
Also not true, they want you to pay for a higher subscription bracket and then use only marginally more than you would have, which I think they’re doing quite effectively for most people based on my interactions.
Flip side is customer psychology. Choosing a more expensive tier leaves better emotion.
Also i doubt there was jira ticket with “make llm more verbose”, rather ticket with “bug makes llms too verbose” gets prioritised taking revenue impact into account.
Higher subscription brackets are likely worse for them. I recall seeing someone calculate that a fully maxed out highest subscription bracket is something like $15K in tokens?
And people paying $100 or $200 are much more likely to max it out for purely psychological reasons - it crosses that threshold where I want to see my money's worth in full. Whereas people on $20 subs are more likely to be there just to get access to better models and features, and are not necessarily even doing any substantial work.
It's always more complicated than that because the prestige and Early Adopter users are what drag other people to also be customers to avoid FOMO.
Your gym members who got a subscription aspirationally and don't show up are absolutely subsidizing the power lifter who is introducing wear on (tens of?) thousands of dollars of equipment three times a week, but if the regulars weren't there you wouldn't have sold those subscriptions at all. Without a poster child there's no poster.
Generally, companies with >150 people can’t use subs. So yeah, it’s mostly a funnel for devs/small companies to eventually vet for the product and convince their enterprise to use it as well.
Not really. The incentive is to make you hooked on the process, so you bring the same process to the workplace, and start paying corporate prices, not individual subscription prices. For that to work Claude Code, prompt, and the rest of the mechanics has to be more or less uniform.
When using Pi, one way to significantly reduce input tokens it yields is to ignore common bookkeeping "dot directories", such as `.git`. How to do so can be found with the following interactive Pi prompt:
How do I configure Pi to ignore git related artifacts, such
as the project's .git directory?
Other local assets to consider ignoring are `.pi`, `.agents`, `*.md`, and language specific output directories such as `__pycache__`, `bin`, `obj`, `target`, etc.
Pi is one of the ways out of this problem (OpenCode another) so I took it as an intentional reference as it is highly relevant. I also use Pi as my daily driver and I think it's a wise choice to figure out how to decouple yourself from lab-specific harnesses that you have little control or observability over.
the amount of system prompt wastage going on in orgs is insane. we identified 400k in annual burn for zero value in just one section of our large company.
and the interesting thing about system prompt wastage is its a cost that scales non linearly with subagent use.
It would be true if there was a unified "Anthropic" entity making every decision from pure rationality. Instead, more tokens increase Claude Code team's metrics of token usage, which most likely has a KPI around token usage and adoption.
To remind Goodhart's law: "When a measure becomes a target, it ceases to be a good measure".
..also to parent's point, yes the upsell is only appealing once user run's out of tokens.
> This is supported by the fact that they won't let you use your sub on a different coding agent
I mean, that's a very weak argument? Isn't a much more plausible explanation that with your tooling you'll have more of a lock-in than with just your model?
This isn’t limited to large system prompts. Coding-agent harnesses are also becoming more aggressive about using tools, even for trivial requests. In our tests, prompts such as “Hey” or “commit” sometimes triggered 30+ tool calls:
I often find myself annoyed when Opus fixes a typo in a comment and decides to run tests, lints and whenever else it can find to run. Often it will start by stashing current changes just to preemptively check if all tests were passing before.
And I can blame myself a bit because my rules do say: verify all changes with tests. But as there is that I in AI that is hyped which you’d think means it knows not to put tomatoes into fruit salad …
> [..] my rules do say: verify all changes with tests
I am a bit surprised that you're disappointed that it does exactly what you told it to - people usually have the opposite complaint.
If you're using it interactively and watching what it changes, I'd trigger the tests when you think it's needed. And if you want to go more hands-off, why not add try to encode the same nuance you'd use into the rule?
Maybe, depends on their workflow. In my human workflow, I tend to use commits as checkpoints and then squash before pushing. I'd usually only run time-consuming tests before squash+push.
But yes, anything you want to ensure really needs to be a hook.
edit: realizing with "precommit" you probably meant a git hook not one in their harness. I'd have written the same response more or less though. :)
Following rules like "verify all changes with tests" down to a tee is usually a desirable trait in LLMs. Personally I'd leave that behavior there (just like with humans for some tasks like aviation you have them go through checklists even if some stuff you can infer is not needed). But otherwise just make it "always run tests unless you're absolutely sure they can be skipped".
That's one of the reasons I started https://beolis.com. Now I have a workflow that says to do things in a TDD and run only new tests and related tests but NEVER run the full suite and then when it finishes the workflow runs all the tests -- if it works, great, if it doesn't then the workflow continues, feeds the output to a cheaper LLM to summary the errors and then get another run to actually fix it, based on the failures and the context based on what should be implemented.
-- note: I've been full time in Beolis for some months already, feedback welcome ;)
> prompts such as “Hey” or “commit” sometimes triggered 30+ tool calls
I read that this is because it wastes time looking through past conversations and other context to figure one what you might want it to do - a less ambiguous prompt would be better.
After reading PUSH_AX's valid comment:
```
This is like saying contractor (A) asked for $33,000 to undertake the work and contractor (B) asked for $7,000
Are we measuring and caring about the right thing?
```
We will update the post to include:
1) A more in-depth task.
2) Qualitative results comparison.
3) As soon as possible, a reproduction of the inputs and outputs.
This isn't accurate since the main reason I'm using Claude Code instead of these other interesting sounding harnesses is the subscription service with highly discounted token usage. With OpenCode, you're paying the full price.
Therefore, you should include the actual costs associated with the task in API token usage or subscription level. Is there a reasonable way to do apples to apples cost comparison?
Maybe related to this minimalism, Pi doesn't come with most of the tools an LLM needs to function efficiently or effectively. I get that a blank slate is the paradigm, and you can add whatever you want, but it's too blank IMO.
I have a functional Pi config, mostly self-made (it has everything I want, incl. subagents, web search, a /btw command, and other misc. addons), and my system prompt is ~3k.
what tools are you talking about? Pi has ALL tools the LLM needs to function efficiently and effectively for coding tasks. It can read,write,edit files and can use any bash tool to search files, execute tests and so on.
Every time I read this comments I have the feeling you are talking about mcp or sub agents, otherwise this makes no sense at all.
That will increase the amount of initial tokens used, because the tools have to be described somewhere. Maybe not as much as Claude Code, but it could get more if you just randomly keep adding tools.
I tried using omp, and really like the interface, but I found it used tokens much much quicker than the Claude cli. Some simple tasks would use all the session tokens in less than an hour, as where I could get easily get 3-4 hours with Claude. Both set to use opus 4.8 auto effort. I tried tweaking the models for agents down to haiku and sonnet in omp, but didn't notice any real difference in the speed tokens were being used.
if the website is any indication, omp is just vibe coded AI slop as well. pi is awesome but it looks like omp is a step down including unnecessary stuff no one needs. it's kinda the opposite of pi
I've heard amazing things about pi and it's effectivenes but when I tried installing it I quickly found out it doesn't respect XDG_BASE_DIRECTORY at all, you need to set some environment variables and the author rejected both a proposal as "going full gpt", seemingly not even knowing about XDG_BASE_DIRECTORY, and even rejected a PR.
I've heard really good things but that being my first experience with pi didn't fill me with confidence about it's code quality either.
For now I stay with OpenCode I think - I was using zed editor and agent for the longest time anyway and think I will go back to that. CLI tools for me seem a bit too disconnected from the code.
Interesting! I wouldn't know, because I'm on mac. I love that it's very basic and I can basically do plug and play with the stuff I need. The load, complexity and system prompt overhang is quite minimal. But in the end, opencode is also fine I guess. I don't need to fiddle around with it too much and just need it to work. So if you're happy, why switch ;)
I actually use zed as an editor w/o agents lol
Sure, but you have to add almost everything, no? It deliberately only comes with read, write, edit, and bash. My point wasn't that you can't add stuff, but that I'd just rather use an harness that's a bit more full featured from the start.
(Pi is a bit like old 3D printing where fettling the printer to work is a central part of the hobby. I'd rather just buy a Prusa.)
The main ones missed immediately were web access/search. Then the to-do list features (it was a nice surprise to try OpenCode and see this working immediately.). There were a couple of other niggles but it was a few months ago. Also, this may not be common, but it seemed to struggle to edit effectively (driven by Qwen 3.6 35b/27b) and often rewrote whole files instead.
Gotcha, I feel like model or provider-specific installs would be a nice QoL improvement in that case. Presumably, part of this issue (beyond the ethos of minimalism) is the aim of shipping shipping an agnostic toolset. For myself, im openai-first, and of course that pushes me to favor their hosted tools (in this case web search), and their native CPT/RL'd stuff (Ie apply-patch).
Though, imo, the fact that pi maintains its "we only include the bare minimum!" statement is part of the draw for me. Especially considering that im in an enterprise env; being able to internally share custom implementations of out-of-the-box Codex/CC stuff is really nice.
I do wonder how they'd go about shipping a default web search tool. Big problem there is the lethal trifecta. Shipping something that arbitrarily allows untrusted content to be retrieved non-deterministically I'm sure is a long conversation on Pi's end. Pushing it off to the user to decide is easy.
I know it's against the ethos of Pi, but I think a lot of people would consider a handful of things like a memory system, web search, subagents, and looping to be a basic/base thing they would add to every agent harness.
I use Pi and love it, but the base install is extremely minimal (a handful of tools, no subagents, etc.). That’s on purpose. Sure, you can add more with packages, but that’s not the base install.
If you really want a minimal agent that you heavily customize, just skip pi (130+ transitive dependencies on the "minimal" pi-coder package) and write your own. You learn a bunch, and it's not hard. You can even ask another LLM to help you get started.
I wrote my own harness in Emacs and it’s completely ridiculous how well it works. Auto-compact is the only missing feature on my list. Claude‘s approach, if I understand it correctly, invalidates a lot of cached context, and I‘m thinking about a more cache-friendly strategy.
Exactly! I just vibe coded (with GPT Sol and Claude whatever-number) my own agent, it's trivial to add now any feature I want - simply ask more powerful model to do it for you. I am happy with end result, however it looks indeed these tools are trained to increase token count - they do quite stupid token-spending steps while making code, but the code itself is also a bit weird - it's like they intentionally do code which is hard to modify on your own without using exactly those authoring models. Interestingly, when I am using DeepSeek with OpenCode, I don't see that - it understands my intent well enough and overall code quality is not bad. I recently switched to local Gemma 4, and I often switch (in opencode) to just that less powerful model, because it understands my intent and has enough skills to provide good quality solution although it's rather for small size projects, and for not coding from scratch, but it's also free and private. It feels slower than any big cloud model, so my model switching is probably most quickest path to robust end result :)
That’s certainly doable, but then you need to create all the add-ons you would have added to Pi. IMO, Pi stands in that sweet spot between being very minimal while still offering a catalog of pluggable functionality that you can add to it. Sure, you could vibe code all those things for your custom agent as well, but why recreate what is essentially Pi all over again (the main loop with all the extension hooks, etc.)? Pi is the “standard” batteries-not-included agent, where you can add from a catalog of pre-defined batteries. I recommend starting with Pi and then using an LLM to code custom extensions where the catalog doesn’t have something you want.
Pi has way too many batteries included, including a bunch I don't want, and lacked the batteries I did want. Pi is a bit like the movie Idiocracy in that the idea is much better than the execution.
Incidentally, I also have zero supply chain attack surface as I have zero dependencies in my agent, just go stdlib. Pi, again, has 130+ transitive dependencies asking me to trust the security of my system to 150+ additional people I've never met in exchange for a bunch of bloat I do not want.
Bloated TUI library, unified multi-provider LLM layer, bloated RPC and SDK modes, the entire plugin framework, the list goes on and on.
For reference, pi-coding-agent, by itself (not including dependencies, tests, or pi-ai, pi-tui, pi-agent-core, etc), is ~41,653 SLOC taking up ~1658.9 KiB across 163 files.
My agent, excluding dependencies (all go stdlib) and tests, is 3 files, 946 SLOC, taking up 36.3 KiB, and includes a basic TUI and an XMPP transport channel (including TLS for XMPP), with dynamically configurable delivery to and receipt from either or both, including allowlists for XMPP message partners. It has tool calling, a permission model with whitelisting and interactive permission querying on a per-tool basis, full thinking support, including the ability to toggle hiding or showing it across either or both transports repeatedly throughout an individual session, the same tools as pi comes with out of the box, plus web search, and a tool to vet, build, and git commit golang projects all in one go, stopping if errors are observed. Configurable model and endpoint, too.
Incidentally, the open source xmpp server (prosody) and metasearch engine (SearXNG) are both self-hosted, too.
I guess better phrasing would be auditability, ease of codebase comprehension, coverage of just the features I want. My agent isn't meant to be for X users across Y providers with Z extensible plugins, it's meant for exactly one user, with exactly one provider, and to minimize the amount of trust granted to third parties.
I have written a few myself to get an understanding.
To learn yourself:
<$20 on a cloud AI api for a chunk of tokens and have the AI teach you. "help me write an AI Agent using (language) and walk me through the steps"
Realize that these agent are REPL/while loops that maintain a conversation state and then based upon the tagging syntax like <TOOL:bash:uptime>uptime for system run time</TOOL> and the agent extracts the tool and then does sub commands.
At a minimum, you need an inference endpoint: either cloud or local.
If going local, llama.cpp is going to be the more beginner friendly local inference engine that supports more processor types (AMD GPUs, Intel GPUs, CPUs, anything that supports Vulkan, not just Nvidia). LM Studio is a nice wrapper for this if you'd rather avoid cloning repo and compiling yourself, provided you don't mind closed source software; it's much less enshittified than Ollama.
If going local, you will also need model weights in the right format for your inference engine, and with a model that can fit on your hardware. This is going to be .GGUF files if you're using llama.cpp or a wrapper for it like LM Studio.
From there, pick a language, go look up the OpenAI /chat/completions API format (or Anthropic's "Responses" API format), create a DS or array or slice to store messages, and build a loop that accepts user input, formats it according to the API format, sends it to the inference server, retrieves and parses the response, adds the response to the DS/array/slice, and repeat.
There's a lot more beyond this - tool calling, other API formats (optionally), MCP servers, transport layers besides terminal stdin/stdout, permission models, starting with a system message, clearing your message stack correctly (hint: don't reset it mid tool-call), message compaction, web searching and page fetching, semantic search RAG over embeddings, memory layers - way too much to cover exhaustively in a single message.
Recently switched to Codex after 6m in Claude. Codex seems more open, it’s easier to follow what the model is doing and the approvals have a better UX. Overall, it just feels more transparent. Cost of switching was close to 0.
I don’t like that Claude became more opaque around February, including the system prompts. 33k feels way too much.
I use both now and agree they're basically interchangeable.
I appreciate that Codex is open source and OpenAI has explicitly said using the subscription with other agents is ok. OpenAI has been much more consumer-friendly recently.
I like Codex for allowing auto_review.policy (basically a prompt to the classifier on what to allow/disallow) to be configured rather than the opaque auto mode in Claude.
They’ve been hiding their thinking tokens more and more, and lately also which tools are being executed when and in which way. It makes it more difficult to assess what it’s doing and jump in to steer it into a different direction in realtime.
CC went from sane defaults in late 2025 to feature scope creep early 2026. So more features might be good, but sounds like an ick for me. But I have zero prestige, I might switch back.
Worth noting/understanding, for those that are not familiar with it, that these 33k tokens are not a single 33k batch of tokens, they persist and increment in every request.. so if you have 10 requests that is 330k tokens, 100 requests 3.3M requests, 3.3M that could be ~5x less if used another harness.
These 33k tokens are mostly because the harness is completely bloated with mcp, skils, plugin, loop, basic alignment and all sort of explanations needed in the system prompt for it to work the way anthropic wants it to...
imo this is sub-optimal and if the model needs that much orientation, it says a lot about its intelligence too.. Good models should be as harness agnostic as possible
I am forced to use cloude code at work but a good solution is to just use --system-prompt "" and be done with it. I wish they allowed for other harnesses.
Does it have any negative impact? If not, I’m not sure why this wouldn’t be the default behavior. It feels like Anthropic is just putting their foot on the scale to drive up costs or for the enterprise, or push consumers to higher subscription tiers.
We should discuss cache performance if we haven't already. That 33k tokens may be a cache hit (I am not certain it's automatically a cache hit) but after the first call, it should certainly be a cache hit. Cache hit tokens are billed at 1/10th the price of cache misses. This is quite opaque, but it's necessary when you're asking "is the system prompt worth its stay" if you can save 33k tokens worth of dynamic discovery across the next few turns, the break-even point is quick and if the system prompt makes task performance increase and/or makes the system more autonomous so that it can string together more cache hits in a row, it becomes way way better. On a personal note, I think of things as aa function of 'supervised time to desired result' and 'cost'. because I find it harder to reason about tokens. I do think they could introduce a "minimal" mode (something like this is probably doable with the Claude agent SDK today)
Anthropic's cache expires after 1 hour when using subscription endpoints, and for those cached tokens cache reads are free. It's generous (compared to API pricing) but it's not 100% free.
I believe it's 5 minutes on API pricing by default, though you can turn off caching or force it to 1 hour. Subscriptions are special, seemingly because Anthropic doesn't want to expose casual users to all of the tokenomics.
It is not the raw prompt size that matters ultimately, otherwise Pi (and variants) would be the lowest costing agents. What matters is how efficient the prompt it. Prompt minimalism often gets conflated with efficiency. Having said that, CC does seem bloated for what it does.
What matters even more is tooling quality. Bad/buggy tooling causes a lot more roundtrips that wipes out all gains from initial greedy approach.
A few months ago, I did a full benchmark run of 7 agents over 8 tasks (https://github.com/dirac-run/dirac has the data and traces). I cannot claim neutrality because of the obvious connection to one of those, but the data should be reproducible and useful. Importantly, Dirac wins there significantly on those tasks because they are mostly refactoring related (which is where approaches like hash-anchoring and AST parsing tend to shine)
The good thing is that the competition in the field is very cutthroat with so many contenders, so if there are gains to be made they will be made, and then broadly adopted by others.
A harness is a part of the intelligence stack. It's no longer about raw access to the model
Also, I have seriously used most harnesses - One feels like it's being built in a place that truly understands AI and where agentic engineering is headed. You might not like it, but peak performance exists in CC when it comes to orchestration of bulk parallel work / subagents. The open source agents are catching up or accell in different way (Im preferable to pi.dev), but I'm not sure they're architecting orchestration the right why.
Early on in experimenting with local models, I found that hooking them up to Claude Code worked very well, but it was also really slow.
I used mitmproxy (setup assisted by Claude, natch) to capture Claude Code's entire initial system prompt and the whole thing was (I just double-checked) 162k of JSON.
This led me to start experimenting with Pi, OpenCode, and Hermes...
4k tokens is 15-20kB. I'd ask you to paste that into a gist, but it might have sensitive data in it, because I suspect what you're seeing is not just the system prompt.
Apologies, you're right - I used imprecise terminology. The entire initial JSON structure that was sent from Claude Claude to the LLM at the start of a session was 162k. This included the system prompt together with a list of tools (some with very extensive explanations), MCP server details, etc.
I was simply supporting the article's data - their reported 33k tokens is probably roughly 150-165k.
That’s entirely dependent on how many plugins, MCP tools, agents you have, and if you have pre-filling of all available tools enabled. Best way to avoid unnecessary expense is to avoid it all and use CLI tools instead.
A smarter approach (progressive disclosure) for tools has been implemented by (I presume all) the harnesses over recent months, but you're 100% right in any case.
I enable tools specific to each project only in that project, and have very very few in my global config. Like <5k tokens worth.
With Fable being per token instead of on the subs (unless they changed it again?), I decided to test Claude code on OpenRouter where I had some credits, with Opus 4.8 and Fable 5.
I asked both a trivial question (summarize last commit). Opus cost 50 cents, Fable about $1.
That checks out because Fable's twice as much in the API (though I think its emphasis on correctness makes the difference larger for bigger tasks).
But, at $1 per question, I think I will stick to the subscription for now! I was certainly glad GPT-5.6-Sol is included in OpenAI's subscription, and I'm curious if they'll be able to do the same for GPT-6.
All the VC money appears to have run out a few weeks ago.
As for context size and harnesses I did make a trivial bash agent based on this "agent in 50 lines" tutorial[0] recently, and found that for trivial work, it was about an order of magnitude cheaper and faster.
I haven't tested it on anything bigger but it doesn't seem to do the kind of proactive testing, that they do in bigger harnesses.
Codex at least has a system prompt that tells it not to consider a feature a complete until it has verified it. I'm not sure about Claude Code.
I suppose I could add that one line to the prompt, and it would get me much closer to agi :) I think Fable does this proactively even without a prompt, but I haven't tested that yet.
If Fable in my own harness is significantly cheaper than Claude Code, that would be very appealing. (I could actually afford to use it for most things!) But I think most of the cost comes from the testing it does. So we'll have to see.
Fable's subscription inclusion theoretically ends EOD today. Anthropic put a wishy-washy "if we have capacity we'll continue it" thing, and given how competitive GPT 5.6 Sol is, and it is included in OpenAI's subscription, I fully expect Anthropic to extend Fable or they will have a serious exodus on their hands.
Anthropic have extended Fable access again to July 19. The notice should pop up in your Claude Code now when you start a new session (also announced on the ClaudeDevs X account first).
Ah, thanks. It's been hard to plan around these last-minute changes. I rushed to implementation on a spec I should have spent more time on because of the looming deadline.
> A small task that cost 121,000 tokens done directly cost 513,000 tokens when fanned out to two subagents, because every subagent has its own bootstrap cost, and the parent then consumes its transcript.
Is that true? My understanding is that the subagent only returns the result of the request without the main agent consuming its entire transcript.
Interesting question - what I learned doing that is that sol ends up injecting its thinking traces as code and shell terminal comments. YMMV, but in my case what I saw was sufficient to use with my routine work in my projects
I've been trying various harnesses like Pi, OpenCode, Qwen Code, and Nanocoder. A common problem I keep running into is failed tool calls, regardless of the model. What is the best harness and on-device model combination right now?
Pi.dev requires some plugins to work well. Using Qwen3.6-27B/35B locally at Q8, I was quite frustrated with failed tool calls and tried many things.
Ultimately this combo worked:
1. https://pi.dev/packages/pi-tool-guard —- corrects key name synonyms and common structure errors, so tool calls succeed automatically (e.g if the model hallucinates old_str instead of oldText). It also wraps top level oldText/newText in an edits array if the tool didn’t do it.
Hashline edit tools didn’t work well for me at all, they confused the model and it still failed to edit correctly. Also line removals would invalidate the rest of the file requiring re-reads. I tried pi-hashline-edit-pro, though I see it now keeps a database of hashes to help keep them stable across edits. Regardless Qwen kept thinking that the hashline prefixes were part of the source.
I'm on a 48 GB M5 Macbook Pro. I use 4-bit quants with a context window of 16-32k. I tried Qwen 3.6 27B, but I can only get around 10 tokens per second, but it's painfully slow, and it often fails during `write_file` tool calls, even with Qwen Code.
Try an 8 bit quant of Qwen 35B, but temper your expectations. Those Qwen 3.6 models are impressive for the size, but you need an order of magnitude more parameters to actually be useful for more than trivial work in my opinion.
That would depend entirely on what your device is. This sounds likely not to be an issue with the harness, but the capabilities of the models you've tried.
I experience almost no tool call failure using my nothing-special harness and DSv4 Flash.
I have just re-analysed most common failed tool-calls and adjusted the tool so that it works. I have a manual repair step on failure that programmatically attempts to fix some things. On failure, the harness reports the error, the repaired function, and the result. Overall, seems to work fine. But it's very model-specific. Most commonly the model fails on shell commands where it hallucinates some programs. If it does it often enough, I just promote those to commands in the PATH. Over time, it has happened less.
The Sleev (the project has been renamed to make a startup) creator was shilling their project in the OpenCode Discord. That person is very convinced they have something that no one has ever built before. They focused on token reduction without any real evals for capability impacts.
I'm generally against this context pruning without prompting or details. Sleev is very opaque about how it works and definitely will bust your cache.
It's definitely not unprecedented, but the plugin version is useful. Sleev seems like a nothingburger, I'm happy with the results I get from DCP already.
right, but the cache retention time is very short for Anthropics LLMs.
5 minutes or 1 hour (with additional costs).
So you have to prompt basically non stop to not get a cache eviction.
A const prompt across all of Anthropic's subscribers could draw from a global cache rather than per-user?
Although saying that out loud makes me question it - each per-user chat and growing cache would need eventually to own its own ~contiguous memory block.
Noob question, but is there, on a harness level, a way to compress or optimize the "pre processing prompt" or any other preamble?
I know you can give instructions to guide an agent, but they are still plain English, the interpretation still isn't absolute and the more rules, the less likely it is to respect the whole set of rules in the preamble, but the whole process seem so... "Unprofessional"?
As I see it, a tool should be reliable and the less bloat, less possible points of failure.LLMs seem overly verbose and you can't shake the feeling it might veer away from the objective at any point. You can pay to mitigate or work around the risk, but it still doesn't inspire confidence in anything with more than 30 lines
I pretty much giving up on Claude. It's Extremely slow, especially fast few days. Computer use takes so long to do anything that sessions expire. I switched to Codex and it's a different story.
I'm not saying that Anthropic is not going to fix their problems, but for me as a user that depends for the service to work, it's not acceptable. So, we switch back and forth between a few LLM's but Claude and Codex are the leaders and I can't afford using anything else if I know I can get better quality results. The cost is important but not as much as the quality of the solution. You might save a few bucks on a cheeper model but you will have to prompt it many times and time is more important to me, as well as the feeling that I'm getting the best result on the first try.
Sending 0k tokens would be a smaller number again. But then it might have no idea of what it is doing. I pay my subscription and get lots of tokens on good models - if I was paying per token I might care more.
In a pay per token situation, there is a huge conflict of interest with the harness provider and the token seller being the same party ... efficiency is less profitable.
I have accused claude code of trying to run up the meter on me and it confirmed I was absolutely right.
> In a pay per token situation, there is a huge conflict of interest with the harness provider and the token seller being the same party ... efficiency is less profitable.
Except there’s a competitive incentive to either use less tokens or make the tokens go further
We are still in a (brief) era where companies are awarding pizza parties to the employees that burn the most tokens.
There is a world where, to hit the next quarters revenue projection, you add 2,000 tokens to the system prompt and "beat" expectations at the next earnings release.
This is posed as some sort of discovery, but both Claude Code and OpenCode display token usage clearly after starting a chat or agent, and 30k and 7k is exactly what you see.
This is all heading in the right direction. Much of AI coding feels magical. But when the costs begin to accrue we start asking questions. We dig into it and try to understand what's going on. I can't help but feel Anthropic is "token maxing" from its side: it controls the levers and with every version upgrade it can build in its own token growth almost unbeknownst to the user. This actually harms it on the long run because it necessitates a cheaper option.
I've been using Claude since January, and whenever I run CCUsage I've been noticing the overall cost creep up pretty much every month. (I'm on a subscription so this would be the hypothetical billing if I were on API prices I suppose. Although I don't really want to test that with my credit card.) The funny thing is the first few months I was token-maxxing as hard as I could just to see how awful of a bill I could run up (mostly as a curiosity thing). At this point the novelty of doing that has worn off for me, but even with me being pretty conservative in my usage now the cost is way higher. I think I was spending like $12 a day in january, and now I'm easily spending $60+ a day in part time work. The amount of work I've been doing has stayed relatively constant (I'm not trying to run agents in parallel or loops or whatever fancy new ways there are to burn money, this is the same workflow I've had since the beginning. The codebase has grown, but I assume input tokens are not to blame for the big cost increases)
One honesty note before my comment (yuck), it's super frustrating to read an LLM produced blog that could be 1/3 the size. Why are you talking about MCP's when comparing OpenCode and Claude Code when they both support that technology?
The only interesting thing is whether OpenCode is more effective at writing code with it's reduced system prompt.
Claude Code sending 33k tokens before reading the prompt is the AI equivalent of a consultant who bills you for the time spent reading your email before they even open it.
The OpenCode CLI does not work as well for me as the PI CLI.
I'm a subscriber of OpenCode Go (the sub, good value for me really) but I had not great experiences with OpenCode CLI.
It multiple times with different models deadlocked itself into listing endlessly to non ending processes (Android Debugging Bridge, COM serial log, ...).
There was also a problem where the OpenCode CLI would crash after sometime with a Bun error.
I switched to the PI CLI and have no problems with hanging processes anymore.
OpenCode Go allows for API access so I'm keeping this sub.
I'm surprised most of that isn't cached token usage. It's true that increasing length is a problem on its own because the model needs to attend to it all, but with caching it should be pretty fast anyway. My system prompt is quite large and I haven't noticed much of a generation penalty in the range from 5k to 10k.
The reasoning built into the models matter so much too. I recently swapped my Qwen3.6 27B to ThinkingLabs’ fine tune and it does what it publishes. I cut my token usage in half, which is a big deal since I only get ~20 TPS for token generation.
We've started using claude code at work and I don't understand the hype. I've been using codex and grok build at home, and they're both faster and in some cases better. Claude has a tendency to do too much. If I don't ask for unit tests and they're not in my agents.md file, then I probably don't want them. It'll try to make new libraries and classes for things that should just be a new function or a comparison check.
In our case the alternative was nothing so I'm happy to have it, but currently claude is not as competitive as I'd have maybe expected given the hype
Its funny, I have seen people look at the sourcecode of opencode and complain about excessive token use, the code to tell the model how to execute bash commands, and they complained how this was way too excessive, teaching the LLM how to write bash.
Would be interesting if you could post more details about what you captured.
Claude code seems to have a pretty robust harness and memory. It's a time saver to not include project context in our prompt, since Claude manages claude.md but I haven't used OpenCode to compare.
No surprise, I've noticed that "agents", not only CC (I am using Copilot) are trying to be "clever", searching for a lot of data. This is good for LLM providers as this eats a lot of tokens.
I've been trying out OpenCode recently, because of the US embargo on frontier models, and found it to be as good as Claude Code, if not better. And it read all my skills, claude.md files, etc. Now I just need to pick a model out of all the choices - currently Deepseek v4 Pro is winning, but I want to try a few more.
I still think the best way to build software using LLMs is to copy-paste snippets/files into the chat and manually guide the work. Humans are still the best orchestrators. Yes the human has to now be hyper-focused and juggle various workflows, but the end result (quality of work and throughput of usable code) becomes very good.
Been my experience as well. Human corrected code is orders better than ai generated slop. Slop might get you the promotion at your day job who is still in fomo tokenmaxx mode, but highly recommend the careful review for your personal projects
I find it hilarious people think they can build on slop or on captured output from quality engineers. Intent is the most important aspect, and no code base could ever capture that
Opencode did have an adapter to use Anthropic models before they were sent legal threats that scared them into nuking the repo.
Remember is it not OpenAI vs Anthropic as bad guys vs good guys. They are all bad guys trying to profit from your data while maximizing dependency. Just buy or rent GPUs.
I used both with Openrouter and GLM 5.2 and I can confirm this is the case. Claude Code burned $10 per task while Open Code burned barely $10 a day which wqs about 4-5 tasks a day. A task usually included database migrations, code audit or documentation.
Claude Code is not just a harness. It is a different product. You pick the smallest subscription that allows you to do your work. My “multiplier” on a $100 subscription is 5+.
If you’re using API, on the other hand, there is absolutely no reason to use Claude Code, or Codex.
But Claude Code in my experience results in more tool calling for smart efficient file reading. Meanwhile Opencode pulled an entire 500kb file (GPU assembly dump) at once. Kilo is better than both, as it uses indexing.
> Claude Code 2.1.207 and OpenCode 1.17.18, both pinned to claude-sonnet-4-5
So not only is this article AI-written, but the testing was entirely done by AI, too? I can't see any other reason to use such an old model.
> Our traffic passes through a local LLM gateway that wraps requests in its own envelope, a constant we measured at roughly 6,200 tokens with bare calibration requests
Why do you need to do calibration requests to figure out how your own gateway is affecting requests?
> Its subagent lane did not complete cleanly through our gateway
> We attempted to toggle extended thinking in both harnesses and are declining to publish numbers. Our gateway applies its own thinking policy, neither harness's toggle demonstrably survived the path, and anything we quoted would be noise.
Why is your own gateway screwing with your testing?
Cost, mainly. The runs went through a Claude Max subscription rather than metered API billing, and pinning an older stable snapshot kept run-to-run comparisons clean and cheap. The fixed harness payload (system prompt plus tool schemas), so the headline numbers shouldn't change too much.
That said, happy to re-run the matrix on Fable and publish the diff; payload figures should barely move, tool-calling behaviour might.
Gateway:
Meridian (github.com/rynfar/meridian); proxy that bridges the Claude Code SDK to a standard Anthropic endpoint so a Claude Max subscription can drive OpenCode-et-al.
It's the auth route for all agent traffic on the machine, not something built for the benchmark.
I recently started using cline instead of opencode and prefer the interface. I'm interested if anyone here here has any arguments for opencode or codex over cline?
OpenCode, Crush and Pi do have the ability to fork a conversation. But cache reuse is up to the provider and not guaranteed. At some point you need to forward the cache to a more recent checkpoint, and you have a finite (unknown) number of parallel cached chats.
I feel like this article isn't saying much. Even with tools disabled, Claude Code still has a crap load of commands and other things that Claude (the model) should know the availability of since it's optimized for them. All of that has to be disabled if this is to be a real harness comparison. And of course the system prompt can be completely replaced, making it a no-brainer to use a more minimal prompt similar to OpenCode. And beyond that nothing else really matters because the rest (cache behavior, etc) lies with the provider's platform, not the harness.
Does your detail plan outline exactly what to implement/ lines of code to edit? I found if I get a nice detailed plan that sonnet is good enough to implement. Did you try that before and found fable better at implementing?
Yeah. So the benefit of Fable is that it'll find gaps in the system as well. So logic errors get caught. I have a separate backend and frontend so plumbing issues are also caught by Fable.
Anthropic wants to produce the best coding agent possible and doesn’t care (is even incentivized) about high costs. Other harnesses have to make trade offs between performance and cost.
Given they're incentivized to increase token use, what guarantees that higher token use improves the effectiveness of the agent and isn't just artificial padding?
Well, nothing really. But I assume there can be some benefits to modifying context. For example, updating file contents or marking them as modified, summarization, injecting additional information, removing irrelevant tool call results, etc.
Grok 4.5 is really fast, has more usage at $10/month than $20/month Claude pro, and Opus-level. Claude pro feels like a demo.
Claude is much better in OpenCode then in Claude Code, OpenCode is just better than Claude Code. Claude Code feels like a complete mess to use comparatively.
True of anything anyone says about anything, including int_19h and simondotau.
I'm quite impressed with Grok 4.5 because its speed and single-task effectiveness feels better than anything else for a human-in-the-loop workflow. (For the stuff I do, I'm not interested in having AI race ahead of what I can oversee.)
All models do things in a way I personally disagree with at least some of the time. The "dumber" models sometimes fail to recognise how to fit a solution into existing code. The "smarter" models sometimes get too clever and over-engineer their solutions. Cleverness is occasionally interesting, but is just as likely to trigger a distracting rabbit hole where I spend time analysing whether something unexpected was a legit insight, or mere opinion.
What really burns tokens is sub agents. I once gave Claude Code a pretty big task, and it immediately launched 7 sub agents which burned through my budget before even one of them was finished. Tried again 5 hours later: same result.
If I let the main agent do the same task sequentially, it was no problem at all. I don't know if it's really just communication and orchestration that makes sub agents so inefficient, or if Anthropic figured that most people using sub agents pay per token on a big corporate account, so this is an easy way to make more money from tokenmaxxers.
As a counterpoint: in a complex project, Fable's "curiosity" may be exactly what you want for an exploration and planning stage - not just for the orchestrator that turns your prompt into different angles with which to explore, but for each subagent whose task is to search the codebase for one of those "angles." If you truly want no stone unturned, letting those subagents spawn their own discoveries, and recursively grow the surface area of the inquiry, then it's quite reasonable to want Fable throughout.
That said, if your project is "do this well-planned thing on a bunch of things in parallel" then you should absolutely be instructing to have subagents "step down" to less curious models. Their output may well be more cohesive as a result!
The curiosity is inefficient though. So many times I have to stop the agent and tell it to just fucking write the code and try compiling it. Otherwise it will fill its entire context tracing through the program logic to derive from the code itself whether the thing it is about to do would work. It completely fails to notice it can just… try.
Everything about LLMs is inefficient. They have their benefits but watching them reason over things that are painfully obvious, that they've literally investigated before (before a memory compaction), never take a step back aand be like 'this is going too slow let me look for a better way', etc. is painful.
It’s got worse though right? Older models from before everyone went off the deep end with CoT don’t do this and just write the code with 1/10 the token usage.
The downside is the code isn’t as good but it is produced a lot faster and more cheaply and often it’s actually fine.
CoT has made LLMs better (say 50% improvement or something) but increases cost by an order of magnitude. That graph is going in the wrong direction and has been for a while now
CoT?
Chain of Thought, according to a quick search
I think I use it differently. I still mainly stick to web UI.
I write a good prompt, paste the code then copy the output code and place it into my project.
So in the end I hand assemble and I only give it what it needs to know so no extra context wasted.
The human in the loop is of course the secret sauce but this way I am highly efficient, no vibecode and I work really fast too. Everything is audited.
This is how I worked with LLMs originally, and I much preferred it. This gave me a much better understanding of the code that I was adding. But, there's no way to keep up with my team like this anymore. It's just too slow when everyone else is working directly in Claude Code.
If the entire team is vibe coding and there is no human audit then there is no way but to vibe code, for sure.
I would also just vibe it if there is no responsibility, but if I do it that way I don't even care what happens with the project.
I get so detached from it that I stop caring and if it has huge critical bugs..I just don't care anymore because it's not my responsibility or my code at all at that point. I'm just there to nudge things along.
Just hook it up to Jira and let the managers add the features then pass it off to QA.
Real engineering is fully automated at that point.
> I just don't care anymore because it's not my responsibility or my code at all at that point
Yep, 100%.
Business has made it clear they don’t care, so there’s no point in burning one’s energy. Throw the whole thing on auto, check out, and go do something else during the day.
> Throw the whole thing on auto, check out, and go do something else during the day.
If that's what you're doing, you're fucked (in today's society, at least). What happens to your job when they figure out that's what you're doing?
Plenty of companies, very much _want_ this.
I agree, that work in these places is likely short lived, if for no other reason than working in them is awful and demoralising. My point is, that these places exist, and have such a hard-on for “oh my god, AI!!!1!1!” that putting in extra effort there, is a waste of your own energy.
They need somebody to blame though, and you're much cheaper than a consultant.
I'm just thinking: The result is net-net the same. Either they terminate your contract as a salaried employee or they do it as a consultant. The fee is higher as a consultant and you never have to say you were fired. You're making a really good argument for getting into consulting.
You can just answer the same rote questions to different companies for max salary, then when the going gets tough you can essentially 'fire' the company and say they've moved past your core consulting paradigm. Maybe suggest a new consultant and move one. Rinse; repeat.
Are the QA team bearing the brunt of the unexpected issues, bugs, performance etc or is it business-as-usual?
Amusing that you think businesses still have QA teams.
Parent comment said they end up "...pass[ing] it off to QA."
Edit: suppose that very well could have been tongue-in-cheek.
Side note: as someone who has been interested in programming for a while, but didn't end up in a software dev track in life, it's been pretty wild watching the ride you all have been going on lately. I used to be pretty bummed I didn't get to do that kind of work for a living, but lately I've been feeling more and more like I dodged a bullet.
Not that I don't have my own AI-related junk I have to deal with where I did end up, of course. I think most have.
Curious what industry your in and what your facing AI wise?
They do, it’s just that these days they are your users or customers.
That is indeed, the joke.
Mine does (Sample size: 1)
My employer has a QA team
Wow, this is almost Dilbertesque level of absurdity.
I like it but how much context does it need for a complex program? If you're giving instructions and using its code, I imagine context is being passed back up in an exponential way. If not, and you give it a very thin context every time, how do you manage to prompt it enough?
Avoid making programs very complex. They can grow big and have a lot of features, but stay as simple as possible.
Depends what I want but I can give a completely new context for every generation.
I try to make everything as simple and human readable as possible because I want the audit to go fast.
I think for me I lean towards an audit optimized approach. Everything is still generated but revolves around the human-in-the-loop for review.
It's tuned for the kinds of tasks where "just try" doesn't get good results.
A major complaint with AI code was that AIs struggle with complex codebases, don't respect existing conventions, reinvent functionality multiple times over, etc. So, newer high end AIs are tuned with the "explore/exploit" dial turned towards "explore".
You could probably get it to do things "quick and dirty" with prompting, but that, of course, requires prompting for it.
I get what you're saying, but these instances are of a different type. It is along the lines of "if I pass None for this parameter, will it default to X or return an error?" and it looks, but finds that the actual logic is distributed across multiple files. And so it quickly falls down that rabbit hole.
All the while, it could have just done a two-line probe test and see what happens when it calls the API with "None" for that parameter. Or just assum it would act as expected and wire in debug logs in case it doesn't.
Perhaps what is missing is a better memory/caching layer to avoid doing the same for explorations over and over again.
That is the usual work of high end programmers, right? Growing codebases as consistent, dependable ontologies?
I feel like most mainstream programming languages do this sort of work for their standard libraries and their official docs. Go and Python come to mind, but plenty others do this reasonably well to the point where one mostly doesn’t need to read the implementation code to effectively use the standard library itself.
I use the human-in-the-loop for managing the context.
Give it only what it needs and do things usually 1 file at a time.
Feels like I'm a sort of manual tape editor, if the context was a tape fed into machine, I assemble that and then watch the machine output the results I need.
that's why i mostly use it for asynchronous work, the inefficiency is something i can bear with because the subscription costs are dirt cheap. if it's token-based, it wouldn't make financial sense.
I need the full context window to get the work done though.
Next time it does that expensive scan, run order it to keep or update an index on the codebase. It really helps prevent these expansive scans if you have additional markdown files for LLM navigation.
That's what the person you replied to is saying. You don't need this model.
Fable and sub agents are two different things. There are many situations for which Fable is great, but Fable doesn't have to run in a sub agent. You can use it for your main agent and that works fine.
Or are you saying my sub agents burned so many tokens because they were all using Fable, whereas my main agent could do the same job with a lesser model?
I think the commenter (who is not me) is saying to use Fable as the main agent but then use lesser models for your subagents so that you get the advantages of Fable to plan but then the subagents don’t cost as much, and may be more focused because they lack Fable’s thinking modes.
I’ve heard the proper pattern is to have Fable write a software design doc and then tell Opus to follow that doc strictly in implementation and testing.
I hear that too, but I'm much more ad-hoc about what model I use for what. Opus can be good at planning too, and Fable is remarkably good at figuring out obscure complexities in the code.
How do I get it to spend fable tokens on “curiosity” then switch to cheaper models? Preferably based on its own judgment of what model is truly needed.
Using VS code if it matters.
Just ask it to. If you want, you can also give it pointers to how to read .claude jsonl/metadata so it keeps track of usage and self-adjusts. It's not perfect, but it's pretty dang good if you just say 'This project is allotted X% of my 5h limit'.
I never thought of that. Thanks!
But how is that better than a single agent searching those "angles" sequentially?
Unless they are orthogonal they most likely require similar context anyway so multiple sub agent is just wasteful.
If the assumption is that they can be searched in parallel and it takes the same amount of tokens as doing it sequentially. What you would gain is a speed up.
I vaguely understand you argument with the context, however is that not solved by sum agents handing their results in to the planner (or a third agent) to run on them again? I'd assume that's what is happening anyway. Let me know if that's wrong
doesnt intelligence involve knowing where to start and what to read and not just throwing everything in the bag.
im on local only AI and subagents are only valuable when they avoid polluting the context with extraneous file reads and parallel exploration when fixes are linear.
as OP is on about, subagents burn tokens because they arnt a deterministic intelligent gatherer but like pooluring water into a maze hoping the exit will illuminate.
Sub agents each have to read part of your code base again to get enough context for the task. And if they take too long, your orchestrator's context is no longer in cache so you pay full price for that again once the subagents finish
If you do it sequentially you only read those files approximately once, and everything hits the same prefix cache
Yes but one of the key things about subagents is they keep all of their tool calls and exploration out of the parent context.
If you plan on continuing on in the parent, and aren't going to necessarily be touching the systems the other agents are exploring, it can be worth it.
It's useful in certain situations where the parent context may need the "10,000 foot" view of something without going back in there. But subsystem-specific AGENTS.md/CLAUDE.md files are still superior and accomplish the same thing. The problem with those is they can become stale.
It seems like there could be a useful strategy of writing a plan with a main agent, and then instead of spawning subagents to implement, fork the main context to write each part. Then use one last fork to verify the work. That way you keep reusing the same context without polluting your main context for when you are ready to continue.
I've started doing this by hand in OpenCode and it works pretty well. But there's no UI support for maintaining a tree of related session forks so there's a little bit of manual fussing involved with session naming to keep organized. I also like to end a session with an "AI-friendly terse but detailed summary" (or some equivalent prompt) that I can then dump out to a Markdown file and then the mainline session can still get info back from the branch session. I don't know how much of this is automatable with OpenCode plugins, or in another hackable harness like Pi.
They are just making the point that it makes sense that subagents would use more tokens because they have none of the parent's context.
Right, so it’s a trade off between contexts. There are two reasons to use subagents, parallelism and tailoring of context. For the second, there is the “personality” of the subagents as well as how much context is injected from the main agent. Ignoring the personality, you ideally want the injected context to be small and focused on a single task so the subagent doesn’t get distracted. You want the main agent to be orchestrating all the subagents, but not reading all the same files they are reading, otherwise you’ll be paying for the same tokens in multiple contexts. IMO, this is where prompt engineering comes in, to be able to guide the main agent as to where subagents are desired and where not.
That is true of Anthropic's implementation but not inherent in sub-agents in general.
For the sake of accuracy Claude Code has a "fork" feature which does what you'd expect and based on traces I skimmed this morning it seems under some circumstances you get fork behavior for subagents as well.
True. For Claude Code, I disabled explore subagents globally by adding this to ~/.claude/settings.json:
Should be
"Agent(Explore)"
Is Explore the only thing subagents are ever used for?
Every subagent send the same ~30k system prompts. If you are using fable/opus, that's easily 30% of a 5-hour window for 7 subagent, before doing any work
The shared prompts are all cached so it's a cache read which is like 10x cheaper than a regular prefill
I recently did a few tests. And always the same prompt has been cached properly.
Cache is usually not shared between agents - they can have different base prompts, tools, and be an entirely different model.
Compared to the primary agent, maybe. But it's highly unlikely that all the agents have different tools and system prompts than each other, and those account for the bulk of the context per the post.
Depends on if they are launched serially or in parallel then.
The system prompt and available tools would likely only change for different agent types. So how they're launched probably doesn't matter.
I say this as I don't actually know how Clade Code does this, since it's not open source, but I fail to see why two agents doing the same thing launched at different times would have different tools and system prompts.
If it's always the same prompt, can't they have it pre-cached globally for all?
I'm pretty sure the system instructions are a function of your environment and not the same universally. That said, there should be a finite number of branches so still cacheable.
System specific stuff is probably quite limited, it can be a short dynamic segment at the end of the system prompt, perhaps.
The system behaviour is totally up to anthropic's discretion. Its current behaviour is verifiable. In claude code, spawn a subagent with
1. Agent("Test")
2. look at your token usage
3. Repeat a few times
I didn't check again as I type this message but am somewhat sure subagent doesn't cache system prompt as of maybe last week
[flagged]
[flagged]
Same for me. I never use them. I use Fable on highest effort to plan things and then record the plan in tickets. I use Kata, which is CLI and agent oriented, but I suppose Jira or other systems would work too. I tell it to put enough context in each ticket to on-board a fresh coding agent to implement it. Then I just do /goal, telling to to run `kata ready` to get new tickets to work and continue until they're all closed according to acceptance criteria or until they're blocked on actions from me. I need to play around with getting it to switch to smaller models (or spawning 1 subagent) to do ticket implementation and then auto compact after each. Either way, it results in really easy workflows and uses very few tokens compared to the built in subagent flows that doing this completely avoids.
Very interesting approach. Thanks for sharing.
Subagents with a fat tailed latency distribution completely masks the trough filling that puts the most downwards pressure on per-token COGS.
This is why the subscription plans are forced through the harness (the "OpenClaw Wars"): it creates a false equivalence in the minds of many customers between API tokens (latency sensitive, easy to measure) and Claude Code tokens (remnant backfill to stay to the right of the roofline, marginal cost often zero).
Selling sausage as sirloin is a great business if people go for it. And there's nothing inherently wrong with spot pricing, as long as you're honest about it...
I had learnt that trick, so now I explicitly disallow Fable subagents.
Yesterday, I wanted to review a complex piece after a large refactoring, and requested a review plan beforehand. The first step was 8 agents + one more to verify the findings (all Fable). Looks good, approved.
The verification step turned into an attempt to throw a party with 41 Fable verifiers.
It will find a way.
Don't do that; limit concurrency
"LGTM"
That'll be $50 — please.
Agreed. The issue is that when working 1:1 you get a feel for how many tokens are being burned but the subagent spawn could be 3 or in one cases it spawned 171 to verify something. The latter was unexpected and burned through my token budget.
And in my experience the sub agent performance is usually worse than just a single agent.
I find it useful for code reviews (spawn a subagent with minimal/no context to review X commit). Of course, this is more or less a shortcut that could be done with a seperate agent. Another use is multiple reviews at once if tokens are not an issue, with seperate "personas" or focuses. As far as implementation goes I have not seen any major usecase.
Yeah, my personal workflow has different reviewers for codebase(patterns, code cleanliness, etc), frontend, security, product fit, etc. So they spawn as separate subagents. Both so that they stay limited to their role, and so they don't have preconceived notions about the implementations. It's a bit heavy-handed but works for me.
Spawning a bunch of agents seems to happen randomly. I almost never want this.
In my CLAUDE.md I put:
> CRITICAL: Do NOT spawn sub-agents for any reason. Perform all work in the main session. If a task is too large, ask me to break it down manually.
> This is a big task, and can easily get too large. However, sub-agents make the situation worse, and eat through our token budget way too fast. Do not use them.
> Take on manageable tasks. Don't try to do everything at once. When you start on a big task, break it down into smaller tasks, and make sure you finish each task before starting on the next one.
Or actually Claude put it there for me. Maybe it's a bit much, but it seems to work.
If there's some "find the file" task, using full context for that isn't ideal.
I think there's some setting to restrict the number of them, or maybe turn them off. Doesn't happen for me ~ever and it's not my $$ (work) so I haven't really looked at it much.
Such is the nature of tool use
> What really burns tokens is sub agents. I once gave Claude Code a pretty big task, and it immediately launched 7 sub agents which burned through my budget before even one of them was finished. Tried again 5 hours later: same result.
Probably because the general purpose subagents inherit the parent model.
I tell Claude explicitly to use Explore subagents, which use Haiku only, now.
They changed it with the release on July 1 Explore now inherits the model, it isn't always haiku.
https://code.claude.com/docs/en/changelog#2-1-198
> The built-in Explore agent now inherits the main session’s model (capped at opus) instead of running on haiku
Urgh, thanks for the heads up. I guess I need to be explicit about my choice of model now.
Yeah, I was surprised. I had Claude make a skill to extract the explore agent from Claude Code, but set the model back to haiku. Here it is if it's helpful:
https://gist.github.com/joshcartme/dd71df7b4c51c356760b28d7f...
> Probably because the general purpose subagents inherit the parent model
only if you don't specify which model should be used
[flagged]
for subagents to be cheap/effective, you have to specify the size of those subagents; i.e. right now by default 5.6-sol spawns many 5.6-sol subagents. 5.4-mini as subagent saves me tons of tokens. 5.6-sol audits the work before accepting it, so there's not really a quality issue.
It’s funny too because I’ll ask fairly simple things and it’s fine, similarly simple question might spin up a bunch of sub agents and I don’t know why….
I feel like maybe it could have asked for clarification or something rather than go and try to calculate all the digits of pi all of a sudden.
Probably both. The default subagent orchestration is designed for infinite pockets.
Maybe when they realize there is need to change this they come up with a more configurable interface for us mere mortals who can't afford to gamble their house on a pay as you go subscription.
They did recently change it so the default explorer agent inherits the session agent (capped at Opus). Before Explore was always haiku. I had Claude write a skill that extracts the built in Explorer agent skill, and then writes an identical Explore agent that uses Haiku
There is a negative incentive to fix problems that result in customers picking a more expensive plan to work around it. There are probably several engineers who have ideas about fixing this and they get apathy from many people and obstruction from a few, and sometimes active hostility by a manager somewhere in the chain.
The best you can do in such an environment is seek to introduce new features at the top tier, and then pull old features down the stack as the cost of those features has been amortized out, or to hurt your competitors by raising the ladder.
I’ve had similar experiences. I now have an explicit line in AGENTS.md to not use subagents unless explicitly requested. It also helps that for the tasks that are big enough to benefit from subagents are also the ones with high chances of going off-rails and/or a poor review phase. I’d rather do the orchestrator role and that way I can split up the review phase in a much more manageable chunk.
Yesterday I gave Claude Fable a difficult task. It then proceeded to spawn 415 agents. It got it done, but damn was it expensive.
They optimized it to burn more token in the recent months I feel. I made a small ~100 line change to a codebase by hand and threw claude at it to review. It spawned several sub-agents and burnt a ton of tokens. I guess the word 'review' now triggers some sort of in-built skill or something. It's absurd how rapid enshittification is taking over.
Indeed it feels like I do the same work, ask the same questions, get the same result.
But somehow the cost has doubled in the last few months.
This is why I happily use Codex.
I run it basically 24/7 on a ~500k line repo, and only rarely run out of quota before the end of the week.
My experience with Claude Code was very good until about 2.5 months ago, and then it suddenly turned unbelievably terrible for me.
I have not and will hopefully never look back.
I still have PTSD from how ungodly terrible it was that last week of using it.
> I still have PTSD from how ungodly terrible it was
Please, for the sake of everyone suffering from actual PTSD: Don't. It's hard enough already for victims to communicate what difficulties they are facing without people watering down terminology like that.
Please don't act as the hyperbole police. People exaggerate all the time (I'm starving, etc). It's normal, and you are being a jerk to call them out.
I am asking them to reconsider and reflect on what that kind of language use does. You're the one reading it as "calling them out".
How else are we supposed to learn from each other, voice our opinions, point out our mistakes to each other? For me, this is communication. And currently 8 upvotes seem to agree with me and my request. Feel free to ignore it, or consider it, for your own use of language. But, sorry, to me, you're the one acting like a jerk and trying to "police", not me.
Have you ever said "I'm starving"? Do you think that undermines the experience of people actually starving in war or famine?
PTSD sucks on its own. Trying to blame other people for its symptoms is a deflection. It would be like someone with ADHD blaming his inability to concentrate on everyone else in the room making noise or moving around. People with OCD tried this, acting like it's a Monopoly property that you have to pay rent for if you land on it / say its name.
If anything, it's a net positive people are talking about mental health and recognize different ailments such as OCD, PTSD, and others.
I agree with what you say? I think you're reading something into my writing that I didn't mean to imply. A headache is not a migraine is not cluster headache is not multiple sclerosis. We have vocabulary so we can express ourselves in an attempt to communicate. I am inviting people to reconsider their use of language when it comes to severe challenges such as PTSD. We are here to exchange opinions after all. If you find that offensive, so be it. You decide for yourself how you express your disapproval; two opinions can stand next to each other without introducing "blame" or "deflection".
I agree with you that it's a low effort, repetitive drone when people say "that cable job irritates my OCD" or "I still have PTSD from dealing with that annoying person". I don't like hearing it because it makes me think that person is a parrot that repeats common phrases instead of developing their own voice. So at least we agree partially.
I disagree that it's insensitive to those or have the illness. If it hurts you when people use PTSD as a literary special effect, I invite you to articulate why exactly it makes you feel (however it makes you feel - I don't want to put words in your mouth). I get the sense that some people take offense the same way a religious fundamentalist doesn't want to hear their Prophet or Messiah used disrespectfully. The source of the anger is built on faith and dogma.
They have Coder PTSD or CPTSD.... Is that a better acronym???
Sorry just teasing.
Can you be more specific about what “unbelievably terrible” means?
This has been my experience as well. Something happened 2-3 months ago with Claude Code. It got slower, starting spinning and getting stuck more and more. I gave codex another shot out of my Claude frustrations, and have never looked back again.
Just tried Claude Code yesterday, and nope, it's the same old bad.
For a while everyone was saying sub agents is how you save tokens, use lower quality models with limited context to do simple parts of the job after a smart planning agent has put it all in place. Is that no longer true or is this just the result of sub agent being used at the wrong time?
No, you can definitely configure low cost search and apply subagents. CC and Codex do not. Not sure if this is to improve the reliability of their subagents, or just a play to increase user consumption.
Well, Anthropic and OpenAI make money selling tokens, so…
I like to use subagents a lot, but I find them to be most useful when explicitly specified. E.g. "assign these tasks to 2 Sonnet, 2 Opus and 1 Fable subagent". Helps keep allocation consumption under control.
Not only a Claude Code issue. Started using OMP with GPT 5.6, and gave up, it loves to use subagents, and it's basically unusable subagents with GPT 5.6 Sol there with Plus limits.
--disallowedTools Task
Did it deploy five AWS m8g.12xlarge instances?
i have globally disabled subagents for claude. otherwise one prompt ended my Pro account
lol I asked fable to help me estimate my TAM and it launched 102 agents and blew my $120 quota in 6 minutes. I do realize I can limit the agent count , hah
It's in the best interest for AI companies to gobble up tokens. I feel like every new release - Fable, etc - is just a way to extract more tokens/money.
(If) something like the current LLM/agent paradigm remains in a few years, and companies settle down into their respective niches, I imagine more user-friendly tools will be built, with more control over subagent spawning, context, caching, etc.
What's happening this year, with secrecy and all, is saddening, but expected.
Of course it is. How could it be anything different? Clearly, that’s how these companies make money.
it's a very handwavey way to "explain" anything. Yes, they make money. But they have competition. And if someone runs out of tokens and switches to deepseek or just goes for a friggin hike in the woods, that does not benefit them. If they get a public image of a ripoff that burns all shit on trivial tasks, that does not do them good either. So there is a limit to this "companies make money" thing.
Sure, fair enough. Clearly, if they increase costs by too much, people will go to their competitors, but those competitors also make money selling tokens, so the whole industry is incentivized to inflate token consumption up to the point of driving people to the competition. And nobody is incentivized to reduce token count.
In fact, the one model with great price/performance is Deepseek v4 Flash and I suspect that they are subsidizing it deeply to get access to everyone’s prompts for training. We may find that they raise prices on the next version (v5) after they’ve mined the user data.
Any AI service that people (and to some extent companies) can afford to pay for today is being heavily subsidized. Will that last forever? I really don't know how those economics work, but I know that bubbles do burst having lived through the dot com burst in 2000. And I know this current one is going to hurt if/when it bursts.
On the issue of the bubble, I’m right there with you, 100%. I’m not sure that “subsidized” is the right word for Anthropic’s or OpenAI’s pricing, though. I’d say it’s forward-priced. Supposedly, they have claimed that inference by itself is profitable; it’s the ongoing training that is not. I don’t know what nuances apply to that, however.
Subagents are quite inefficient and the lossy context transfer between them does lead to more cost and more waiting. However I have found it to produce more reliable output, whether that is worth it for a given task has been a consideration.
My opinion is that claude code uses more tokens simply because Anthropic makes more money that way and forces people into their subscriptions. This is supported by the fact that they won't let you use your sub on a different coding agent. I use pi btw.
Once I realized that Anthropic is a token merchant, I start to understand Anthropic’s decision more. They are always finding reasons for you to use more tokens through them unless the users revolt or demand some guardrails.
I've done a couple side by sides on web chat with the same prompt on Opus 4.6, 4.7, and 4.8 and the output gets longer/more verbose on version increment. The enerr variants are definitely much wordier.
On the other hand, the newer variants also tend to benchmark higher so it's not quite a clean argument of "hey the new version eats more tokens"
I think both things can be true: new models benchmark higher and eat more tokens.
From my experience new models are slower and use more tokens even on questions which gpt 4 answered correctly. It is mostly because newer models tend to be more verbose (even with prompt requesting short answers).
Unless somebody improved on the underlying transformer architecture... Surely AI is smart enough to do it by now
I've done a couple side by sides on web chat with the same prompt on local 4b, 14b, 32b open models and the output gets longer/more verbose on version increment.
Its rather frustrating, slower tokens and more tokens.
The Agents are more like Double Agents. Purporting to work for you, but with the primary goal of siphoning your wallet to its handler.
I bailed on Anthropic the moment they started blocking alternative harnesses like pi on their subscription plans.
If I were anthropic I’d force that too. They offer the harness and if they control the entire pipeline then they can optimize the entire experience. It doesn’t have to be nefarious.
This is kind of a strange comment as it implies a false dichotomy.
Its not 'nefarious' in that its in their best business interests.
But it'd be difficult to take anyone serious who thinks Anthropic's motivation was to improve the UX, and the other effect were by accident. At the time they specifically started blocking based on openclaw prompt text. Its a walled-garden tactic.
A walled garden is nefarious to people who do not want to be inside one.
It's like Microsoft banning Vim users that use Azure
They didn't ban people from using Claude, though. They banned them from their flat-fee subscription and required that you pay per token.
It's still questionable but I don't think it's in the same ballpark as what you describe.
I don't think it's in the same ballpark at all. I checked the `/usage` in my session which uses a Max x5 plan. One day I had used $400 of tokens and 20% of my Fable allocation. Anthropic is effectively giving us more tokens per $ on the monthly plans but it comes at the cost of Anthropic being the prompt-writers and managers of the agents pretty much entirely. I don't think this is a bad deal.
Whether or not it's a bad deal depends on what you're comparing it to. Compared to API pricing, of course a subscription through CC is a good deal. But when OAI offers their super-subsidized plan and allows you to use your own harness which is 75% more token-efficient, then the CC deal starts looking like a bad one in comparison.
It’s really not. Vim isn’t instrumental to Azure usage.
CC isnt instrumental to use Anthropic LLMs. Yet here we are.
> if they control the entire pipeline then they can optimize the entire experience
So what? When you care about optimising the entire experience, you offer sane defaults.
When you prevent people from changing the defaults, it's about control, not experience.
> if they control the entire pipeline then they can optimize the entire experience
The only issue is that Anthropic optimizes the entire experience for their bottom line. User experience and price only suffer becaue of that.
Sounds like they're modeling their PR on the classic Apple playbook: "choice is bad, and you should appreciate the constraints we've generously imposed"
> It doesn’t have to be nefarious.
The nefarious part is because it's non optional. They could give you an option and compete by being better, instead you're given the finger as the option is taken from you. Competition is hard and banning people to create more FUD serves business need better.
You've obviously been gaslit so badly you're desperate to find a way to defend a shitty move and pretend it's the only way to increase usability. But you don't have to deny really! You're allowed to admit control is easier for a company than competition, and that they didn't have to, but did because it increases their control of the ecosystem.
If you want to defend someone, good? But at least save it for someone who actually deserves it. They don't; and you insult you and your readers intelligence by trying.
But they gave us double the tokens! Then a limited time more usage! Then even more tokens "off peak" times! Then some new model released but apparently it inherently used 1.69x tokens! Then Fable is here but "it uses much more usage". But only until ~~the US banned it~~ ~~7th July~~ ~~19th July~~ who even knows.
At this point I think Dario is just in his wellness retreat adjusting a revenue/profit dial.
Ah, the ol' retail switcharoo.
Increase the price by 70% and then cut it by 50%, resulting in a 15% cut that sounds like a major deal.
now reealize that LLMs are trained to produce tokens and like the halting problem, cant be trained not to produce tokens and youll realiE the AI labs are the perfect essential capitalist and like cancer, will keep growing useless tokens until it kills its host.
no amount of alignment will stop aomeone drom just shutting up.
LLMs might be trained to produce tokens, but Anthropic don’t have to price by tokens. If an organization is a ‘non-profit’ and they decided to design their pricing to be tokens-based, I get it. If a for-profit design their pricing to be tokens-based, I don’t know where are they drawing the line between profit vs benefit. That doubts makes it hard for me to be a customer. Disclaimer, I still use Claude…
tokens definitely measure compute.
You can ask it to verbatim produce training data and that takes very little compute for a lot of output tokens
i dont think you understand how these models operate.
You can burn kilowatts generating 10 tokens, and conversely produce millions of tokens burning very few watts. You're comment is horse shit
Serious Willy Wonka energy?
Seems unlikely they'd be this dumb. The way to get us to use more tokens is to make those tokens more useful, not less. Anthropic is full of people (including higher-ups) who know this.
But it is much much simpler to make it consume more tokens.
It’s like that saying “What Andy giveth, Bill taketh away”, but in this case it is one company.
There is definitely a conflict of interest.
It's the same conflict of interest quite literally any business has. What stops any business from over-charging? Competition.
> What stops any business from over-charging? Competition.
I fully agree.
> It's the same conflict of interest quite literally any business has.
I know that you know what I meant ;) In the long term it is just as you say - overcharging (eventually corrected by competition forces), but in the short term it can be additional revenue, blamed on a bug, but making some manager look good.
I thought I read somewhere that according to filings for going public, subscription revenue is tiny… like 5%.
Edit: consumer Claude subs are the 5%. I’d bet most all of CC subs lump in under enterprise.
So the incentive to have Claude Code use more tokens should be even stronger then as AI & Enterprise are using consumption based pricing.
The vast majority of my company's enterprise plan use is through Claude Code even though we have access to the API and could be using OpenCode instead.
I don't fully agree with the premise that they intentionally increase system prompts, but the enterprise plan usage is going to make that a huge income for Anthropic.
The fact that individuals are more likely to use the alternatives than businesses is telling.
Anthropic is fine, as long as someone else (a clueless employer drinking Dario Koolaid) is paying for it. But the moment you have to pay for it, people just bail and go for DeepSeek, Kimmi, OpenRouter, OpenCode Go and other alternatives that give more bang for the buck than Anthropic.
Yep. That's my case.
I now have unlimited Anthropic and OpenAI plans at work because CEOs bought the hype.
But for personal projects $10/mo OpenCode Go serves me with DeepSeek V4 Flash, MiMo 2.5 and GLM 5.2.
You're making the opposite argument. Anthropic is incentivized to use less tokens in Claude Code because people are paying a fixed monthly fee for subscriptions.
Nope, that’s not true, because they want you to pay for the higher subscription bracket.
Can confirm — they got me paying $100/mo this way.
Also I think it’s well known that OpenAI is the much less expensive option (in tokens and $$). For the same $20 you get a lot more mileage.
Curious if folks have strong opinions about the overall UX of OpenCode vs CC…
For me as well, at least this month to use more of Fable. We'll see if they extend Fable access because of people like me.
That strategy only makes sense if there's an abundance of tokens, but that's not the case. AI companies are spending a ton of resources on improving token efficiency because they are all severely GPU constrained. Anthropic instead nudges you to move to a higher tier by setting rate limits.
Also not true, they want you to pay for a higher subscription bracket and then use only marginally more than you would have, which I think they’re doing quite effectively for most people based on my interactions.
If they wanted to play games with sub tiers they would just change the rate limits rather than wasting inference.
Flip side is customer psychology. Choosing a more expensive tier leaves better emotion.
Also i doubt there was jira ticket with “make llm more verbose”, rather ticket with “bug makes llms too verbose” gets prioritised taking revenue impact into account.
Higher subscription brackets are likely worse for them. I recall seeing someone calculate that a fully maxed out highest subscription bracket is something like $15K in tokens?
And people paying $100 or $200 are much more likely to max it out for purely psychological reasons - it crosses that threshold where I want to see my money's worth in full. Whereas people on $20 subs are more likely to be there just to get access to better models and features, and are not necessarily even doing any substantial work.
It's always more complicated than that because the prestige and Early Adopter users are what drag other people to also be customers to avoid FOMO.
Your gym members who got a subscription aspirationally and don't show up are absolutely subsidizing the power lifter who is introducing wear on (tens of?) thousands of dollars of equipment three times a week, but if the regulars weren't there you wouldn't have sold those subscriptions at all. Without a poster child there's no poster.
Well since what you get for your subscription is unknown it would be trivial to get that result without burning tokens.
Especially since compute is such a scarce resource.
Generally, companies with >150 people can’t use subs. So yeah, it’s mostly a funnel for devs/small companies to eventually vet for the product and convince their enterprise to use it as well.
They could just use less tokens and finish your quota sooner. So even tho I think are a bad company, I can’t say they do this for the reason you said.
Enterprise users are not paying a fixed fee, though
Yeah, I strongly recommend against Claude Enterprise, it is ridiculously expensive and hard to control costs.
Not really. The incentive is to make you hooked on the process, so you bring the same process to the workplace, and start paying corporate prices, not individual subscription prices. For that to work Claude Code, prompt, and the rest of the mechanics has to be more or less uniform.
> I use pi btw.
When using Pi, one way to significantly reduce input tokens it yields is to ignore common bookkeeping "dot directories", such as `.git`. How to do so can be found with the following interactive Pi prompt:
Other local assets to consider ignoring are `.pi`, `.agents`, `*.md`, and language specific output directories such as `__pycache__`, `bin`, `obj`, `target`, etc.> I use pi btw
Not sure if intentionally meant as a reference, but it gives "I use Arch btw" vibes.
Pi is one of the ways out of this problem (OpenCode another) so I took it as an intentional reference as it is highly relevant. I also use Pi as my daily driver and I think it's a wise choice to figure out how to decouple yourself from lab-specific harnesses that you have little control or observability over.
[dead]
the amount of system prompt wastage going on in orgs is insane. we identified 400k in annual burn for zero value in just one section of our large company.
and the interesting thing about system prompt wastage is its a cost that scales non linearly with subagent use.
The non-linearity is interesting. Is the default behavior for subagents in CC/OpenCode loading the same full system prompt (or AGENTS.md)?
I'm sorry, what! 400k...?
> My opinion is that claude code uses more tokens simply because Anthropic makes more money that way and forces people into their subscriptions.
OTOH, this makes typical subscriptions usages consume more tokens, which are included in their flat fee.
This sounds more like incompetence than malice.
It would be true if there was a unified "Anthropic" entity making every decision from pure rationality. Instead, more tokens increase Claude Code team's metrics of token usage, which most likely has a KPI around token usage and adoption.
To remind Goodhart's law: "When a measure becomes a target, it ceases to be a good measure".
..also to parent's point, yes the upsell is only appealing once user run's out of tokens.
Anecdotally at least Claude code uses less api money for me than other harnesses. I think people might be missing some caching discount?
> This is supported by the fact that they won't let you use your sub on a different coding agent
I mean, that's a very weak argument? Isn't a much more plausible explanation that with your tooling you'll have more of a lock-in than with just your model?
Neither is mutually exclusive.
They get lock-in, and through that lock-in are more effectively able to inflate token usage.
This isn’t limited to large system prompts. Coding-agent harnesses are also becoming more aggressive about using tools, even for trivial requests. In our tests, prompts such as “Hey” or “commit” sometimes triggered 30+ tool calls:
https://quesma.com/blog/the-true-cost-of-saying-hi-to-an-ai-...
Tokenflation seems very real: the number of tokens consumed by simple tasks keeps increasing.
I often find myself annoyed when Opus fixes a typo in a comment and decides to run tests, lints and whenever else it can find to run. Often it will start by stashing current changes just to preemptively check if all tests were passing before. And I can blame myself a bit because my rules do say: verify all changes with tests. But as there is that I in AI that is hyped which you’d think means it knows not to put tomatoes into fruit salad …
> [..] my rules do say: verify all changes with tests
I am a bit surprised that you're disappointed that it does exactly what you told it to - people usually have the opposite complaint.
If you're using it interactively and watching what it changes, I'd trigger the tests when you think it's needed. And if you want to go more hands-off, why not add try to encode the same nuance you'd use into the rule?
Rather than bake that into the prompt - wouldn’t it be better to just set up a pre commit hook that runs tests and linting?
Maybe, depends on their workflow. In my human workflow, I tend to use commits as checkpoints and then squash before pushing. I'd usually only run time-consuming tests before squash+push.
But yes, anything you want to ensure really needs to be a hook.
edit: realizing with "precommit" you probably meant a git hook not one in their harness. I'd have written the same response more or less though. :)
Oh yes - definitely the git kind of hook. Also, I always forget that there’s a pre-push hook as well. So you don’t need to do things every commit.
But then you could just be storing up a lot of problems…
Indeed. That's why I think it depends on the individual's workflow where it should live.
Following rules like "verify all changes with tests" down to a tee is usually a desirable trait in LLMs. Personally I'd leave that behavior there (just like with humans for some tasks like aviation you have them go through checklists even if some stuff you can infer is not needed). But otherwise just make it "always run tests unless you're absolutely sure they can be skipped".
Add "... unless the changes are trivial, docs-only, or typo fixes" to the "always verify with tests" instruction and see how that does
That's one of the reasons I started https://beolis.com. Now I have a workflow that says to do things in a TDD and run only new tests and related tests but NEVER run the full suite and then when it finishes the workflow runs all the tests -- if it works, great, if it doesn't then the workflow continues, feeds the output to a cheaper LLM to summary the errors and then get another run to actually fix it, based on the failures and the context based on what should be implemented.
-- note: I've been full time in Beolis for some months already, feedback welcome ;)
> prompts such as “Hey” or “commit” sometimes triggered 30+ tool calls
I read that this is because it wastes time looking through past conversations and other context to figure one what you might want it to do - a less ambiguous prompt would be better.
Why are you asking the LLM to commit? Can’t you do that yourself?
Why are you asking the LLM to code? Can’t you do that by yourself?
UPDATE:
After reading PUSH_AX's valid comment: ``` This is like saying contractor (A) asked for $33,000 to undertake the work and contractor (B) asked for $7,000 Are we measuring and caring about the right thing? ``` We will update the post to include:
1) A more in-depth task. 2) Qualitative results comparison. 3) As soon as possible, a reproduction of the inputs and outputs.
This isn't accurate since the main reason I'm using Claude Code instead of these other interesting sounding harnesses is the subscription service with highly discounted token usage. With OpenCode, you're paying the full price.
Therefore, you should include the actual costs associated with the task in API token usage or subscription level. Is there a reasonable way to do apples to apples cost comparison?
We are using Claude Max with OpenCode. See the post for details.
If cost were the only factor of course you'd use B, but presumably you also care about quality quite a bit.
Thanks, I'm looking forward to this!
I wonder if a lot of the 33k is context, like from recent conversations.
We've updated the post now!
Thank you!
And pi agent is even less.
The entire agent system prompt can be seen here:
https://github.com/earendil-works/pi/blob/main/packages%2Fco...
Maybe related to this minimalism, Pi doesn't come with most of the tools an LLM needs to function efficiently or effectively. I get that a blank slate is the paradigm, and you can add whatever you want, but it's too blank IMO.
I have a functional Pi config, mostly self-made (it has everything I want, incl. subagents, web search, a /btw command, and other misc. addons), and my system prompt is ~3k.
Would you mind sharing?
Exactly. It’s minimal to start but the sky is the limit at the other end. I love the plugability of Pi.
what tools are you talking about? Pi has ALL tools the LLM needs to function efficiently and effectively for coding tasks. It can read,write,edit files and can use any bash tool to search files, execute tests and so on.
Every time I read this comments I have the feeling you are talking about mcp or sub agents, otherwise this makes no sense at all.
That will increase the amount of initial tokens used, because the tools have to be described somewhere. Maybe not as much as Claude Code, but it could get more if you just randomly keep adding tools.
Oh-my-pi has more tools than claude and opencode, and uses them much more efficiently. my favorites are /collab and the gortex mcp
I tried using omp, and really like the interface, but I found it used tokens much much quicker than the Claude cli. Some simple tasks would use all the session tokens in less than an hour, as where I could get easily get 3-4 hours with Claude. Both set to use opus 4.8 auto effort. I tried tweaking the models for agents down to haiku and sonnet in omp, but didn't notice any real difference in the speed tokens were being used.
if the website is any indication, omp is just vibe coded AI slop as well. pi is awesome but it looks like omp is a step down including unnecessary stuff no one needs. it's kinda the opposite of pi
I've heard amazing things about pi and it's effectivenes but when I tried installing it I quickly found out it doesn't respect XDG_BASE_DIRECTORY at all, you need to set some environment variables and the author rejected both a proposal as "going full gpt", seemingly not even knowing about XDG_BASE_DIRECTORY, and even rejected a PR.
I've heard really good things but that being my first experience with pi didn't fill me with confidence about it's code quality either.
For now I stay with OpenCode I think - I was using zed editor and agent for the longest time anyway and think I will go back to that. CLI tools for me seem a bit too disconnected from the code.
Interesting! I wouldn't know, because I'm on mac. I love that it's very basic and I can basically do plug and play with the stuff I need. The load, complexity and system prompt overhang is quite minimal. But in the end, opencode is also fine I guess. I don't need to fiddle around with it too much and just need it to work. So if you're happy, why switch ;) I actually use zed as an editor w/o agents lol
It's easy to add using plugins.
What do you miss? I ask because I do some heavy work with pi + GLM 5.2 (using opencode Go subscription) and my workflow is plan -> implement.
> It's easy to add using plugins.
Sure, but you have to add almost everything, no? It deliberately only comes with read, write, edit, and bash. My point wasn't that you can't add stuff, but that I'd just rather use an harness that's a bit more full featured from the start.
(Pi is a bit like old 3D printing where fettling the printer to work is a central part of the hobby. I'd rather just buy a Prusa.)
I'd like to understand what features you're referring to that are missing from base-install Pi CLI.
The main ones missed immediately were web access/search. Then the to-do list features (it was a nice surprise to try OpenCode and see this working immediately.). There were a couple of other niggles but it was a few months ago. Also, this may not be common, but it seemed to struggle to edit effectively (driven by Qwen 3.6 35b/27b) and often rewrote whole files instead.
Gotcha, I feel like model or provider-specific installs would be a nice QoL improvement in that case. Presumably, part of this issue (beyond the ethos of minimalism) is the aim of shipping shipping an agnostic toolset. For myself, im openai-first, and of course that pushes me to favor their hosted tools (in this case web search), and their native CPT/RL'd stuff (Ie apply-patch).
Though, imo, the fact that pi maintains its "we only include the bare minimum!" statement is part of the draw for me. Especially considering that im in an enterprise env; being able to internally share custom implementations of out-of-the-box Codex/CC stuff is really nice.
I do wonder how they'd go about shipping a default web search tool. Big problem there is the lethal trifecta. Shipping something that arbitrarily allows untrusted content to be retrieved non-deterministically I'm sure is a long conversation on Pi's end. Pushing it off to the user to decide is easy.
I know it's against the ethos of Pi, but I think a lot of people would consider a handful of things like a memory system, web search, subagents, and looping to be a basic/base thing they would add to every agent harness.
I use Pi and love it, but the base install is extremely minimal (a handful of tools, no subagents, etc.). That’s on purpose. Sure, you can add more with packages, but that’s not the base install.
You know, seeing a total of 3 replies on this topic, I think the lesson here is to, from now on, with comments like this, to clearly state
"Pi has *ALL* the tools, can you name one it does not".
It starts lean, yeah. Did you know you can prompt it to add the capability you feel you’re missing?
[dead]
If you really want a minimal agent that you heavily customize, just skip pi (130+ transitive dependencies on the "minimal" pi-coder package) and write your own. You learn a bunch, and it's not hard. You can even ask another LLM to help you get started.
I wrote my own harness in Emacs and it’s completely ridiculous how well it works. Auto-compact is the only missing feature on my list. Claude‘s approach, if I understand it correctly, invalidates a lot of cached context, and I‘m thinking about a more cache-friendly strategy.
Claude is very cache friendly, however there have been some inconsistencies with non anthropic endpoints that led to cache breakages
Exactly! I just vibe coded (with GPT Sol and Claude whatever-number) my own agent, it's trivial to add now any feature I want - simply ask more powerful model to do it for you. I am happy with end result, however it looks indeed these tools are trained to increase token count - they do quite stupid token-spending steps while making code, but the code itself is also a bit weird - it's like they intentionally do code which is hard to modify on your own without using exactly those authoring models. Interestingly, when I am using DeepSeek with OpenCode, I don't see that - it understands my intent well enough and overall code quality is not bad. I recently switched to local Gemma 4, and I often switch (in opencode) to just that less powerful model, because it understands my intent and has enough skills to provide good quality solution although it's rather for small size projects, and for not coding from scratch, but it's also free and private. It feels slower than any big cloud model, so my model switching is probably most quickest path to robust end result :)
That’s certainly doable, but then you need to create all the add-ons you would have added to Pi. IMO, Pi stands in that sweet spot between being very minimal while still offering a catalog of pluggable functionality that you can add to it. Sure, you could vibe code all those things for your custom agent as well, but why recreate what is essentially Pi all over again (the main loop with all the extension hooks, etc.)? Pi is the “standard” batteries-not-included agent, where you can add from a catalog of pre-defined batteries. I recommend starting with Pi and then using an LLM to code custom extensions where the catalog doesn’t have something you want.
Pi has way too many batteries included, including a bunch I don't want, and lacked the batteries I did want. Pi is a bit like the movie Idiocracy in that the idea is much better than the execution.
Incidentally, I also have zero supply chain attack surface as I have zero dependencies in my agent, just go stdlib. Pi, again, has 130+ transitive dependencies asking me to trust the security of my system to 150+ additional people I've never met in exchange for a bunch of bloat I do not want.
Agreed that the node cesspool is a risk. That said, what batteries does Pi have that you don’t want?
Bloated TUI library, unified multi-provider LLM layer, bloated RPC and SDK modes, the entire plugin framework, the list goes on and on.
For reference, pi-coding-agent, by itself (not including dependencies, tests, or pi-ai, pi-tui, pi-agent-core, etc), is ~41,653 SLOC taking up ~1658.9 KiB across 163 files.
My agent, excluding dependencies (all go stdlib) and tests, is 3 files, 946 SLOC, taking up 36.3 KiB, and includes a basic TUI and an XMPP transport channel (including TLS for XMPP), with dynamically configurable delivery to and receipt from either or both, including allowlists for XMPP message partners. It has tool calling, a permission model with whitelisting and interactive permission querying on a per-tool basis, full thinking support, including the ability to toggle hiding or showing it across either or both transports repeatedly throughout an individual session, the same tools as pi comes with out of the box, plus web search, and a tool to vet, build, and git commit golang projects all in one go, stopping if errors are observed. Configurable model and endpoint, too.
Incidentally, the open source xmpp server (prosody) and metasearch engine (SearXNG) are both self-hosted, too.
Sure, if your fundamental issue is “bloat,” you can always write a less general-purpose system that is smaller. No doubt about that.
I guess better phrasing would be auditability, ease of codebase comprehension, coverage of just the features I want. My agent isn't meant to be for X users across Y providers with Z extensible plugins, it's meant for exactly one user, with exactly one provider, and to minimize the amount of trust granted to third parties.
Got it. Yea, if those are your priorities then it certainly makes sense to write your own.
This is a truly underrated approach IMO
Any tips on how to get started?
I have written a few myself to get an understanding.
To learn yourself:
<$20 on a cloud AI api for a chunk of tokens and have the AI teach you. "help me write an AI Agent using (language) and walk me through the steps"
Realize that these agent are REPL/while loops that maintain a conversation state and then based upon the tagging syntax like <TOOL:bash:uptime>uptime for system run time</TOOL> and the agent extracts the tool and then does sub commands.
At a minimum, you need an inference endpoint: either cloud or local.
If going local, llama.cpp is going to be the more beginner friendly local inference engine that supports more processor types (AMD GPUs, Intel GPUs, CPUs, anything that supports Vulkan, not just Nvidia). LM Studio is a nice wrapper for this if you'd rather avoid cloning repo and compiling yourself, provided you don't mind closed source software; it's much less enshittified than Ollama.
If going local, you will also need model weights in the right format for your inference engine, and with a model that can fit on your hardware. This is going to be .GGUF files if you're using llama.cpp or a wrapper for it like LM Studio.
From there, pick a language, go look up the OpenAI /chat/completions API format (or Anthropic's "Responses" API format), create a DS or array or slice to store messages, and build a loop that accepts user input, formats it according to the API format, sends it to the inference server, retrieves and parses the response, adds the response to the DS/array/slice, and repeat.
There's a lot more beyond this - tool calling, other API formats (optionally), MCP servers, transport layers besides terminal stdin/stdout, permission models, starting with a system message, clearing your message stack correctly (hint: don't reset it mid tool-call), message compaction, web searching and page fetching, semantic search RAG over embeddings, memory layers - way too much to cover exhaustively in a single message.
Quick self-correction: "Responses" is a newer OpenAI API format, "Messages" is the Anthropic format.
I was here looking for this comment = )
Read through it an I'm curious whether setting the date and cmd on every system prompt call will cause the cache to invalidate.
I guess the cache would only be invalid if the day changed or the root directory, which would technically happen infrequently enough.
I get 95% or more cache hit rate with pi and DeepSeek or MiMo so it doesn't invalidate.
But I'll investigate how that works in a session. You got me curious.
Recently switched to Codex after 6m in Claude. Codex seems more open, it’s easier to follow what the model is doing and the approvals have a better UX. Overall, it just feels more transparent. Cost of switching was close to 0.
I don’t like that Claude became more opaque around February, including the system prompts. 33k feels way too much.
I use both now and agree they're basically interchangeable.
I appreciate that Codex is open source and OpenAI has explicitly said using the subscription with other agents is ok. OpenAI has been much more consumer-friendly recently.
And OpenAI didn't try to silently degrade performance of their top model if its (extremely sensitive) safety sensors went off ...
Anthropic is the silver lining keeping p(doom) below 1.0
I like Codex for allowing auto_review.policy (basically a prompt to the classifier on what to allow/disallow) to be configured rather than the opaque auto mode in Claude.
What settings have you tried since it "became more opaque"? They've got a lot more settings now.
They’ve been hiding their thinking tokens more and more, and lately also which tools are being executed when and in which way. It makes it more difficult to assess what it’s doing and jump in to steer it into a different direction in realtime.
CC went from sane defaults in late 2025 to feature scope creep early 2026. So more features might be good, but sounds like an ick for me. But I have zero prestige, I might switch back.
Are you familiar with "the tyranny of the marginal user"? They have to add more features right now.
6 minutes, 6 months, $6 million, 6 million tokens?
Worth noting/understanding, for those that are not familiar with it, that these 33k tokens are not a single 33k batch of tokens, they persist and increment in every request.. so if you have 10 requests that is 330k tokens, 100 requests 3.3M requests, 3.3M that could be ~5x less if used another harness. These 33k tokens are mostly because the harness is completely bloated with mcp, skils, plugin, loop, basic alignment and all sort of explanations needed in the system prompt for it to work the way anthropic wants it to... imo this is sub-optimal and if the model needs that much orientation, it says a lot about its intelligence too.. Good models should be as harness agnostic as possible
I am forced to use cloude code at work but a good solution is to just use --system-prompt "" and be done with it. I wish they allowed for other harnesses.
> --system-prompt ""
Doesn't the model need at least a basic system prompt to understand what tools are available?
The flag name is overloaded. It won't affect the tools available, just the other system instructions.
No, tool definitions are provided via some other mechanism.
Yep, have been using this for a long time now. No idea why everyone doesn’t.
Does it have any negative impact? If not, I’m not sure why this wouldn’t be the default behavior. It feels like Anthropic is just putting their foot on the scale to drive up costs or for the enterprise, or push consumers to higher subscription tiers.
I didn’t know you could do this. Is there any analysis of the impact, before and after? I’d love to see some charts of efficacy in real world usage.
It shows up in /context, but never spend time validating it much. Some people run a proxy to modify their messages.
Do you start Claude with this option? Or do you send this with every prompt?
yep I pass it to the CLI, I also pass --model
We should discuss cache performance if we haven't already. That 33k tokens may be a cache hit (I am not certain it's automatically a cache hit) but after the first call, it should certainly be a cache hit. Cache hit tokens are billed at 1/10th the price of cache misses. This is quite opaque, but it's necessary when you're asking "is the system prompt worth its stay" if you can save 33k tokens worth of dynamic discovery across the next few turns, the break-even point is quick and if the system prompt makes task performance increase and/or makes the system more autonomous so that it can string together more cache hits in a row, it becomes way way better. On a personal note, I think of things as aa function of 'supervised time to desired result' and 'cost'. because I find it harder to reason about tokens. I do think they could introduce a "minimal" mode (something like this is probably doable with the Claude agent SDK today)
Anthropic's cache expires after 1 hour when using subscription endpoints, and for those cached tokens cache reads are free. It's generous (compared to API pricing) but it's not 100% free.
Isn’t it 5 minutes ttl now?
I believe it's 5 minutes on API pricing by default, though you can turn off caching or force it to 1 hour. Subscriptions are special, seemingly because Anthropic doesn't want to expose casual users to all of the tokenomics.
It is not the raw prompt size that matters ultimately, otherwise Pi (and variants) would be the lowest costing agents. What matters is how efficient the prompt it. Prompt minimalism often gets conflated with efficiency. Having said that, CC does seem bloated for what it does.
What matters even more is tooling quality. Bad/buggy tooling causes a lot more roundtrips that wipes out all gains from initial greedy approach.
A few months ago, I did a full benchmark run of 7 agents over 8 tasks (https://github.com/dirac-run/dirac has the data and traces). I cannot claim neutrality because of the obvious connection to one of those, but the data should be reproducible and useful. Importantly, Dirac wins there significantly on those tasks because they are mostly refactoring related (which is where approaches like hash-anchoring and AST parsing tend to shine)
The good thing is that the competition in the field is very cutthroat with so many contenders, so if there are gains to be made they will be made, and then broadly adopted by others.
A harness is a part of the intelligence stack. It's no longer about raw access to the model
Also, I have seriously used most harnesses - One feels like it's being built in a place that truly understands AI and where agentic engineering is headed. You might not like it, but peak performance exists in CC when it comes to orchestration of bulk parallel work / subagents. The open source agents are catching up or accell in different way (Im preferable to pi.dev), but I'm not sure they're architecting orchestration the right why.
Early on in experimenting with local models, I found that hooking them up to Claude Code worked very well, but it was also really slow.
I used mitmproxy (setup assisted by Claude, natch) to capture Claude Code's entire initial system prompt and the whole thing was (I just double-checked) 162k of JSON.
This led me to start experimenting with Pi, OpenCode, and Hermes...
This is interesting, because if I start a fresh session of Claude Code right now and run /context, I see the following:
4k tokens is 15-20kB. I'd ask you to paste that into a gist, but it might have sensitive data in it, because I suspect what you're seeing is not just the system prompt.Apologies, you're right - I used imprecise terminology. The entire initial JSON structure that was sent from Claude Claude to the LLM at the start of a session was 162k. This included the system prompt together with a list of tools (some with very extensive explanations), MCP server details, etc.
I was simply supporting the article's data - their reported 33k tokens is probably roughly 150-165k.
That’s entirely dependent on how many plugins, MCP tools, agents you have, and if you have pre-filling of all available tools enabled. Best way to avoid unnecessary expense is to avoid it all and use CLI tools instead.
Agree. It's a fairly minimal list with very few extras added.
Current /context on a fresh session (compare to that above) is:
A lot of people will just add as many tools as they can think of. I don’t think it’s obvious that this costs money.
A smarter approach (progressive disclosure) for tools has been implemented by (I presume all) the harnesses over recent months, but you're 100% right in any case.
I enable tools specific to each project only in that project, and have very very few in my global config. Like <5k tokens worth.
Ah that makes sense, wasn't trying to be pedantic. Thanks for clarifying.
With Fable being per token instead of on the subs (unless they changed it again?), I decided to test Claude code on OpenRouter where I had some credits, with Opus 4.8 and Fable 5.
I asked both a trivial question (summarize last commit). Opus cost 50 cents, Fable about $1.
That checks out because Fable's twice as much in the API (though I think its emphasis on correctness makes the difference larger for bigger tasks).
But, at $1 per question, I think I will stick to the subscription for now! I was certainly glad GPT-5.6-Sol is included in OpenAI's subscription, and I'm curious if they'll be able to do the same for GPT-6.
All the VC money appears to have run out a few weeks ago.
As for context size and harnesses I did make a trivial bash agent based on this "agent in 50 lines" tutorial[0] recently, and found that for trivial work, it was about an order of magnitude cheaper and faster.
I haven't tested it on anything bigger but it doesn't seem to do the kind of proactive testing, that they do in bigger harnesses.
Codex at least has a system prompt that tells it not to consider a feature a complete until it has verified it. I'm not sure about Claude Code.
I suppose I could add that one line to the prompt, and it would get me much closer to agi :) I think Fable does this proactively even without a prompt, but I haven't tested that yet.
If Fable in my own harness is significantly cheaper than Claude Code, that would be very appealing. (I could actually afford to use it for most things!) But I think most of the cost comes from the testing it does. So we'll have to see.
[0] https://minimal-agent.com/
Fable's subscription inclusion theoretically ends EOD today. Anthropic put a wishy-washy "if we have capacity we'll continue it" thing, and given how competitive GPT 5.6 Sol is, and it is included in OpenAI's subscription, I fully expect Anthropic to extend Fable or they will have a serious exodus on their hands.
Competition is good.
Anthropic have extended Fable access again to July 19. The notice should pop up in your Claude Code now when you start a new session (also announced on the ClaudeDevs X account first).
Ah, thanks. It's been hard to plan around these last-minute changes. I rushed to implementation on a spec I should have spent more time on because of the looming deadline.
Mine sends even less - https://maki.sh
Nice!
> When context gets too long, maki compacts history automatically: strips images, thinking blocks, and summarizes older turns.
Don’t the summaries of older turns effectively invalidate the context cache, such that you consume less tokens but more expensive tokens?
Only once per compaction
This is like saying contractor (A) asked for $33,000 to undertake the work and contractor (B) asked for $7,000
Are we measuring and caring about the right thing?
Anecdotally, the results from OpenCode + Claude appear to be the same if not better for our uses over the past year.
> A small task that cost 121,000 tokens done directly cost 513,000 tokens when fanned out to two subagents, because every subagent has its own bootstrap cost, and the parent then consumes its transcript.
Is that true? My understanding is that the subagent only returns the result of the request without the main agent consuming its entire transcript.
pi sends 1k (or less) -> https://github.com/earendil-works/pi/blob/main/packages/codi...
My $20 sub using gpt 5.6 sol thinking-off lasts for hours using pi.
Why turn thinking off? I mean, yes, it uses less tokens, but you're using the best model OpenAI offers, but then making it as dumb as cheaper models.
Interesting question - what I learned doing that is that sol ends up injecting its thinking traces as code and shell terminal comments. YMMV, but in my case what I saw was sufficient to use with my routine work in my projects
We are yet to try Pi!
Is it not a conflict of interest for a model provider to supply the harness? They are not motivated to minimize your costs.
They sort of are, in that they want subscription users to have clients that behave well with the KV cache etc.
If you don't use a subscription, and pay per token instead, you can easily move to another harness.
I've been trying various harnesses like Pi, OpenCode, Qwen Code, and Nanocoder. A common problem I keep running into is failed tool calls, regardless of the model. What is the best harness and on-device model combination right now?
Pi.dev requires some plugins to work well. Using Qwen3.6-27B/35B locally at Q8, I was quite frustrated with failed tool calls and tried many things.
Ultimately this combo worked:
1. https://pi.dev/packages/pi-tool-guard —- corrects key name synonyms and common structure errors, so tool calls succeed automatically (e.g if the model hallucinates old_str instead of oldText). It also wraps top level oldText/newText in an edits array if the tool didn’t do it.
2. https://pi.dev/packages/@aboutlo/pi-smart-edit - white-space-tolerant edits, as Qwen would sometimes add a fifth space to a four space indent
Hashline edit tools didn’t work well for me at all, they confused the model and it still failed to edit correctly. Also line removals would invalidate the rest of the file requiring re-reads. I tried pi-hashline-edit-pro, though I see it now keeps a database of hashes to help keep them stable across edits. Regardless Qwen kept thinking that the hashline prefixes were part of the source.
You can't afford the best model. What are your specs and what models + quants have you tried?
Qwen 3.6 35B A3B and Qwen 3.6 27B can both do reliable tool calls on Pi at Q4_K_M using llama.cpp
I'm on a 48 GB M5 Macbook Pro. I use 4-bit quants with a context window of 16-32k. I tried Qwen 3.6 27B, but I can only get around 10 tokens per second, but it's painfully slow, and it often fails during `write_file` tool calls, even with Qwen Code.
Try an 8 bit quant of Qwen 35B, but temper your expectations. Those Qwen 3.6 models are impressive for the size, but you need an order of magnitude more parameters to actually be useful for more than trivial work in my opinion.
> and on-device model combination right now
That would depend entirely on what your device is. This sounds likely not to be an issue with the harness, but the capabilities of the models you've tried.
I experience almost no tool call failure using my nothing-special harness and DSv4 Flash.
I'm looking for something that runs on an M5 Macbook Pro with 48 GB of unified memory.
I have just re-analysed most common failed tool-calls and adjusted the tool so that it works. I have a manual repair step on failure that programmatically attempts to fix some things. On failure, the harness reports the error, the repaired function, and the result. Overall, seems to work fine. But it's very model-specific. Most commonly the model fails on shell commands where it hallucinates some programs. If it does it often enough, I just promote those to commands in the PATH. Over time, it has happened less.
I recommend that Opencode users try Dynamic Context Pruning as well: https://github.com/Opencode-DCP/opencode-dynamic-context-pru...
It works great for long-horizon tasks, and feels like it saves a boatload of tokens.
The Sleev (the project has been renamed to make a startup) creator was shilling their project in the OpenCode Discord. That person is very convinced they have something that no one has ever built before. They focused on token reduction without any real evals for capability impacts.
I'm generally against this context pruning without prompting or details. Sleev is very opaque about how it works and definitely will bust your cache.
It's definitely not unprecedented, but the plugin version is useful. Sleev seems like a nothingburger, I'm happy with the results I get from DCP already.
Doesn't that mean these 33k tokens can be cached, since they don't depend on the input? The model can just start generation on the 33k+1th token.
right, but the cache retention time is very short for Anthropics LLMs. 5 minutes or 1 hour (with additional costs). So you have to prompt basically non stop to not get a cache eviction.
Anthropic even changed this silently: https://www.reddit.com/r/ClaudeAI/comments/1sk3m12/followup_...
A const prompt across all of Anthropic's subscribers could draw from a global cache rather than per-user?
Although saying that out loud makes me question it - each per-user chat and growing cache would need eventually to own its own ~contiguous memory block.
Yes, but all subsequent tokens do depend on them, so they are still being charged for
Noob question, but is there, on a harness level, a way to compress or optimize the "pre processing prompt" or any other preamble?
I know you can give instructions to guide an agent, but they are still plain English, the interpretation still isn't absolute and the more rules, the less likely it is to respect the whole set of rules in the preamble, but the whole process seem so... "Unprofessional"?
As I see it, a tool should be reliable and the less bloat, less possible points of failure.LLMs seem overly verbose and you can't shake the feeling it might veer away from the objective at any point. You can pay to mitigate or work around the risk, but it still doesn't inspire confidence in anything with more than 30 lines
I pretty much giving up on Claude. It's Extremely slow, especially fast few days. Computer use takes so long to do anything that sessions expire. I switched to Codex and it's a different story. I'm not saying that Anthropic is not going to fix their problems, but for me as a user that depends for the service to work, it's not acceptable. So, we switch back and forth between a few LLM's but Claude and Codex are the leaders and I can't afford using anything else if I know I can get better quality results. The cost is important but not as much as the quality of the solution. You might save a few bucks on a cheeper model but you will have to prompt it many times and time is more important to me, as well as the feeling that I'm getting the best result on the first try.
Sending 0k tokens would be a smaller number again. But then it might have no idea of what it is doing. I pay my subscription and get lots of tokens on good models - if I was paying per token I might care more.
In a pay per token situation, there is a huge conflict of interest with the harness provider and the token seller being the same party ... efficiency is less profitable.
I have accused claude code of trying to run up the meter on me and it confirmed I was absolutely right.
> In a pay per token situation, there is a huge conflict of interest with the harness provider and the token seller being the same party ... efficiency is less profitable.
Except there’s a competitive incentive to either use less tokens or make the tokens go further
We are still in a (brief) era where companies are awarding pizza parties to the employees that burn the most tokens.
There is a world where, to hit the next quarters revenue projection, you add 2,000 tokens to the system prompt and "beat" expectations at the next earnings release.
> based off of a hunch
This is posed as some sort of discovery, but both Claude Code and OpenCode display token usage clearly after starting a chat or agent, and 30k and 7k is exactly what you see.
This is all heading in the right direction. Much of AI coding feels magical. But when the costs begin to accrue we start asking questions. We dig into it and try to understand what's going on. I can't help but feel Anthropic is "token maxing" from its side: it controls the levers and with every version upgrade it can build in its own token growth almost unbeknownst to the user. This actually harms it on the long run because it necessitates a cheaper option.
I've been using Claude since January, and whenever I run CCUsage I've been noticing the overall cost creep up pretty much every month. (I'm on a subscription so this would be the hypothetical billing if I were on API prices I suppose. Although I don't really want to test that with my credit card.) The funny thing is the first few months I was token-maxxing as hard as I could just to see how awful of a bill I could run up (mostly as a curiosity thing). At this point the novelty of doing that has worn off for me, but even with me being pretty conservative in my usage now the cost is way higher. I think I was spending like $12 a day in january, and now I'm easily spending $60+ a day in part time work. The amount of work I've been doing has stayed relatively constant (I'm not trying to run agents in parallel or loops or whatever fancy new ways there are to burn money, this is the same workflow I've had since the beginning. The codebase has grown, but I assume input tokens are not to blame for the big cost increases)
It's wild that people will continue to pay Anthropic when there's a better, faster model available for a tenth of the price.
One honesty note before my comment (yuck), it's super frustrating to read an LLM produced blog that could be 1/3 the size. Why are you talking about MCP's when comparing OpenCode and Claude Code when they both support that technology?
The only interesting thing is whether OpenCode is more effective at writing code with it's reduced system prompt.
Claude Code sending 33k tokens before reading the prompt is the AI equivalent of a consultant who bills you for the time spent reading your email before they even open it.
Well, I have to open the lid on my computer and remember my password, no?
The OpenCode CLI does not work as well for me as the PI CLI. I'm a subscriber of OpenCode Go (the sub, good value for me really) but I had not great experiences with OpenCode CLI. It multiple times with different models deadlocked itself into listing endlessly to non ending processes (Android Debugging Bridge, COM serial log, ...). There was also a problem where the OpenCode CLI would crash after sometime with a Bun error.
I switched to the PI CLI and have no problems with hanging processes anymore. OpenCode Go allows for API access so I'm keeping this sub.
I'm surprised most of that isn't cached token usage. It's true that increasing length is a problem on its own because the model needs to attend to it all, but with caching it should be pretty fast anyway. My system prompt is quite large and I haven't noticed much of a generation penalty in the range from 5k to 10k.
The reasoning built into the models matter so much too. I recently swapped my Qwen3.6 27B to ThinkingLabs’ fine tune and it does what it publishes. I cut my token usage in half, which is a big deal since I only get ~20 TPS for token generation.
We've started using claude code at work and I don't understand the hype. I've been using codex and grok build at home, and they're both faster and in some cases better. Claude has a tendency to do too much. If I don't ask for unit tests and they're not in my agents.md file, then I probably don't want them. It'll try to make new libraries and classes for things that should just be a new function or a comparison check.
In our case the alternative was nothing so I'm happy to have it, but currently claude is not as competitive as I'd have maybe expected given the hype
[flagged]
Its funny, I have seen people look at the sourcecode of opencode and complain about excessive token use, the code to tell the model how to execute bash commands, and they complained how this was way too excessive, teaching the LLM how to write bash.
Would be interesting if you could post more details about what you captured.
Claude code seems to have a pretty robust harness and memory. It's a time saver to not include project context in our prompt, since Claude manages claude.md but I haven't used OpenCode to compare.
No surprise, I've noticed that "agents", not only CC (I am using Copilot) are trying to be "clever", searching for a lot of data. This is good for LLM providers as this eats a lot of tokens.
OpenAI, to their credit, seems to be focusing pretty heavily on token efficiency in GPT 5.5 and beyond.
I've been trying out OpenCode recently, because of the US embargo on frontier models, and found it to be as good as Claude Code, if not better. And it read all my skills, claude.md files, etc. Now I just need to pick a model out of all the choices - currently Deepseek v4 Pro is winning, but I want to try a few more.
Reality is- Anthropic is a tokens dealer. If they can hook you up for bigger spend -> they will.
We already know company is not making any profit. To break even they need ppl to use a lot more tokens AND pay for them premium price.
We also know LLMs dont give such a huge productivity boost do warrant spending of THAT size.
At this point you only wait for more and more shady plays.
I still think the best way to build software using LLMs is to copy-paste snippets/files into the chat and manually guide the work. Humans are still the best orchestrators. Yes the human has to now be hyper-focused and juggle various workflows, but the end result (quality of work and throughput of usable code) becomes very good.
Been my experience as well. Human corrected code is orders better than ai generated slop. Slop might get you the promotion at your day job who is still in fomo tokenmaxx mode, but highly recommend the careful review for your personal projects
I find it hilarious people think they can build on slop or on captured output from quality engineers. Intent is the most important aspect, and no code base could ever capture that
Opencode did have an adapter to use Anthropic models before they were sent legal threats that scared them into nuking the repo.
Remember is it not OpenAI vs Anthropic as bad guys vs good guys. They are all bad guys trying to profit from your data while maximizing dependency. Just buy or rent GPUs.
Beginning-of-prompt common prefixes are very cheap for the provider depending on their caching architecture.
Wouldn't be surprised if you're paying full price for those tokens, but they cost ~$0 for the provider.
I used both with Openrouter and GLM 5.2 and I can confirm this is the case. Claude Code burned $10 per task while Open Code burned barely $10 a day which wqs about 4-5 tasks a day. A task usually included database migrations, code audit or documentation.
Claude Code is not just a harness. It is a different product. You pick the smallest subscription that allows you to do your work. My “multiplier” on a $100 subscription is 5+.
If you’re using API, on the other hand, there is absolutely no reason to use Claude Code, or Codex.
Selfplugging a bit here: clai[0] sends only exacly what you want it to send.
[0]: https://github.com/baalimago/clai
UPDATE – 13/07/26:
1) We have added a repository reproducing the methods.
2) We have commented on output quality in the post (in addition to tokens/cost).
3) Per a previous update, we now handle Fable 5 in the post.
But Claude Code in my experience results in more tool calling for smart efficient file reading. Meanwhile Opencode pulled an entire 500kb file (GPU assembly dump) at once. Kilo is better than both, as it uses indexing.
> Claude Code 2.1.207 and OpenCode 1.17.18, both pinned to claude-sonnet-4-5
So not only is this article AI-written, but the testing was entirely done by AI, too? I can't see any other reason to use such an old model.
> Our traffic passes through a local LLM gateway that wraps requests in its own envelope, a constant we measured at roughly 6,200 tokens with bare calibration requests
Why do you need to do calibration requests to figure out how your own gateway is affecting requests?
> Its subagent lane did not complete cleanly through our gateway
> We attempted to toggle extended thinking in both harnesses and are declining to publish numbers. Our gateway applies its own thinking policy, neither harness's toggle demonstrably survived the path, and anything we quoted would be noise.
Why is your own gateway screwing with your testing?
Model:
Cost, mainly. The runs went through a Claude Max subscription rather than metered API billing, and pinning an older stable snapshot kept run-to-run comparisons clean and cheap. The fixed harness payload (system prompt plus tool schemas), so the headline numbers shouldn't change too much.
That said, happy to re-run the matrix on Fable and publish the diff; payload figures should barely move, tool-calling behaviour might.
Gateway:
Meridian (github.com/rynfar/meridian); proxy that bridges the Claude Code SDK to a standard Anthropic endpoint so a Claude Max subscription can drive OpenCode-et-al.
It's the auth route for all agent traffic on the machine, not something built for the benchmark.
I recently started using cline instead of opencode and prefer the interface. I'm interested if anyone here here has any arguments for opencode or codex over cline?
Why don't we have some equivalent of "fork" if we are talking the same context and tokens, you'd think that could all just be loaded into the gpu.
OpenCode, Crush and Pi do have the ability to fork a conversation. But cache reuse is up to the provider and not guaranteed. At some point you need to forward the cache to a more recent checkpoint, and you have a finite (unknown) number of parallel cached chats.
Proof the real genius was in the mathematics not the AI "engineers"
I feel like this article isn't saying much. Even with tools disabled, Claude Code still has a crap load of commands and other things that Claude (the model) should know the availability of since it's optimized for them. All of that has to be disabled if this is to be a real harness comparison. And of course the system prompt can be completely replaced, making it a no-brainer to use a more minimal prompt similar to OpenCode. And beyond that nothing else really matters because the rest (cache behavior, etc) lies with the provider's platform, not the harness.
Sorry for asking here, but nobody seems to know.
If I self host a local model is there some way to make Android studio not time out after 10 minutes?
I think this doesn't mean much; the axes that matter are intelligence x dollars x time; tokens by themselves mean nothing.
Nothing about the time taken to complete the task? Users are definitely sensitive to time, not only token consumption.
https://archive.is/O2BFs
It adds a TON of skills by default your project might not even need.
that makes sense, claude code actually does inflates token usage
Why don't people fix their costs (rent a gpu) and just write their own harness (about 200 lines of code).
Supposed to be hacker news and half the posts are like "this harness steals this" like it cant be avoided.
These API costs are mad.
GLM isn't good enough yet.
It pays to be marginally ahead of people stuck on open models.
I have been using claude code for a big project for a while and I feel like I have optimized my workflow now.
Brainstorm - Gemini/Antigravity
Plan - Gemini/Antigravity
Detailed Plan - Sonnet
Coding - Fable
Do not use any subagents, especially the default ones. They are dumb. The top level agent works well enough
Does your detail plan outline exactly what to implement/ lines of code to edit? I found if I get a nice detailed plan that sonnet is good enough to implement. Did you try that before and found fable better at implementing?
Yeah. So the benefit of Fable is that it'll find gaps in the system as well. So logic errors get caught. I have a separate backend and frontend so plumbing issues are also caught by Fable.
Check all MCP you have enabled in claude.ia
Anthropic wants to produce the best coding agent possible and doesn’t care (is even incentivized) about high costs. Other harnesses have to make trade offs between performance and cost.
Given they're incentivized to increase token use, what guarantees that higher token use improves the effectiveness of the agent and isn't just artificial padding?
Well, nothing really. But I assume there can be some benefits to modifying context. For example, updating file contents or marking them as modified, summarization, injecting additional information, removing irrelevant tool call results, etc.
Is there evidence that it is actually a better agent though?
There’s evidence it’s a worse agent actually. I’m just saying in theory.
And pi less than 1k
How do you perform worse in your own harness is beyond my understanding?
So? it doesnt matter, after the first turn it's cached. We are probably talking about single digit cents.
Maybe it is because anthropic prefers a larger system prompt?
https://github.com/asgeirtj/system_prompts_leaks/blob/main/A...
not even surprised
if you use them for long enoug tfor big project and conplex workflws, both of them have their own caveats.
Thats why i got irritated and wanted something thats scalable and lightweight.
https://github.com/corporatepiyush/yantra-coding-agent
caching
Imagine pi.dev…
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
[flagged]
[dead]
[flagged]
[flagged]
[flagged]
[flagged]
[dead]
[flagged]
[flagged]
[flagged]
[flagged]
We have never read your blog or your content before.
Suspect that many have covered the "Comparing agentic coding tools" angle before, and that the differentiator is depth of analysis + conclusions.
Do you think you were the first person to write a blog post about coding harness token usage?
Intellectual property is a dead concept.
Grok 4.5 is really fast, has more usage at $10/month than $20/month Claude pro, and Opus-level. Claude pro feels like a demo.
Claude is much better in OpenCode then in Claude Code, OpenCode is just better than Claude Code. Claude Code feels like a complete mess to use comparatively.
Elon saying that it is "Opus level" doesn't actually make it so.
True of anything anyone says about anything, including int_19h and simondotau.
I'm quite impressed with Grok 4.5 because its speed and single-task effectiveness feels better than anything else for a human-in-the-loop workflow. (For the stuff I do, I'm not interested in having AI race ahead of what I can oversee.)
All models do things in a way I personally disagree with at least some of the time. The "dumber" models sometimes fail to recognise how to fit a solution into existing code. The "smarter" models sometimes get too clever and over-engineer their solutions. Cleverness is occasionally interesting, but is just as likely to trigger a distracting rabbit hole where I spend time analysing whether something unexpected was a legit insight, or mere opinion.
I've compared them, have you?