So using this harness five qwen3.8-27b models roughly matched Fable 5 on a saturated coding benchmark from early 2025. No comparison to astra/6.1 sol/fable 5.1/opus 5.5. No comparison on a benchmark that reflects actual agentic work (and agentic behavior is what modern models optimize for anyway - probably makes qwen look worse than it is?). And also obviously using a very old benchmark like this:
> The updated release of the dataset with problems released between May 2023 and Apr 2025 containing 1055 problems
means essentially guarenteed contamination.
I have no idea if this harness is any good. Agressive orchestration could be interesting given how hard models are RLed on orchestration? But the methods used to evaluate this harness are not informative
I don't think we should trust 'small model beats giant frontier model' either... Don't get me wrong I think small models are and could be great, but matching something like fable 5?
I guess one point is you can run Qwen on a local machine.
It is not public knowledge how Fable 5 works - it is all hidden away behind a web service - for what it is worth it might a hybrid architecture with harness, loops, tools etc. combined with a traditional llm like what is done here.
Every role but the verifier is the same
underlying model, called in a fresh context
and coordinating through a shared workspace on disk:
<ws>/task.md - the problem statement
<ws>/plan.md - the manager’s overarching plan
<ws>/tasks.json - a debug mirror of the task list
<ws>/notes.md - curated ideas / findings / partial proofs
<ws>/solution.py - the current solution
Then they describe how there is a ping-pong between manager and worker with fresh context for the worker, but i wonder, is that part or even the worker really necessary?
My use of Qwen 3.8 27B (which is the only model i use almost since it came out) is really doing something very similar to the above where first a task is described, a `PLAN.md` file is made with the (current) plan and tasks are either in the plan file or in a separate issue tracker the LLM has access to via a CLI tool, a `STATUS.md` that tracks the current status and a `GOTCHAS.md` file for writing down various notes/tidbits/gotchas (TBH this is largely because Qwen 3.8 27B does want to note down stuff anyway so it mainly acts as an attractor for not writing its notes down in the other two places :-P).
There isn't any real harness driving this, i just give it a task, tell it how it'll be handled and off it goes. I guess i'm playing the "role" of the harness there as i tell it to not go and do everything in one go :-P but i never bother with fresh contexts, Pi's "rolling" approach to compaction (i.e. summarizing the older messages to make space while keeping the recent messages as-is) make it almost a non-issue.
I have considered making it a planning tool, though as some tool it can use via the CLI (it is running in a container), not a harness - but at the same time it feels like it can do the same thing with the `PLAN.md`/`STATUS.md`/`GOTCHAS.md` triplet. The only thing the tool would help with is to provide a UI for me to check what it's currently doing, but even then i usually just stop it and ask it directly :-P then tell it to continue.
Perhaps at some point i'll make some custom benchmark with stuff that are vaguely similar to what i work on and try a bunch of different approaches. It is something i want to do anyway to see how and if Qwen4 27B (whenever that comes it) is an improvement over 3.8.
> There isn't any real harness driving this, i just give it a task, tell it how it'll be handled and off it goes.
The tool that implements this is the harness. An LLM by itself cannot "go off" and "do things" - it can only complete text blocks. A harness bridges this to real world actions or just a chat box interface.
I can't edit it to clarify, but what i meant is that the process i use isn't driven by a harness but done "by hand", not that there isn't any harness at all. The PDF describes a harness that is actually used to drive the manager/worker role.
As i wrote in the other comment, i know, i just meant is that the process i use isn't driven by a harness but done manually. The PDF i quoted mentions a harness does that driving so i wrote "There isn't any real harness driving this" to mean that i do that by hand instead of being done automatically.
This is close to how I do it also. Except I have never interrupted it in a task, I always worried it would destabilize it and it would leave some function half finished or whatever. I just have standing instructions for it to pause places and ask “ready for the next phase?” Do you find it responds well and correctly to being interrupted in the middle of things?
The model itself (Qwen 3.8 27B specifically) can handle being interrupted just fine and i often interrupt it to clarify things when i see it finding my instructions vague or to add some extra info.
However there is a bug somewhere between my instructions and the model - i think it is the jinja template, though i'm not sure as i haven't actually looked too deep into it - that in some cases make my last message be sent repeatedly to the model after each tool call, making the model think that i'm getting impatient or something :-P. When i notice it, i tell the model that this is some harness bug and it can ignore the repeated messages, which it does.
It seems to happen when i stop it while it is thinking after a tool call but i'm not certain. I try to stop it while some tool call is in process so it cancels the call as that seems to avoid the issue. Restarting llama-server also seems to make the issue go away so it might be a llama.cpp bug, but TBH i haven't found a clear repro case aside from it happening when i interrupt it while it is thinking. I think it is more common when the context is close to full, or at least i mainly remember it happening when i've been using a session for some time (i.e. not in fresh sessions, which does make reproducing it a bit harder).
In any case, the model itself seems to deal with all the above fine (and accepts the bug explanation for repeated messages), it is the rest of the stack that can show an issue with interrupting it (but even that is relatively rare).
Thank you! Yeah I think I would have experimented more with interrupting it by now, but my jobs only recently got longer and multi step. Plus I could never figure out, if I do want to interrupt this, what’s the best time to do it?
This is pretty close to a process I use (regardless of model size) and I would agree getting it into a workflow I can run with a one-liner is a good idea. There are a few ways of doing this and I have tended to fallback to markdown engineering. (custom agents and subs in opencode)
It's really interesting to see the progress in the 27-30B range, they have gotten quite good but still make noticeably more hallucinations or bad assumptions (ime).
size doesn't really have a correlation if u look at the hallucination benchmarks. this is mostly mitigated thru post training imo. also, deepseek 4.1 is notorious for hallucinations, so kinda ironic to mention it as an example here =)
> before the confabulation problem is really under control
It's definitely much better at bigger sizes, but I've still seen it at the frontier when coworkers ask me about changes their Ai made to code I originally wrote. (a good habit that organically emerged on the dev team)
Everybody is converging on similar ideas, but the execution isn't super clean in a lot of "software factories" popping up. Coordinating through shared written state becomes obvious once you build something complex enough. Anyways, I have been building Pullboard (open source). The original concept came from needing agent coherency on a very complicated project.
it definitely feels like sooner than later there will be “Mythos level” cyber threats available on open source / uncensorable models. I think the only thing that can be done is continued work on hardening anything that is remotely close to “critical infrastructure” and software supply chain.
On one hand, self hosting will always be niche so you could pass a law like “if your serving inference at scale you need to have safeguards/ classifiers in place” since > 95% of inference will go through hosted services in the long run … but that could turn into thought policing dystopia real quick.
For folks with 128GB inference machines that's definitely the more interesting model. 27B is dog slow on a Strix Halo. And, while Qwen 3.8 Flash is quite snug in 128GB at four bits (a bit more comfortably at three bits, but intelligence is really dropping by then) but is notably faster than the dense model. It thinks a little less and thinks a little faster, adding up to a real improvement. At four bits, it feels a little bit smarter than 27B even at eight.
If someone has one or two 32GB GPUs with decent bandwidth (faster than the Strix Halo or DGX Spark), the dense model is the obviously better choice, and I think that case is more common for the average developer.
I've been running Qwen 3.8 Flash Next on Strix Halo with halogen-flash-server which is a pretty good experience (although unfortunately halogen is closed source).
But for coding and cybersec tasks I think GLM 5.3 might be the best one to use still.
Nobody's running GLM 5.3 on a Strix Halo. But, yes, I'm using cloud models for everything hard, including coding. I use local models for smaller API tasks or sub-agent tasks and experiments with fine-tuning and benchmarking.
… on one benchmark, that has 2 modes of running. 1 shot and 1 shot with repair. Zero tools.
Then ignore the rules of the bench (which is a pretty obsolete one anyway), give it an agentic loop of sorts via delegation and python eyes and then win.
Harness changes dynamically depending on your LLM intelligence.
The smarter LLMs get, you apply less, as LLM takes care of pre-existing harnesses.
This is a shitty, unreliable, and unmaintainable way of using harness because the harness is built around for the sake of getting good benchmark scores (someone mentioned overfitting in comments, so that)
What are people using for a local offline harness? I've heard that OpenCode has lax approach to security and has some default dial home settings. Aider? Want to use Qwen3.8-27b locally for coding on 24gb VRAM. Best option?
I wrote my own harness designed to feel like a unix util. If you're worried about security, mine is a single file shell script less than 1,000 loc. It only depends on llama-cli and a few posix shell tools. So it has no way to dial anywhere and no security problems hiding in some dependency.
But it's not optimized for coding specifically. It can edit files fine using Aider style edits. I've used it to write to do lists as a test. But I more designed it with the idea to be my personal debugging tool if I ran into any weird things while using Linux on a laptop.
It just barely fits in my RX 7900 XTX's 24GB of VRAM (TBH i'm not sure if dflash is much better than MTP - which is included as part of the GGUF - and MTP saves ~1GB of VRAM). I'm using Linux with Xorg/Window Maker as my DE so it uses no VRAM at all (Qt-based stuff - and i guess modern Gtk too - tend to use some VRAM for themselves, even opening Kate shows up in amdgpu_top :-P). The above gives me on average ~50 tokens/sec generation (drops to about 35 when the context is close to full) and ~700 tokens/sec prompt processing, which while not particularly speedy is fine for agentic stuff as i just give the AI a task and leave it in the background doing things. To save some VRAM i run the multimodal projector for vision on the CPU (the --no-mmproj-offload) and only one task is ever executed at the same time (--parallel 1). Also the kvcache is quantized to q8_0 - supposedly this makes the model a bit dumber, though the extra VRAM needed for the full f16 kvcache makes it more than ten times slower :-P (and AFAIK dense models like the 27B aren't as affected by it as MoE models are).
I'm using Pi[1] (a slightly older version that i've had remain offline and disabled the telemetry) for now (will replace it at some point since i'm not a fan of the ~500MB dependency tree, the almost-dark patterns for online access/telemetry and the increasingly worrying push for the devs' services, but i do like how it approaches compaction and that for the most part it doesn't need any handholding when some error happens - which is ideal for "let the AI do something and check it some time later").
It is running inside a container so it doesn't get any access to my real stuff (mainly to avoid the LLM making any mess - of which it sometimes does in the container, but well, it is contained :-P). In the container Pi gives the LLM some basic tools - read, write, edit and bash - and the last one is basically used to do most of the work (i've also made a bunch of tools the LLM can use via the CLI).
Yeah though AFAIK it is discontinued now (or at least i can't find it in the local price aggregator). I think R9700 is a decent alternative, a bit slower (256bit bus vs 384bit bus) but has 32GB of RAM, still isn't too overpriced (at least compared to Nvidia alternatives) and has an opensource stack too (which is why i went with AMD in recent years).
I'm in the same boat. I was the primary advocate for fixing the config security regressions in the OpenCode tool (separate from the console hack days after the fix). My impression is that they are improving quality across the board. I will not being updating to v2 because I do not jive with the new UX. I would recommend checking it out, it's the best open harness I've used so far, though I'm only interesed in TUIs.
I'm currently using the Pi harness as well, it's closer to OpenCode v1 with a few plugins. However, I have zero faith in their security or maturity after several communications with the CEO. Pi is also developed by an Israeli team, if that is something that matters to you, re: Mossad concerns or Palestinians rights.
I'm going to go back to my own custom harness and do a full rewrite, now that I have seen what 6+ other harnesses offer.
We may both be partially right, the main dev I interacted with has Tel Aviv on his GitHub (also 'Hanna' being around the project, but apparently that is a surname not a first name, and a mistaken extrapolation on my part)
The lack of professionalism or seriousness was mainly in a private email chain with Armin. He added a single comment to this issue after that chain which was the gist of a couple of his replies, less snarky. Either way, the interaction left me disappointed, not a founder who seems interested in listening or learning imo.
How do we provide org / ops determined configuration that users and agents cannot change while still giving them some flexibility to set the values we do not determine?
I'm not surprised. I used Claude Code for the first time again since April because I had been hearing good things about Opus 5.5 and Fable 5.1, but I was shocked by how bad the harness is and how its only gotten worse since spring? I may be doing something wrong, but getting Opus/Fable to properly spin up non-blocking sub-agents takes like four or five attempts and it still cannot reliably poll tasks like test suites that take more than 30 seconds for proper completion.
I think even if you have the world's best model (not my opinion, but seems to be the popular consensus these days with Anthropic), if the harness around it sucks none of that intelligence is going to matter.
I am routinely using swarms of agents, with complex sequencing between them and agents watching other agents, and then triggering forks and/or new design cycles. My normal flow is to have a few design tasks open per codebase during the day, with each design agent aware of the other sessions, and having an unattended nightly build which sometimes takes 8hrs and also requires coordination for tests, merges, and so on.
I experienced a regression some days ago, where Claude Code with Opus 5.5 wouldn't progress as expected, getting stuck on steps. So I had to repeatedly ask it what the status was and to proceed with X, and so on.
But that was just one day, the days before and after it has done lots of fairly autonomous work, including spinning off multiple subagents to run conformance tests which take 30-50 mins and work on other stuff in the meantime, then acting on the results autonomously.
Not fanboying or playing favorite, but I've never come across what you mentioned while using Claude Code. You may indeed be doing something wrong or some part of your config isn't right.
I use Copilot CLI at work, and Claude Code at home, and all harnesses are pretty good these days.
Hey, Boris from the team here. This is pretty surprising, given background subagents are the default. Could you confirm you’re running the latest, and run /bug and paste the id here? Happy to debug what’s happening
First off you need to pick a harness that you can use with every model. This way you isolate model problems from harness problems. If you had to switch harnesses (Claude Code) in order to try out Fable, you're doing something wrong.
Secondly yeah, all models have issues with foreground vs background subagents. So I just modified my harness to delete foreground subagents. No more confusion. If they need the subagent's output tokens they can read it from the session transcript.
"Take away tools" is often the best solution for model-gets-confused problems. Narrow down your toolset until there's only one way to do things, and then make it the maximally-powerful way.
So using this harness five qwen3.8-27b models roughly matched Fable 5 on a saturated coding benchmark from early 2025. No comparison to astra/6.1 sol/fable 5.1/opus 5.5. No comparison on a benchmark that reflects actual agentic work (and agentic behavior is what modern models optimize for anyway - probably makes qwen look worse than it is?). And also obviously using a very old benchmark like this:
> The updated release of the dataset with problems released between May 2023 and Apr 2025 containing 1055 problems
means essentially guarenteed contamination.
I have no idea if this harness is any good. Agressive orchestration could be interesting given how hard models are RLed on orchestration? But the methods used to evaluate this harness are not informative
I do not think we should trust this harnesses claims.
I don't think we should trust 'small model beats giant frontier model' either... Don't get me wrong I think small models are and could be great, but matching something like fable 5?
Also why not comparing harnessed Qwen to harnessed other LLMs.
I guess one point is you can run Qwen on a local machine.
It is not public knowledge how Fable 5 works - it is all hidden away behind a web service - for what it is worth it might a hybrid architecture with harness, loops, tools etc. combined with a traditional llm like what is done here.
Fair points. But harness at this point makes more difference than using the best model.
Interesting but (from the PDF):
Then they describe how there is a ping-pong between manager and worker with fresh context for the worker, but i wonder, is that part or even the worker really necessary?My use of Qwen 3.8 27B (which is the only model i use almost since it came out) is really doing something very similar to the above where first a task is described, a `PLAN.md` file is made with the (current) plan and tasks are either in the plan file or in a separate issue tracker the LLM has access to via a CLI tool, a `STATUS.md` that tracks the current status and a `GOTCHAS.md` file for writing down various notes/tidbits/gotchas (TBH this is largely because Qwen 3.8 27B does want to note down stuff anyway so it mainly acts as an attractor for not writing its notes down in the other two places :-P).
There isn't any real harness driving this, i just give it a task, tell it how it'll be handled and off it goes. I guess i'm playing the "role" of the harness there as i tell it to not go and do everything in one go :-P but i never bother with fresh contexts, Pi's "rolling" approach to compaction (i.e. summarizing the older messages to make space while keeping the recent messages as-is) make it almost a non-issue.
I have considered making it a planning tool, though as some tool it can use via the CLI (it is running in a container), not a harness - but at the same time it feels like it can do the same thing with the `PLAN.md`/`STATUS.md`/`GOTCHAS.md` triplet. The only thing the tool would help with is to provide a UI for me to check what it's currently doing, but even then i usually just stop it and ask it directly :-P then tell it to continue.
Perhaps at some point i'll make some custom benchmark with stuff that are vaguely similar to what i work on and try a bunch of different approaches. It is something i want to do anyway to see how and if Qwen4 27B (whenever that comes it) is an improvement over 3.8.
> There isn't any real harness driving this, i just give it a task, tell it how it'll be handled and off it goes.
The tool that implements this is the harness. An LLM by itself cannot "go off" and "do things" - it can only complete text blocks. A harness bridges this to real world actions or just a chat box interface.
I can't edit it to clarify, but what i meant is that the process i use isn't driven by a harness but done "by hand", not that there isn't any harness at all. The PDF describes a harness that is actually used to drive the manager/worker role.
No, pi is the harness. You may have chosen the most minimal harness available, but it's still a harness.
As i wrote in the other comment, i know, i just meant is that the process i use isn't driven by a harness but done manually. The PDF i quoted mentions a harness does that driving so i wrote "There isn't any real harness driving this" to mean that i do that by hand instead of being done automatically.
This is close to how I do it also. Except I have never interrupted it in a task, I always worried it would destabilize it and it would leave some function half finished or whatever. I just have standing instructions for it to pause places and ask “ready for the next phase?” Do you find it responds well and correctly to being interrupted in the middle of things?
The model itself (Qwen 3.8 27B specifically) can handle being interrupted just fine and i often interrupt it to clarify things when i see it finding my instructions vague or to add some extra info.
However there is a bug somewhere between my instructions and the model - i think it is the jinja template, though i'm not sure as i haven't actually looked too deep into it - that in some cases make my last message be sent repeatedly to the model after each tool call, making the model think that i'm getting impatient or something :-P. When i notice it, i tell the model that this is some harness bug and it can ignore the repeated messages, which it does.
It seems to happen when i stop it while it is thinking after a tool call but i'm not certain. I try to stop it while some tool call is in process so it cancels the call as that seems to avoid the issue. Restarting llama-server also seems to make the issue go away so it might be a llama.cpp bug, but TBH i haven't found a clear repro case aside from it happening when i interrupt it while it is thinking. I think it is more common when the context is close to full, or at least i mainly remember it happening when i've been using a session for some time (i.e. not in fresh sessions, which does make reproducing it a bit harder).
In any case, the model itself seems to deal with all the above fine (and accepts the bug explanation for repeated messages), it is the rest of the stack that can show an issue with interrupting it (but even that is relatively rare).
Thank you! Yeah I think I would have experimented more with interrupting it by now, but my jobs only recently got longer and multi step. Plus I could never figure out, if I do want to interrupt this, what’s the best time to do it?
This is pretty close to a process I use (regardless of model size) and I would agree getting it into a workflow I can run with a one-liner is a good idea. There are a few ways of doing this and I have tended to fallback to markdown engineering. (custom agents and subs in opencode)
It's really interesting to see the progress in the 27-30B range, they have gotten quite good but still make noticeably more hallucinations or bad assumptions (ime).
Yeah unfortunately the confabulations are the real problem with models of this size.
IMHO you need at least 500B weights before the confabulation problem is really under control. i.e. DeepSeek 4.1 or GLM.
I admit some days I'm just lazy and throw a whole Kimi or QwenMax at it.
size doesn't really have a correlation if u look at the hallucination benchmarks. this is mostly mitigated thru post training imo. also, deepseek 4.1 is notorious for hallucinations, so kinda ironic to mention it as an example here =)
Deepseek 4.0 is far worse at hallucinating.
It used to be my go-to "cheap model". I'm glad I have 4.1 now for that.
> before the confabulation problem is really under control
It's definitely much better at bigger sizes, but I've still seen it at the frontier when coworkers ask me about changes their Ai made to code I originally wrote. (a good habit that organically emerged on the dev team)
Everybody is converging on similar ideas, but the execution isn't super clean in a lot of "software factories" popping up. Coordinating through shared written state becomes obvious once you build something complex enough. Anyways, I have been building Pullboard (open source). The original concept came from needing agent coherency on a very complicated project.
https://github.com/pullboard-dev/pullboard
Wait, you can write a paper about this?
This is literally just how I've been using QFN all this time.
Seems like it just overfitting the problems?
allot of the models are just constantly eval maxxing
Very impressive given the cost, especially since running locally you just have one time acquisition of hardware and then electricity.
10KW+ of solar could help ;)
You only need about 1kW to run a local inference machine. Even a 5kW array is overkill if you are using it during sunlight hours.
> Even a 5kW array is overkill if you are using it during sunlight hours.
Angle of incidence, time of the year, daylight hours, battery charging.
Great progress the open world is making, thanks for sharing! Soon Qwen 4 will proof to be another step in the right direction.
it definitely feels like sooner than later there will be “Mythos level” cyber threats available on open source / uncensorable models. I think the only thing that can be done is continued work on hardening anything that is remotely close to “critical infrastructure” and software supply chain.
On one hand, self hosting will always be niche so you could pass a law like “if your serving inference at scale you need to have safeguards/ classifiers in place” since > 95% of inference will go through hosted services in the long run … but that could turn into thought policing dystopia real quick.
It's poignant how available these open doors were to resourced intelligence agencies - and still are today.
Opens trench-coat: so you want to buy some compute
the qwen flash next result is the actual interesting part. The 27b dense model has more active parameters than, e.g. latest deepseek flash.
For folks with 128GB inference machines that's definitely the more interesting model. 27B is dog slow on a Strix Halo. And, while Qwen 3.8 Flash is quite snug in 128GB at four bits (a bit more comfortably at three bits, but intelligence is really dropping by then) but is notably faster than the dense model. It thinks a little less and thinks a little faster, adding up to a real improvement. At four bits, it feels a little bit smarter than 27B even at eight.
If someone has one or two 32GB GPUs with decent bandwidth (faster than the Strix Halo or DGX Spark), the dense model is the obviously better choice, and I think that case is more common for the average developer.
I've been running Qwen 3.8 Flash Next on Strix Halo with halogen-flash-server which is a pretty good experience (although unfortunately halogen is closed source).
But for coding and cybersec tasks I think GLM 5.3 might be the best one to use still.
Nobody's running GLM 5.3 on a Strix Halo. But, yes, I'm using cloud models for everything hard, including coding. I use local models for smaller API tasks or sub-agent tasks and experiments with fine-tuning and benchmarking.
… on one benchmark, that has 2 modes of running. 1 shot and 1 shot with repair. Zero tools.
Then ignore the rules of the bench (which is a pretty obsolete one anyway), give it an agentic loop of sorts via delegation and python eyes and then win.
Sorry but this is not how benchmarking works
Harness changes dynamically depending on your LLM intelligence.
The smarter LLMs get, you apply less, as LLM takes care of pre-existing harnesses.
This is a shitty, unreliable, and unmaintainable way of using harness because the harness is built around for the sake of getting good benchmark scores (someone mentioned overfitting in comments, so that)
What are people using for a local offline harness? I've heard that OpenCode has lax approach to security and has some default dial home settings. Aider? Want to use Qwen3.8-27b locally for coding on 24gb VRAM. Best option?
I wrote my own harness designed to feel like a unix util. If you're worried about security, mine is a single file shell script less than 1,000 loc. It only depends on llama-cli and a few posix shell tools. So it has no way to dial anywhere and no security problems hiding in some dependency.
But it's not optimized for coding specifically. It can edit files fine using Aider style edits. I've used it to write to do lists as a test. But I more designed it with the idea to be my personal debugging tool if I ran into any weird things while using Linux on a laptop.
github.com/patrickjh/ssa
opencode connected to ornith 1.5 35b q8 running on a jetson agx orin
I'm using Qwen 3.8 27B with Unsloth's UD-Q4_K_XL quantization[0] via llama.cpp with 131k context - full command line:
llama-server --model Qwen3.8-27B-UD-Q4_K_XL.gguf --ctx-size 131072 --parallel 1 --cache-type-k q8_0 --cache-type-v q8_0 --host 0.0.0.0 --port 8080 --spec-type draft-dflash --spec-draft-model dflash-Qwen3.8-27B-Q4_0.gguf --spec-draft-n-max 3 --spec-draft-type-k q8_0 --spec-draft-type-v q8_0 --mmproj mmproj-BF16.gguf --no-mmproj-offload --image-min-tokens 2048 --reasoning-preserve
It just barely fits in my RX 7900 XTX's 24GB of VRAM (TBH i'm not sure if dflash is much better than MTP - which is included as part of the GGUF - and MTP saves ~1GB of VRAM). I'm using Linux with Xorg/Window Maker as my DE so it uses no VRAM at all (Qt-based stuff - and i guess modern Gtk too - tend to use some VRAM for themselves, even opening Kate shows up in amdgpu_top :-P). The above gives me on average ~50 tokens/sec generation (drops to about 35 when the context is close to full) and ~700 tokens/sec prompt processing, which while not particularly speedy is fine for agentic stuff as i just give the AI a task and leave it in the background doing things. To save some VRAM i run the multimodal projector for vision on the CPU (the --no-mmproj-offload) and only one task is ever executed at the same time (--parallel 1). Also the kvcache is quantized to q8_0 - supposedly this makes the model a bit dumber, though the extra VRAM needed for the full f16 kvcache makes it more than ten times slower :-P (and AFAIK dense models like the 27B aren't as affected by it as MoE models are).
I'm using Pi[1] (a slightly older version that i've had remain offline and disabled the telemetry) for now (will replace it at some point since i'm not a fan of the ~500MB dependency tree, the almost-dark patterns for online access/telemetry and the increasingly worrying push for the devs' services, but i do like how it approaches compaction and that for the most part it doesn't need any handholding when some error happens - which is ideal for "let the AI do something and check it some time later").
It is running inside a container so it doesn't get any access to my real stuff (mainly to avoid the LLM making any mess - of which it sometimes does in the container, but well, it is contained :-P). In the container Pi gives the LLM some basic tools - read, write, edit and bash - and the last one is basically used to do most of the work (i've also made a bunch of tools the LLM can use via the CLI).
[0] https://huggingface.co/unsloth/Qwen3.8-27B-GGUF/blob/main/Qw...
[1] https://pi.dev/
+1 to the RX 7900 XTX and also the RX 7900 XT model (20gb), much more affordable if you're trying to dip your toes into localgen
Yeah though AFAIK it is discontinued now (or at least i can't find it in the local price aggregator). I think R9700 is a decent alternative, a bit slower (256bit bus vs 384bit bus) but has 32GB of RAM, still isn't too overpriced (at least compared to Nvidia alternatives) and has an opensource stack too (which is why i went with AMD in recent years).
I'm in the same boat. I was the primary advocate for fixing the config security regressions in the OpenCode tool (separate from the console hack days after the fix). My impression is that they are improving quality across the board. I will not being updating to v2 because I do not jive with the new UX. I would recommend checking it out, it's the best open harness I've used so far, though I'm only interesed in TUIs.
I'm currently using the Pi harness as well, it's closer to OpenCode v1 with a few plugins. However, I have zero faith in their security or maturity after several communications with the CEO. Pi is also developed by an Israeli team, if that is something that matters to you, re: Mossad concerns or Palestinians rights.
I'm going to go back to my own custom harness and do a full rewrite, now that I have seen what 6+ other harnesses offer.
Last time I checked Pi is developed by Austrians?
We may both be partially right, the main dev I interacted with has Tel Aviv on his GitHub (also 'Hanna' being around the project, but apparently that is a surname not a first name, and a mistaken extrapolation on my part)
https://github.com/davidbrai
https://github.com/earendil-works/pi/issues/10895
The lack of professionalism or seriousness was mainly in a private email chain with Armin. He added a single comment to this issue after that chain which was the gist of a couple of his replies, less snarky. Either way, the interaction left me disappointed, not a founder who seems interested in listening or learning imo.
Security is not the harness' job, it's the sandbox's job. These are two different tools, you need both of them.
harness have a say in security too, sandboxes are one part of posture, but not the only one
You can decide for yourself: https://github.com/earendil-works/pi/issues/10895
Anthropic, OpenAi, and OpenCode all support this feature, we have non-developers to support and MDM is widely used.
For comparison, this excellent reply from Aiden on the OpenCode team: https://github.com/anomalyco/opencode/pull/54921#issuecommen...
Anthropic, OpenAi, and OpenCode all support this feature
... which is a foolish mistake.
Also, they have an economic incentive to make themselves the kitchen sink.
> which is a foolish mistake
Why do you think it is foolish?
How do we provide org / ops determined configuration that users and agents cannot change while still giving them some flexibility to set the values we do not determine?
But in 10x the time?
that's a matter of hardware, Cerebras reports 1500 tok/s
https://inference-docs.cerebras.ai/models/qwen-3.8-27b
Input $0.99 Output $1.49
That times 5 is steep!
Getting close to 80 tok/s on the dgx spark with NVFP4
https://spark-arena.com/leaderboard
(you'll have to filter model yourself, no query params)
I'm not surprised. I used Claude Code for the first time again since April because I had been hearing good things about Opus 5.5 and Fable 5.1, but I was shocked by how bad the harness is and how its only gotten worse since spring? I may be doing something wrong, but getting Opus/Fable to properly spin up non-blocking sub-agents takes like four or five attempts and it still cannot reliably poll tasks like test suites that take more than 30 seconds for proper completion.
I think even if you have the world's best model (not my opinion, but seems to be the popular consensus these days with Anthropic), if the harness around it sucks none of that intelligence is going to matter.
You are definitely doing something wrong. All of my sessions constantly use subagents and are able to poll with background tasks.
You are doing something wrong.
I am routinely using swarms of agents, with complex sequencing between them and agents watching other agents, and then triggering forks and/or new design cycles. My normal flow is to have a few design tasks open per codebase during the day, with each design agent aware of the other sessions, and having an unattended nightly build which sometimes takes 8hrs and also requires coordination for tests, merges, and so on.
Did you try once on a bad day?
I experienced a regression some days ago, where Claude Code with Opus 5.5 wouldn't progress as expected, getting stuck on steps. So I had to repeatedly ask it what the status was and to proceed with X, and so on.
But that was just one day, the days before and after it has done lots of fairly autonomous work, including spinning off multiple subagents to run conformance tests which take 30-50 mins and work on other stuff in the meantime, then acting on the results autonomously.
Not fanboying or playing favorite, but I've never come across what you mentioned while using Claude Code. You may indeed be doing something wrong or some part of your config isn't right.
I use Copilot CLI at work, and Claude Code at home, and all harnesses are pretty good these days.
Hey, Boris from the team here. This is pretty surprising, given background subagents are the default. Could you confirm you’re running the latest, and run /bug and paste the id here? Happy to debug what’s happening
Seriously hope you get that trace because as much as I like to dunk on Claude Code as a harness, it’s definitely not as bad as OP stated, ha.
First off you need to pick a harness that you can use with every model. This way you isolate model problems from harness problems. If you had to switch harnesses (Claude Code) in order to try out Fable, you're doing something wrong.
Secondly yeah, all models have issues with foreground vs background subagents. So I just modified my harness to delete foreground subagents. No more confusion. If they need the subagent's output tokens they can read it from the session transcript.
"Take away tools" is often the best solution for model-gets-confused problems. Narrow down your toolset until there's only one way to do things, and then make it the maximally-powerful way.
Just bumping this conversation because it has something to do with AI. Albeit to ask a question.
My desktop computer is at 57tuu56.intercom.rapidseedbox.com. Is it possible for me to employ AI in my process?