I’m glad to find other people who actually care enough to sandbox their agents. I don’t have a kanban board, but I also built a multi-agent manager on top of gvisor (also kubernetes, but still runs on a single machine with kind). Recently switched to git reference clones instead of worktrees for better security. The transparent egress proxy + secure credential isolation was also kind of a pain to figure out. Thought I’d link here in case any of the code is useful to you, or if you want to try it out: https://yaac.dev/
I'll have to explore reference clones, I'm not familiar with it. Looks like it fundamentally provides better isolation, but not sure how of an advantage it is compared to worktrees with a secured .git. Thanks for highlighting it.
I've been looking for a basic harness that uses a kanban board for durable agent handoff.
I'd resigned myself to using openclaw's work board, but for throwaway environments, it was a bit too heavy.
I've not looked at this project in depth yet but does the harness support task push from the board (to start an LLM)?
Does it also support lifecycle events for LLM outages (mark things expired, failed, support idempotent tasks and put back a step in the queue if they reach those states etc)?
If this could support Cron based starting of tasks too, that would cover all my use cases...
The primary use case for this is actually to push work to LLMs and handle its lifecycle (including failures). Each card on the board is a different task, and pressing start on the task triggers the LLM loop. So "task push from the board" is its raison d'etre.
In working with local models, the main issues I've seen are failed tool calls, getting stuck in loops, or some other upstream issue with the inference server. Auto-retry logic helps keep things going, and tasks will sit in review for your action if they get stuck. In those situations, you can either adjust the instructions or change to a more... obedient model and put it back in the work queue.
I've experimented with a few inference services too, which tend to have other issues like how they handle rate limits. I added configurable client side limits (concurrency and reqs per minute) to make this run more smoothly.
For programmatic access: this features a RESTful Web API using a Bearer token for authorization. After generating a token (via Settings -> About), you could them have a cronjob run basic curl with a payload to start tasks. I haven't spent much time on the documentation for this yet; the current docs were generated by agents so they're a bit painful to read. I only reviewed them for accuracy.
Define the task, delegate, then review is pretty much how I work with agents now too. How does a card move to review: does the agent decide it's done, or do tests in the sandbox have to pass first?
It depends on the operating mode you choose and your prompt/AGENTS.md file.
The "Act" operating mode has strict guardrails, and a card only moves into review if the agent explicitly asks for your review (via tool call) or has a question. It's less prone to unnecessary interruptions. For example agents often decide to pause during a multi-phase plan even though no human input is needed. If an agent decides to pause in this mode, the harness automatically nudges it forward.
The "Explore" operating mode works more like most other harnesses: if the agent decides it's done then it lands in review.
Most recent models (pretty much anything released within the past year) will automatically run tests before it thinks it's done, but you can define this for example by specifying a TDD approach in AGENTS.md. In practice when a task is sitting in review, the agent typically provides a summary about test status; and this is without any special guidance.
I also implemented a "Plan" operating mode, but nowadays models are good enough at following instructions that it might be a bit overkill. This mode mounts the workspace as read-only. I originally made this out of frustration when I used to use Cline (the agent would make changes even when their plan mode was enabled). Another reason why it's overkill is because each task in MIKE already has an isolated workspace. This mode still has some advantages though, especially if you have independent subtasks that can run in parallel.
Hi, very interesting. I am curious: when an agent says it is done, who checks that the agent at least did due diligence tests, and who checks that the real end-to-end tests were really done, maybe based on a list of scenario situations created by agents to be tested?
By default, you'd be the reviewer. This project does not attempt to automate any reviews, and instead tries to make it easier for a human to review via easy access to diffs and terminal within the agent's workspace. Personally I ask the agent questions and look at the code diffs, similar to what I'd normally do in a code review.
If you wanted to be more thorough, you could scroll up in the chat and see the output of the test runs. Or to help ensure that the agent doesn't cheat, the test suite could be mounted as read-only. A custom extension could also be set up to run tests independently, and set up as a pre-review gate. However in practice (with recent models), a simple "run e2e tests" instruction in AGENTS.md would make the agent do this reliably, and I don't think I've ever seen a situation where the agent requests review without having run tests (again, with recent models). Specifying that you want to use a Test-Driven Development approach seriously goes a long way too.
I’m glad to find other people who actually care enough to sandbox their agents. I don’t have a kanban board, but I also built a multi-agent manager on top of gvisor (also kubernetes, but still runs on a single machine with kind). Recently switched to git reference clones instead of worktrees for better security. The transparent egress proxy + secure credential isolation was also kind of a pain to figure out. Thought I’d link here in case any of the code is useful to you, or if you want to try it out: https://yaac.dev/
Hah, yeah apparently there's just a few of us.
I'll have to explore reference clones, I'm not familiar with it. Looks like it fundamentally provides better isolation, but not sure how of an advantage it is compared to worktrees with a secured .git. Thanks for highlighting it.
Fun easter egg on your homepage btw!
I've been looking for a basic harness that uses a kanban board for durable agent handoff.
I'd resigned myself to using openclaw's work board, but for throwaway environments, it was a bit too heavy.
I've not looked at this project in depth yet but does the harness support task push from the board (to start an LLM)?
Does it also support lifecycle events for LLM outages (mark things expired, failed, support idempotent tasks and put back a step in the queue if they reach those states etc)?
If this could support Cron based starting of tasks too, that would cover all my use cases...
Thanks for creating this :)
The primary use case for this is actually to push work to LLMs and handle its lifecycle (including failures). Each card on the board is a different task, and pressing start on the task triggers the LLM loop. So "task push from the board" is its raison d'etre.
In working with local models, the main issues I've seen are failed tool calls, getting stuck in loops, or some other upstream issue with the inference server. Auto-retry logic helps keep things going, and tasks will sit in review for your action if they get stuck. In those situations, you can either adjust the instructions or change to a more... obedient model and put it back in the work queue.
I've experimented with a few inference services too, which tend to have other issues like how they handle rate limits. I added configurable client side limits (concurrency and reqs per minute) to make this run more smoothly.
For programmatic access: this features a RESTful Web API using a Bearer token for authorization. After generating a token (via Settings -> About), you could them have a cronjob run basic curl with a payload to start tasks. I haven't spent much time on the documentation for this yet; the current docs were generated by agents so they're a bit painful to read. I only reviewed them for accuracy.
Thanks for checking it out!
Define the task, delegate, then review is pretty much how I work with agents now too. How does a card move to review: does the agent decide it's done, or do tests in the sandbox have to pass first?
It depends on the operating mode you choose and your prompt/AGENTS.md file.
The "Act" operating mode has strict guardrails, and a card only moves into review if the agent explicitly asks for your review (via tool call) or has a question. It's less prone to unnecessary interruptions. For example agents often decide to pause during a multi-phase plan even though no human input is needed. If an agent decides to pause in this mode, the harness automatically nudges it forward.
The "Explore" operating mode works more like most other harnesses: if the agent decides it's done then it lands in review.
Most recent models (pretty much anything released within the past year) will automatically run tests before it thinks it's done, but you can define this for example by specifying a TDD approach in AGENTS.md. In practice when a task is sitting in review, the agent typically provides a summary about test status; and this is without any special guidance.
I also implemented a "Plan" operating mode, but nowadays models are good enough at following instructions that it might be a bit overkill. This mode mounts the workspace as read-only. I originally made this out of frustration when I used to use Cline (the agent would make changes even when their plan mode was enabled). Another reason why it's overkill is because each task in MIKE already has an isolated workspace. This mode still has some advantages though, especially if you have independent subtasks that can run in parallel.
Hi, very interesting. I am curious: when an agent says it is done, who checks that the agent at least did due diligence tests, and who checks that the real end-to-end tests were really done, maybe based on a list of scenario situations created by agents to be tested?
By default, you'd be the reviewer. This project does not attempt to automate any reviews, and instead tries to make it easier for a human to review via easy access to diffs and terminal within the agent's workspace. Personally I ask the agent questions and look at the code diffs, similar to what I'd normally do in a code review.
If you wanted to be more thorough, you could scroll up in the chat and see the output of the test runs. Or to help ensure that the agent doesn't cheat, the test suite could be mounted as read-only. A custom extension could also be set up to run tests independently, and set up as a pre-review gate. However in practice (with recent models), a simple "run e2e tests" instruction in AGENTS.md would make the agent do this reliably, and I don't think I've ever seen a situation where the agent requests review without having run tests (again, with recent models). Specifying that you want to use a Test-Driven Development approach seriously goes a long way too.
Thank you, this is clear and helpful.
Thanks for using gVisor! Love to see it used in the wild for hobby projects.
And thank you and the team for making it easier to secure services!
I read MAGI right around the time I was looking for existing solutions earlier this year, also enjoyed the Eva vibes.