The hard part with agent skill is that "read this file and run these tools" basically makes your repo content part of what the agent has to trust. Static inspection is useful, but I wonder how much malicious behaviour can realistically be detected before runtime?
Looking to what's going on in our company I see that majority of employees doesn't understand efficient usage and how better to use models. Even programmers sometimes do crazy stupid things. I would think twice before allowing users to decide something that may directly influence spends
We recently rolled out a dashboard so that you can see your token usage. Looking at my usage vs others on my team, I just cannot imagine what they're doing. I use Claude for literally everything and my usage is 10x smaller. I genuinely do not know how they can use that many more tokens. There's gotta be some kind of curve that shows token vs effectiveness and it is not linear.
i built nexalione.com cause i was tired paying other website for things i could get for free but the most thing that reduced my cost is by hosting it alone with cloudflare it reduces cot of hosting by almost 100% and helps you run a smooth website
Not enterprise, but one thing that transferred from running a lot of agents on a small budget: the cost that got away from us was never the big deliberate runs, it was the default behaviours nobody had looked at. Two we measured and turned off:
1. Re-reading whole files after every edit "to verify". The edit result already tells you it applied. Forbidding the re-read unless a test fails cut a noticeable slice of tokens on long sessions with zero quality change.
2. Guessing loops. An agent that gets a fix wrong twice will happily try a third and fourth variant. We put a rule in the harness: after the second failed attempt it has to add instrumentation and report what it observed before it is allowed to change code again. That turned several hour-long loops into ten-minute fixes, and the token savings were incidental to the time savings.
On limits: a hard monthly cap per person mostly moved the spend to the last week of the month. What worked better for us was a visible remaining-budget gauge in the tool people actually work in, so the number is in front of them while they decide whether to kick off another run. People self-regulate surprisingly well when the meter is on the dashboard rather than in a monthly report.
If you do tier by role, I would tier by "how expensive is a wrong answer" rather than tech vs non-tech. A non-technical person running one careful summarisation a day is cheap; a developer with an agent in a retry loop is where the money goes.
The hard part with agent skill is that "read this file and run these tools" basically makes your repo content part of what the agent has to trust. Static inspection is useful, but I wonder how much malicious behaviour can realistically be detected before runtime?
Looking to what's going on in our company I see that majority of employees doesn't understand efficient usage and how better to use models. Even programmers sometimes do crazy stupid things. I would think twice before allowing users to decide something that may directly influence spends
We recently rolled out a dashboard so that you can see your token usage. Looking at my usage vs others on my team, I just cannot imagine what they're doing. I use Claude for literally everything and my usage is 10x smaller. I genuinely do not know how they can use that many more tokens. There's gotta be some kind of curve that shows token vs effectiveness and it is not linear.
i built nexalione.com cause i was tired paying other website for things i could get for free but the most thing that reduced my cost is by hosting it alone with cloudflare it reduces cot of hosting by almost 100% and helps you run a smooth website
Praying that productivity explodes as well.
Not enterprise, but one thing that transferred from running a lot of agents on a small budget: the cost that got away from us was never the big deliberate runs, it was the default behaviours nobody had looked at. Two we measured and turned off:
1. Re-reading whole files after every edit "to verify". The edit result already tells you it applied. Forbidding the re-read unless a test fails cut a noticeable slice of tokens on long sessions with zero quality change.
2. Guessing loops. An agent that gets a fix wrong twice will happily try a third and fourth variant. We put a rule in the harness: after the second failed attempt it has to add instrumentation and report what it observed before it is allowed to change code again. That turned several hour-long loops into ten-minute fixes, and the token savings were incidental to the time savings.
On limits: a hard monthly cap per person mostly moved the spend to the last week of the month. What worked better for us was a visible remaining-budget gauge in the tool people actually work in, so the number is in front of them while they decide whether to kick off another run. People self-regulate surprisingly well when the meter is on the dashboard rather than in a monthly report.
If you do tier by role, I would tier by "how expensive is a wrong answer" rather than tech vs non-tech. A non-technical person running one careful summarisation a day is cheap; a developer with an agent in a retry loop is where the money goes.
Thanks, great ideas to try out here!