1. How a this better than just using any other script language?
2. Inventing a new language complicates large models ability to generate such scrips compared to a well-known language. Did you find it to be a problem? How did you mitigate?
3. The AI is showing. :) I had the similar discussion with ChapGPT and some phrasing is near the same. Not a dig, just a funny observation.
4. Consider the recursive nature of the problem you’re solving - large model updates workflow which you review each time, worker models generate plans and tool calls which you don’t review. A constrained language is useful in both cases to guide the model.
5. This Earlier discussion can provide useful background for why this is needed. You have probably seen it, but the readers will likely appreciate. https://news.ycombinator.com/item?id=48051562
The problem is real, Thank you for taking a stab and sharing your findings.
I really appreciate the comments and the encouragement.
"How <is> this better than just using any other script language?"
I wanted to make the language something built for a machine to write and a human to approve with a narrow scope and my control of what extends that. What shell commands, if any. What MCP and what tools in that MCP, and so on. If an agent generates a Python script to run unattended every morning, I feel I can't control it.
"Inventing a new language complicates large models ability to generate such scripts..."
I tried to keep it as small as possible and borrow a lot from what I knew models had seen in makefiles, YAML-ish, etc.
That was actually the question I began with. What would a make file look like if I were using it to make a classic markdown skill? Asking that question, sort of spawned the rest of it.
I tested the language against cold agents in both frontier model and local model camps to see where they had difficulty nad made changes as required. The lint approach as well as help topics from the MCP helped a lot.
"AI Showing..."
Yea there is a certain irony there. I freely admit product was written with Claude Code. I wrote the Product Requirements & Engineering Requirements and reviewed the code, so I can say I own it.
"Consider the recursive nature of the problem you’re solving..."
That framing is better than mine. When a frontier model writes a skillscript, there's not much chance of something slipping by. Over time, I've become cautiously open to the model adapting the skillscript to solve problems as they occur. Example, a skillscript runs every morning and checks github for PRs and issues. The agent when woken after the run 'noticed' an issue, fixed the script and it was ready for me to approve in the morning. Yes, that could have been done in Python, but I'd not be as confident about it.
It’s not clear from you answer if we’re on the same page. Your options are allow the worker agent run wild or constrain it with a rigid guidance. Skillscript can serve dual purpose - constrain the workflow to make it more observable and constrain elemental parts to make them more directional. Apologies if that makes no sense - the AI red shift (expanding universe) makes it very hard to communicate.
It makes sense. I think my response skewed things a bit. The worker can not run wild for exactly the reasons you state. Constraints of the language and the workflow are easy for a human to understand (especially a non-programmer human).
I expect, in general, agents to be writing automation for non-programmers to approve which is another case for why not bash or python.
I'm biased since I built a dbt[1] inspired utility[2] that turns markdown into a runnable DAG, but I think a new language is the wrong abstraction.
In a time where people are reading less and less code, introducing a new surface area -- that you don't have a good feel for -- to handle orchestration feels risky.
Honestly, not far from where I started. The origin of this was "what would a Makefile look like if it built markdown skills?" You'll notice the syntax is very much makefile like.
"new language is the wrong abstraction": I'd resist using "language" as it carries way too much weight here. Closer to your DAG than to a general purpose language.
I guess the way I look at it, if people are reading less and less code, what an agent hands me needs to be small and skimmable. The web tool helps by highlighting ops, and I even built a code to mermaid function (which honestly is there but unused right now) because the skillscripts are generally straightforward.
Skim the examples, I hope you'll see its not that different.
So the real benefit is succinctness and readability... would be nice to have a comparison against other scripting languages or plain text, to prove better performance or accuracy.
I would say you're right in that with succinctness comes constraint. And the constraint is really the point, not the brevity. It's small because it can't do arbitrary things, which is exactly what lets you read one and approve it (and for hopefully anyone to understand), and what makes it run the same way every time.
A comparison would be useful. No published benchmark yet, but that's fair to ask for, and probably the next thing worth putting out.
It seems like you’re dramatically overcomplicating what could be a 2 bash scripts and a single LLM call. Natural language and LLMs are great for searching the problem space to find a solution, once you find the solution, shrink the stochastic parts (the MD saying “check GitHub” or whatever) and grow the deterministic parts (a bash script) as much as possible.
That's a fair comment. I sorta feel you're describing my thesis, "shrink the stochastic, grow the deterministic" is the point. Where we differ, maybe, is what the deterministic part is made of.
The problem I'm trying to solve for me is a step removed. The agent is the one writing the script, and I want it to run unattended against my systems every morning. I can't let it write and run bash. Even though my agent lives in a container (NanoClaw), I still need it to reach out to other systems. Github, my other servers, MCPs, etc.
Skillscript is how I let it reach out without holding the keys. It can invoke a skillscript that hits GitHub, but it never runs the raw command or holds the token itself. The runtime holds the credential and only lets it through the skillscript I approved.
It's fair to say a skillscript basically is your two bash scripts and an LLM call, but fenced in.
> The runtime holds the credential and only lets it through the skillscript I approved.
What about skillscript is unique that couldn't be done with bash or python as a permissioned tool? (Trying to understand where you see the difference.)
---
My original impression from the repo was that the language/toolkit is overengineered, but then I saw on the website that the intention is for the agents to write their own tools. That helps explain some of the complexity.
I think the rest of the perceived complexity is the over-explaining in the README.
I don't think anyone's going to really engage with all of that so you might have better luck chopping it down 80% to only highlight the stuff that matters.
What's the difference? if you're the one writing the Python and approving each script, probably little. The difference shows up when the agent writes it unattended, over and over. Skillscript can only reach the binaries and tools I allowlisted, where I allowlisted them. An agent can't escalate by writing something clever that get by me.
I think you're right on. I let the readme get out of hand. It became a README, changelog mash-up. Going to rewrite it, and the 80% metric is a good one. Is there an example of what you consider the perfect readme?
Re: difference -- Ok, I think I understand. Ultimately, we need a "permissions" layer and you've built a solve for that.
Re: README -- I can't recall a specific repo with one off the top of my head so took a stab at editing yours[1] instead of hunting around. It's not perfect -- I'd want to trim the bullet lists further, for example -- but is much more scannable in my opinion.
Oh wow. Talking about going above and beyond! Thanks for this. I'll use it as a template for sure. You might have guessed, first time building in public per se. Everything else I do is pretty much for clients and C/C++ work.
It's hilarious how emotional and upset sieabahlpark is about this all, complaining about ragebaiting, yet every single one of his posts are sputtering incoherent automatically flagged dead temper tantrums of self loathing rage -- just read his sad posting history and self pitying profile.
>Fuck of Don, you don’t know jack shit about anything these days other than to ragebait to try to remain relevant.
ieabahlpark: I'm truly sorry you're having such a hard time coping with me asking you to simply not to marry someone of your same gender, instead of incessently whining, lashing out, and expressing bigotry.
If it's really that difficult and upsetting for you not to, and you're so emotionally triggered and distraught at the mere mention of human rights for other people, then you really need to look deep inside yourself and reflect, figure out why you're so terrified of other gays getting married while none of them want to marry you, and get some help.
The answer is obvious to all of us except for you, so you've got to figure it out all by yourself. Don't die closeted, angry, self loathing, and alone like Lindsey Graham. Neither of you ever fooled anyone but yourselves.
Nobody will love you until you love yourself. I understand how irrelevant that makes you feel, but no need to lash out and project your butthurt feelings onto me. Have a gay and fabulous day, and I hope you get laid.
Generating but not programming in them. They need to be reminded of the complete language definition they generated in every prompt, which is extremely costly, inefficient, and ultimately pointless, since any language you make up can't hold a candle to Python and its ecosystem, because it doesn't have an ecosystem, and the language itself doesn't exist in the training data.
How can you not get that? Do you believe LLMs remember what you show them between calls? That's not how they work. Each call starts from a clean slate, you have to re-describe the new language each and every call. There's no way to get around that. They are not magic. They do not learn from your prompts, which have absolutely no effect on the model itself.
If you think they do, you are falling for an illusion. ChatGPT is appending each of your incremental prompts to the full prompt, and it grows and grows longer and longer every time you add something. Sure, it summarizes when the full prompt gets to long, but that makes it distort and forget your language definition, and you have to add it again. If you give it the prompt to generate the language from scratch each time instead of the generated language itself, it generates a different language every time. You can't "cleverly hack" or "wish" your way out of that.
They may be good at generating new languages, but one thing that LLMs aren't good at apparently is warning you it's futile to generate a new language intended for llms to program instead of just using existing languages. They just play along and do ridiculous useless things out of syncophancy.
That should be pretty obvious. How about new programming languages for human beings to use? No reason not to make more of those. But this discussion is about a programming language designed for LLMs to use, not humans. The world doesn't need any more of those.
Design programming languages for humans first. If they're good enough to catch on, and people write lots of code in them, ask and answer lots of questions about them, write tutorial and have hacker news discussions about them, then they will naturally and eventually end up in the training data, and the models will know about them. Problem solved.
It's ridiculous and costly to design a language for LLMs but not humans to use, and then necessarily and repeatedly insert the entire language definition and examples into every single prompt, instead of building it into the model. It's a tragic waste of electricity and money, and has a huge carbon footprint. Why isn't this obvious?
1. How a this better than just using any other script language?
2. Inventing a new language complicates large models ability to generate such scrips compared to a well-known language. Did you find it to be a problem? How did you mitigate?
3. The AI is showing. :) I had the similar discussion with ChapGPT and some phrasing is near the same. Not a dig, just a funny observation.
4. Consider the recursive nature of the problem you’re solving - large model updates workflow which you review each time, worker models generate plans and tool calls which you don’t review. A constrained language is useful in both cases to guide the model.
5. This Earlier discussion can provide useful background for why this is needed. You have probably seen it, but the readers will likely appreciate. https://news.ycombinator.com/item?id=48051562
The problem is real, Thank you for taking a stab and sharing your findings.
I really appreciate the comments and the encouragement.
"How <is> this better than just using any other script language?"
I wanted to make the language something built for a machine to write and a human to approve with a narrow scope and my control of what extends that. What shell commands, if any. What MCP and what tools in that MCP, and so on. If an agent generates a Python script to run unattended every morning, I feel I can't control it.
"Inventing a new language complicates large models ability to generate such scripts..."
I tried to keep it as small as possible and borrow a lot from what I knew models had seen in makefiles, YAML-ish, etc.
That was actually the question I began with. What would a make file look like if I were using it to make a classic markdown skill? Asking that question, sort of spawned the rest of it.
I tested the language against cold agents in both frontier model and local model camps to see where they had difficulty nad made changes as required. The lint approach as well as help topics from the MCP helped a lot.
"AI Showing..."
Yea there is a certain irony there. I freely admit product was written with Claude Code. I wrote the Product Requirements & Engineering Requirements and reviewed the code, so I can say I own it.
"Consider the recursive nature of the problem you’re solving..."
That framing is better than mine. When a frontier model writes a skillscript, there's not much chance of something slipping by. Over time, I've become cautiously open to the model adapting the skillscript to solve problems as they occur. Example, a skillscript runs every morning and checks github for PRs and issues. The agent when woken after the run 'noticed' an issue, fixed the script and it was ready for me to approve in the morning. Yes, that could have been done in Python, but I'd not be as confident about it.
I also really appreciate the link.
> I tested the language against cold agents
I was procrastinating on that, thanks!
> the recursive nature
It’s not clear from you answer if we’re on the same page. Your options are allow the worker agent run wild or constrain it with a rigid guidance. Skillscript can serve dual purpose - constrain the workflow to make it more observable and constrain elemental parts to make them more directional. Apologies if that makes no sense - the AI red shift (expanding universe) makes it very hard to communicate.
It makes sense. I think my response skewed things a bit. The worker can not run wild for exactly the reasons you state. Constraints of the language and the workflow are easy for a human to understand (especially a non-programmer human).
I expect, in general, agents to be writing automation for non-programmers to approve which is another case for why not bash or python.
I'm biased since I built a dbt[1] inspired utility[2] that turns markdown into a runnable DAG, but I think a new language is the wrong abstraction.
In a time where people are reading less and less code, introducing a new surface area -- that you don't have a good feel for -- to handle orchestration feels risky.
---
[1] https://github.com/dbt-labs/dbt-core
[2] https://github.com/pdthq/pdt
Honestly, not far from where I started. The origin of this was "what would a Makefile look like if it built markdown skills?" You'll notice the syntax is very much makefile like.
"new language is the wrong abstraction": I'd resist using "language" as it carries way too much weight here. Closer to your DAG than to a general purpose language.
I guess the way I look at it, if people are reading less and less code, what an agent hands me needs to be small and skimmable. The web tool helps by highlighting ops, and I even built a code to mermaid function (which honestly is there but unused right now) because the skillscripts are generally straightforward.
Skim the examples, I hope you'll see its not that different.
So the real benefit is succinctness and readability... would be nice to have a comparison against other scripting languages or plain text, to prove better performance or accuracy.
I would say you're right in that with succinctness comes constraint. And the constraint is really the point, not the brevity. It's small because it can't do arbitrary things, which is exactly what lets you read one and approve it (and for hopefully anyone to understand), and what makes it run the same way every time.
A comparison would be useful. No published benchmark yet, but that's fair to ask for, and probably the next thing worth putting out.
It seems like you’re dramatically overcomplicating what could be a 2 bash scripts and a single LLM call. Natural language and LLMs are great for searching the problem space to find a solution, once you find the solution, shrink the stochastic parts (the MD saying “check GitHub” or whatever) and grow the deterministic parts (a bash script) as much as possible.
That's a fair comment. I sorta feel you're describing my thesis, "shrink the stochastic, grow the deterministic" is the point. Where we differ, maybe, is what the deterministic part is made of.
The problem I'm trying to solve for me is a step removed. The agent is the one writing the script, and I want it to run unattended against my systems every morning. I can't let it write and run bash. Even though my agent lives in a container (NanoClaw), I still need it to reach out to other systems. Github, my other servers, MCPs, etc.
Skillscript is how I let it reach out without holding the keys. It can invoke a skillscript that hits GitHub, but it never runs the raw command or holds the token itself. The runtime holds the credential and only lets it through the skillscript I approved.
It's fair to say a skillscript basically is your two bash scripts and an LLM call, but fenced in.
> The runtime holds the credential and only lets it through the skillscript I approved.
What about skillscript is unique that couldn't be done with bash or python as a permissioned tool? (Trying to understand where you see the difference.)
---
My original impression from the repo was that the language/toolkit is overengineered, but then I saw on the website that the intention is for the agents to write their own tools. That helps explain some of the complexity.
I think the rest of the perceived complexity is the over-explaining in the README.
I don't think anyone's going to really engage with all of that so you might have better luck chopping it down 80% to only highlight the stuff that matters.
What's the difference? if you're the one writing the Python and approving each script, probably little. The difference shows up when the agent writes it unattended, over and over. Skillscript can only reach the binaries and tools I allowlisted, where I allowlisted them. An agent can't escalate by writing something clever that get by me.
I think you're right on. I let the readme get out of hand. It became a README, changelog mash-up. Going to rewrite it, and the 80% metric is a good one. Is there an example of what you consider the perfect readme?
Re: difference -- Ok, I think I understand. Ultimately, we need a "permissions" layer and you've built a solve for that.
Re: README -- I can't recall a specific repo with one off the top of my head so took a stab at editing yours[1] instead of hunting around. It's not perfect -- I'd want to trim the bullet lists further, for example -- but is much more scannable in my opinion.
---
[1] https://gist.github.com/thedatadavis/fbbe556348eb43731659456...
Oh wow. Talking about going above and beyond! Thanks for this. I'll use it as a template for sure. You might have guessed, first time building in public per se. Everything else I do is pretty much for clients and C/C++ work.
No prob
It's very nice.. One thing I'd say is the README is a tad long..
yea, I let it get out of control. A new leaner readme is on its way thanks in big part to christoff12.
In Brave, on stock Google Pixel 10, guard enabled, docs site layout is broken.
Thanks. I'm guessing since the docs are mintlify hosted, the CDN assets might be being blocked. I'll look and see if adding a domain sidesteps it.
same, Zen Browser (Firefox fork) on Linux
Is what you are seeing is the topics sidebar overlapping the text?
dark theme went to shits. white sidebar with white text, occasionally white-on-white spots in the docs.
[flagged]
Brave is Chromium, presumably this means the site is broken on mobile chrome
It's hilarious how emotional and upset sieabahlpark is about this all, complaining about ragebaiting, yet every single one of his posts are sputtering incoherent automatically flagged dead temper tantrums of self loathing rage -- just read his sad posting history and self pitying profile.
https://news.ycombinator.com/user?id=sieabahlpark
https://news.ycombinator.com/item?id=48882882
>sieabahlpark 13 hours ago [dead] | prev [–]
>Fuck of Don, you don’t know jack shit about anything these days other than to ragebait to try to remain relevant.
ieabahlpark: I'm truly sorry you're having such a hard time coping with me asking you to simply not to marry someone of your same gender, instead of incessently whining, lashing out, and expressing bigotry.
If it's really that difficult and upsetting for you not to, and you're so emotionally triggered and distraught at the mere mention of human rights for other people, then you really need to look deep inside yourself and reflect, figure out why you're so terrified of other gays getting married while none of them want to marry you, and get some help.
The answer is obvious to all of us except for you, so you've got to figure it out all by yourself. Don't die closeted, angry, self loathing, and alone like Lindsey Graham. Neither of you ever fooled anyone but yourselves.
Nobody will love you until you love yourself. I understand how irrelevant that makes you feel, but no need to lash out and project your butthurt feelings onto me. Have a gay and fabulous day, and I hope you get laid.
[dead]
[flagged]
[flagged]
LLMs are fantastic at generating new languages given docs and examples.
Generating but not programming in them. They need to be reminded of the complete language definition they generated in every prompt, which is extremely costly, inefficient, and ultimately pointless, since any language you make up can't hold a candle to Python and its ecosystem, because it doesn't have an ecosystem, and the language itself doesn't exist in the training data.
How can you not get that? Do you believe LLMs remember what you show them between calls? That's not how they work. Each call starts from a clean slate, you have to re-describe the new language each and every call. There's no way to get around that. They are not magic. They do not learn from your prompts, which have absolutely no effect on the model itself.
If you think they do, you are falling for an illusion. ChatGPT is appending each of your incremental prompts to the full prompt, and it grows and grows longer and longer every time you add something. Sure, it summarizes when the full prompt gets to long, but that makes it distort and forget your language definition, and you have to add it again. If you give it the prompt to generate the language from scratch each time instead of the generated language itself, it generates a different language every time. You can't "cleverly hack" or "wish" your way out of that.
They may be good at generating new languages, but one thing that LLMs aren't good at apparently is warning you it's futile to generate a new language intended for llms to program instead of just using existing languages. They just play along and do ridiculous useless things out of syncophancy.
Not sure I understood well your comment.
Do you propose just ask AI to generate orchestration in Python?
[flagged]
So no new languages ever?
That's not what I said at all.
Do you want to describe what the exception is to what you said then?
That should be pretty obvious. How about new programming languages for human beings to use? No reason not to make more of those. But this discussion is about a programming language designed for LLMs to use, not humans. The world doesn't need any more of those.
Design programming languages for humans first. If they're good enough to catch on, and people write lots of code in them, ask and answer lots of questions about them, write tutorial and have hacker news discussions about them, then they will naturally and eventually end up in the training data, and the models will know about them. Problem solved.
It's ridiculous and costly to design a language for LLMs but not humans to use, and then necessarily and repeatedly insert the entire language definition and examples into every single prompt, instead of building it into the model. It's a tragic waste of electricity and money, and has a huge carbon footprint. Why isn't this obvious?