More and more such experiments. I felt sad for a couple months when I realized that writing code will not be the same since. Now I am on the other side.
LLMs are interesting in their own ways but as an engineer, this is a way to unlock a new way of building software.
I recently build a Claude-assisted Excel/CSV parser for a US based property management system (tax compliance). Uses Haiku and has a lot of deterministic code to extract column/row combinations to check known formats and finally handing out the headers to Haiku to give us a translation plan to our support columns.
These would eventually become part of the software, in a tiny LLM. The gap between training (such tiny LLMs) and inference will shrink. We can consult Claude for edge cases, create sample dataset and train a the tiny LLM on demand so we go to Claude less.
The tooling that a project needs is really important. Something I have been feeling as well. Not just in LLM building projects, but regular software projects that are LLM generated.
I think I am on your arc as well. My learning on different topics is growing every day, but there’s a limit to how much I can absorb. With the LLMs the experiments stay just beyond that horizon and I keep chasing.
Stated too strongly, but I think this could be the model for education (some subjects anyway). Everything personalized to your learning goals, grounded in experiments that give a tight feedback loop and with a model that never gets tired of re-explaining something for the 10th time.
Yes they are great for learning at own pace, trying out new things.
I have accepted two things that make me a happy engineer now: AGI is not here no matter what they say and LLMs are still very useful if one knows how to use them.
They are another layer of abstraction and like you said they do not tire. There is a lot of optimization needed so we can reduce wastage (running 1T+ LLMs for most work is wastage).
Haha, I thought about this today. I'm traveling and walked through a university campus. The new students are coming in and I thought wow, wouldn't it be wild to be a fresh in student in the age of AI (it's been a long time since I was college). But, then I thought, actually this could be an incredible time to be a student. If your curious and motivated the agents can be amazing partners. I think the students who crack how to use AI now are going to be the ones who really run the table as this economic wave crests. But, yeah, you're right on with the Wall-E comment. I would guess many (most?) will turn their brain off and then what's the point of even being there?
This is really neat! I'd be tempted to try this again targetting ~1B params and the entire cookbook of "current" small model ideas: gated delta nets (or is ~2K context too short to benefit?), per layer or n-gram embeddings, gated residuals etc.
I hope we will soon have an open-source project for training such small LLMs where one can just pick the architecture (like Qwen / DeepSeek / etc.), parameter count, dataset, ... and then let it run on a local/rented GPUs...
I’m with you. But what are we going to get? I think this goal sits in a funny place between knowledge and convenience.
On the one hand, tons of “products” promising this.
On the other hand, I’m sure we can find student works—sharing the code they created from a course or book. And I expect there will be gaps, niches filled by pro-coders who see a pro need and fill it.
What I don’t want to do is see the market for model training filled with whatever Microsoft thinks will make money.
I've been training different architecture 1.33b and 3.33b models using nvidia megatron for about a month. You can train a 1.33b on a 46gb mac and 3.33b is about as big as you can go with a single 96gb blackwell with regular checkpoints etc
I don’t blame you for using an LLM to write an article about an LLM that you built. I’ve been reading so much LLM output that now I see it everywhere. I wonder if humans will start writing more like LLMs?
I realized the same while reading it, but my impression is that the output has been well edited. The typical Claude density has been pulled out into something more human readable, but with many of the Claud-isms still intact.
Personally I find this a productive way to produce documents. Claude makes the first pass and I edit line by line. It’s a good understanding check. If I can’t reword something into my own voice it means I don’t understand it. Then have Claude loop again to fact check the rewrites and repeat until clear (or I disagree with Claude on all it’s remaining points).
That’s the same edit loop I used without LLMs, but I get speed in drafting up front (and fact/number pulling) and the check passes make the edits better.
When we hyper focus on finding something, we find it all the time.
The world of tiny LLMs is so interesting. It is unlocking novel ways to encode information. Why focus on the style of writing instead of the subject matter?
Anyone can level a charge of AI writing, but it's getting to the point that you must write sloppy to seem human. and never never write well with clear technical prose. It is getting to be just ridiculous McCarthyism.
Pangram says 50% human. That seems to check out, heavily edited yet the LLM aha sentences and cadence are still definitely there, but not overbearing.
I read some Dostoevsky recently and I found my speech to be substantially altered for a while, so it makes sense that LLM writing would have an influence.
More and more such experiments. I felt sad for a couple months when I realized that writing code will not be the same since. Now I am on the other side.
LLMs are interesting in their own ways but as an engineer, this is a way to unlock a new way of building software.
I recently build a Claude-assisted Excel/CSV parser for a US based property management system (tax compliance). Uses Haiku and has a lot of deterministic code to extract column/row combinations to check known formats and finally handing out the headers to Haiku to give us a translation plan to our support columns.
These would eventually become part of the software, in a tiny LLM. The gap between training (such tiny LLMs) and inference will shrink. We can consult Claude for edge cases, create sample dataset and train a the tiny LLM on demand so we go to Claude less.
The tooling that a project needs is really important. Something I have been feeling as well. Not just in LLM building projects, but regular software projects that are LLM generated.
I think I am on your arc as well. My learning on different topics is growing every day, but there’s a limit to how much I can absorb. With the LLMs the experiments stay just beyond that horizon and I keep chasing.
Stated too strongly, but I think this could be the model for education (some subjects anyway). Everything personalized to your learning goals, grounded in experiments that give a tight feedback loop and with a model that never gets tired of re-explaining something for the 10th time.
Yes they are great for learning at own pace, trying out new things.
I have accepted two things that make me a happy engineer now: AGI is not here no matter what they say and LLMs are still very useful if one knows how to use them.
They are another layer of abstraction and like you said they do not tire. There is a lot of optimization needed so we can reduce wastage (running 1T+ LLMs for most work is wastage).
Guys, you are the 1%; 99% of LLM will be Wall-E.
Haha, I thought about this today. I'm traveling and walked through a university campus. The new students are coming in and I thought wow, wouldn't it be wild to be a fresh in student in the age of AI (it's been a long time since I was college). But, then I thought, actually this could be an incredible time to be a student. If your curious and motivated the agents can be amazing partners. I think the students who crack how to use AI now are going to be the ones who really run the table as this economic wave crests. But, yeah, you're right on with the Wall-E comment. I would guess many (most?) will turn their brain off and then what's the point of even being there?
This is really neat! I'd be tempted to try this again targetting ~1B params and the entire cookbook of "current" small model ideas: gated delta nets (or is ~2K context too short to benefit?), per layer or n-gram embeddings, gated residuals etc.
I hope we will soon have an open-source project for training such small LLMs where one can just pick the architecture (like Qwen / DeepSeek / etc.), parameter count, dataset, ... and then let it run on a local/rented GPUs...
> “open-source project”
I’m with you. But what are we going to get? I think this goal sits in a funny place between knowledge and convenience.
On the one hand, tons of “products” promising this.
On the other hand, I’m sure we can find student works—sharing the code they created from a course or book. And I expect there will be gaps, niches filled by pro-coders who see a pro need and fill it.
What I don’t want to do is see the market for model training filled with whatever Microsoft thinks will make money.
I've been training different architecture 1.33b and 3.33b models using nvidia megatron for about a month. You can train a 1.33b on a 46gb mac and 3.33b is about as big as you can go with a single 96gb blackwell with regular checkpoints etc
I don’t blame you for using an LLM to write an article about an LLM that you built. I’ve been reading so much LLM output that now I see it everywhere. I wonder if humans will start writing more like LLMs?
I don't see it. I sit in a mild state of rage trying to decode llm prose all day, this isn't it.
I realized the same while reading it, but my impression is that the output has been well edited. The typical Claude density has been pulled out into something more human readable, but with many of the Claud-isms still intact.
Personally I find this a productive way to produce documents. Claude makes the first pass and I edit line by line. It’s a good understanding check. If I can’t reword something into my own voice it means I don’t understand it. Then have Claude loop again to fact check the rewrites and repeat until clear (or I disagree with Claude on all it’s remaining points).
That’s the same edit loop I used without LLMs, but I get speed in drafting up front (and fact/number pulling) and the check passes make the edits better.
When we hyper focus on finding something, we find it all the time.
The world of tiny LLMs is so interesting. It is unlocking novel ways to encode information. Why focus on the style of writing instead of the subject matter?
> Why focus on the style of writing instead of the subject matter?
Because just as with human writing, a poor writing style obscures the topic where it should illuminate
[dead]
Anyone can level a charge of AI writing, but it's getting to the point that you must write sloppy to seem human. and never never write well with clear technical prose. It is getting to be just ridiculous McCarthyism.
Pangram says 50% human. That seems to check out, heavily edited yet the LLM aha sentences and cadence are still definitely there, but not overbearing.
I read some Dostoevsky recently and I found my speech to be substantially altered for a while, so it makes sense that LLM writing would have an influence.
Very much Fable 5.1 assisted. I noticed Fable 5.1 recently started outputting graphs like that used when doing LLM work.
Nice work. But I cannot find a link to the actual code of the model or the training data. Did I miss it somewhere?
Would be very interested how much it would cost if you distill from Qwen's best ~30B model.
Very cool. I wonder what's the minimum amount of money you'd need to get it to score well on AIME
the evenings plus rented gpus path is inspiring. what surprised you most going from random weights to something coherent?
[flagged]
[dead]
[flagged]
[flagged]