Our 10 skill demo teaches 10 different skills to an architecturally modified Qwen 3.6 27B, 1 example at a time (batch=1), ~100 tokens per example, for thousands of examples of 1 narrow skill at a time, one skill after another. The entire run is task label free and the model is never told when 1 skill starts and ends. In total 63,282 examples are trained incrementally. Every acquired skill is retained through all the sequential teaches and the base eval holds constant.
Continual learning in AI models is needed to improve tokens per watt efficiency in the short term, and to tackle difficult challenges where you need compounding over weeks, months or even years, something in context learning can simply not provide.
The goal of online continual learning is to train incrementally on new and narrow task distributions, without having to replay past data. When this is done in existing architectures, the model fits to the current distribution and forgets its base capabilities and previously learnt skills. This is called catastrophic forgetting and it has been an open problem in the field for ~40 years. For context, even labs doing controlled mid training use large batches with mixed data distributions to prevent forgetting during their training runs.
In our setup, data always arrives as one stream. There are no optimizer resets between task distribution boundaries and the model is never told which task or domain it is looking at, or where one ends and the next begins. After every stage we go back and measure everything taught before it again, on held-out data in addition to doing comprehensive base evals. Where we compare, we compare against LoRA with a matched number of trainable parameters (eg LoRA rank 256), on the same data in the same order. In addition, there is no replay of any kind during training. For each skill the base model scores ~0 before training. Of course longer experiments will be run to keep proving this at longer and longer scales.
In our capacity matched comparison with LoRA on four text domains, Learner 1.0 showed 82–125% of LoRA’s 300-update loss reduction in just 25 updates, measured from each condition’s recorded base reference. Earlier domains also slightly improved when later ones trained (positive backward transfer). https://learnerlabs.ai/demos/skills/four-domains
The API we are releasing is in research preview only. We are prioritizing API access to researchers, with an initial focus on understanding the safety implications of this new architecture. The overall experience is slow today due to limited GPU supply. A single user always gets a dedicated GPU for their weights for inference and training, billed for the time they are using the GPU. The cost of training 1 Million tokens of skill data today is ~$6.
I am an engineer who has spent the last decade wrestling with biology, cellular systems and how memory works in them. The above results are only possible due to new architectural and algorithmic primitives that I believe do for continual adaptation of neural nets what attention does for long sequence modeling. The core mechanism for obvious reasons is not disclosed.
Hope you enjoy reading the demonstrations and trying out the API. Appreciate any feedback.
Our 10 skill demo teaches 10 different skills to an architecturally modified Qwen 3.6 27B, 1 example at a time (batch=1), ~100 tokens per example, for thousands of examples of 1 narrow skill at a time, one skill after another. The entire run is task label free and the model is never told when 1 skill starts and ends. In total 63,282 examples are trained incrementally. Every acquired skill is retained through all the sequential teaches and the base eval holds constant.
Ten skill report: https://learnerlabs.ai/demos/skills/ten-skills
Training data, evals and every recorded answer: https://github.com/learnerlabs-ai/replications
Full context, the letter: https://learnerlabs.ai/
Continual learning in AI models is needed to improve tokens per watt efficiency in the short term, and to tackle difficult challenges where you need compounding over weeks, months or even years, something in context learning can simply not provide.
The goal of online continual learning is to train incrementally on new and narrow task distributions, without having to replay past data. When this is done in existing architectures, the model fits to the current distribution and forgets its base capabilities and previously learnt skills. This is called catastrophic forgetting and it has been an open problem in the field for ~40 years. For context, even labs doing controlled mid training use large batches with mixed data distributions to prevent forgetting during their training runs.
In our setup, data always arrives as one stream. There are no optimizer resets between task distribution boundaries and the model is never told which task or domain it is looking at, or where one ends and the next begins. After every stage we go back and measure everything taught before it again, on held-out data in addition to doing comprehensive base evals. Where we compare, we compare against LoRA with a matched number of trainable parameters (eg LoRA rank 256), on the same data in the same order. In addition, there is no replay of any kind during training. For each skill the base model scores ~0 before training. Of course longer experiments will be run to keep proving this at longer and longer scales.
In our capacity matched comparison with LoRA on four text domains, Learner 1.0 showed 82–125% of LoRA’s 300-update loss reduction in just 25 updates, measured from each condition’s recorded base reference. Earlier domains also slightly improved when later ones trained (positive backward transfer). https://learnerlabs.ai/demos/skills/four-domains
The API we are releasing is in research preview only. We are prioritizing API access to researchers, with an initial focus on understanding the safety implications of this new architecture. The overall experience is slow today due to limited GPU supply. A single user always gets a dedicated GPU for their weights for inference and training, billed for the time they are using the GPU. The cost of training 1 Million tokens of skill data today is ~$6.
I am an engineer who has spent the last decade wrestling with biology, cellular systems and how memory works in them. The above results are only possible due to new architectural and algorithmic primitives that I believe do for continual adaptation of neural nets what attention does for long sequence modeling. The core mechanism for obvious reasons is not disclosed.
Hope you enjoy reading the demonstrations and trying out the API. Appreciate any feedback.