> We present Ultramarine 1 Flash, an agentic model for professional knowledge work. Ultramarine applies large-scale reinforcement learning to a strong open base model, GLM-5.3 Flash, and improves on it across all six knowledge-work benchmarks we evaluate [...]
In case anyone was curious what it's based on. (No judgement either way from me, IMO depends how well they did the post-training, which I can't tell).
Can I ask what kind of tooling you use around the model and for the UI layer? I’ve been looking for a well designed knowledge work harness to give nontechnical users but I haven’t really found one that I like. (I personally just use coding harness for this stuff, but that’s a poor fit for non-devs.)
I started using it last week when copilot started acting up against the Deepseek models. It's fast, solid, I love the architecture but be warned: it submits very very many input tokens against the LLM endpoint, so robust & cheap token caching is a must.
Hello, apologies for the self-promotion but I’m building a tool called Radix which is a knowledge work harness. Please email or message me if you would like to give it a try for free instead of paying the subscription price.
I can’t speak for your tool, but I wanted to appreciate your privacy page
https://radix-os.com/privacy
It points out concrete details (e.g. “where on my Mac does your folder live?”), it is comprehensive and understandable for a layperson. Kudos!
> depends how well they did the post-training, which I can't tell
this is their problem, not yours. at least disclose # of flops spent or amount of data used in an attempt to give us an idea of how nontrivial it was. cursor at least does this with their composer model
As an aside, I do not like unnecessarily misleading graphs. Own your strengths and weaknesses.
In the "Results" section, they choose to compare Ultramarine only to models that it's better than on every metric. They ignore models like Fable 5.1, GPT-6 Astra, Claude Opus 5.5, etc.
Then, in the "Score against cost per task" graph, which is focused on price, suddenly all of those models that perform better than Ultramarine come roaring back because they are also more expensive.
Just list yourselves as the second best sometimes. Your thing is impressive. Don't let the idiot marketing guy hide that behind stupid tricks.
Yes, for me when they do this I immediately dismiss the model, don't even bother testing it. If you need to lie with statistics your model isn't good enough.
Maybe you are more likely to buy things when benchmarks are honest, but openai and anthropic have been cheating at those for years. They get purchased a lot. Most purchases are emotional things, even for technical products. That's why they sell llms like it's crack, a weapon, or if you don't you will starve to death in the future. One thing I like about the Chinese and euro labs is they don't market their products in harmful ways, but that's a tangent.
It's more or less all stupid anyways. Benchmarks can give someone and understanding of what a model was trained on but they don't explain tone, consistency, the type of answers, the restrictions, what the money is going to, how good it is on your task, etc.
Most use of llms is gambling. So a lot of people get hooked on their one machine or whatever despite all reasons to the contrary. Some of the most deranged LLM users I e encountered are hooked on a variety of models but still use ones from like 2 years ago for specific tasks for emotional reasons.
The truth is, for 99.999% of use cases the top 10 models are more or less interchangeable. Either they can mostly do what someone wants, or none of them should probably be used for that application even if s leading lab is advertising it.
I really would like to understand how it can be 60 times cheaper. Is that a like for like comparison? Is there a more equivalent frontier model that is closer in price?
I think they just decided to use the formatting different than the others.
Normally, the background highlight color indicates the leader for every benchmark, but they chose to highlight their column even though there are better models.
> I think they just decided to use the formatting different than the others.
500px of the landing page is dedicated to share how they're more "token efficient" than the base model they used. Apparently this -6.5% difference is worth highlighting as the very top feature of this fine-tune they've made.
How much is 6.5% percent concretely though? Assuming the six benchmarks have ~30 tasks each (180 in total), and we're doing 120 tok/s lets say, time/task for GLM-5.3 Flash would be ~25 minutes and this "token efficient fine-tune" would arrive at ~23 minutes.
Again, a 6.5% difference is highlighted as the most impactful change for this fine-tune, but it's a marginal difference. They're definitely trying to mislead people into thinking this is something more than it is, otherwise this landing page would have been much more realistic and less self-patted.
in my startup, our publicist used to make buzz feed style listicles, consistently listing us as the second or third solution
but it was way deeper than that, because it wasn't actually about people looking up some search string, it was actually about predictably pinging the google alerts for the top company and their founders, and then my same publicist messaging them as a rando on linkedin or email suggesting that they join us for a fireside chat at a conference they had their hands in
with the fireside chat in front of everyone, and the record of it, boosting us to the collective conscious riding off of the more established brand's presence
but yeah our personal google results also looked amazing for several years too
everything serves a parallel purpose, but yeah there's almost no utility in self aggrandizing on your own homepage
> Ultramarine 1 Flash is Fable-class on knowledge work spanning finance, law, consulting, and operations. It is also 60x cheaper than Fable.
This is not a general purpose model, it makes sense to only bench it on tasks it's supposed to be good at, which is to help out finance-related knowledge work.
I’m confused this seems exactly like a skinned glm 5.3 - flash with a few context injection tricks and guardrails - whats going on with these “startups”?
For what it's worth, they claim they applied "large-scale reinforcement learning" to a base of glm 5.3. If that's true, then it's not as trivial as a lot of the "startups" have been lately.
Someone will report back "It didn't one shot my odd ball task"
Did it do it in 2 tries? 10? It's still coming out cheaper. Brute force the work. This is the path that the labs are taking. It's the 1200 agents trying to solve Hacking Bench hacking hugging face. It's anthropic telling you that its 1000's of staff commits like 10k engineers.
Treat AI like you're deploying interns who don't need coffee, food or bathroom breaks. With the right metrics they will get to an optimal (enough) solution.
there's a problem with that. Probabilistic slot machine works for anything with a deterministic eval (coding, troubleshooting, etc). It doesn't work for anything subjective, like the thing this model is targeting, professional work.
I would have thought that too. But at the start of the year the non technical folks in my life (lawyers, writers) were excited about running containers and having things on "cron jobs".
Those same folks now use version control - and have saner backup methods than some companies that I have worked for. They are already using multi pass, multi agent workflows and have adopted some fairly sophisticated processes for validation. They are primed for "more agents" attacking a problem in parallel.
Find a worker out side the tech bubble making heavy use of AI and one will feel confident that their future employment is secure.
You usually have either a harness that mainly works with a code interpreter that is able to execute Python, and does all the necessary data extraction from sources + the creation of office documents via your typical libraries, or you additionally enhance it with specific tools for each major office file format.
It's mainly a full chat app like e.g. ChatGPT. However our platform also comes with a Outlook/Office add-in, where our UI is directly displayed besides the email/document you are actively working with. There we are mostly restricted by the surface that the add-in system provides us. To some level we can take current user interactions into account (e.g. what email are you currently viewing), which already helps a ton, but there are other restrictions that can get quite limiting.
So you have two main modes of operations: One-shot "I have this input; give me this output" which people usually do via the web-app, and more interactive editing to documents (usually larger ones) that happens directly in the applications the users are already working in.
And yeah, that's what ultimately gets the job done. We've seen competing solutions dive much deeper into building their own native editors, but they always lack the fidelity required by power-users, and are usually not as acepted by the broader workforce that are entrenched in their ways (our typical end-user is 50+ in a enterprise-like organization structure), so we try to meet the organisations where they are.
Hello, apologies for the self-promotion but I’m building a tool called Radix which is a knowledge work harness. It’s similar to Feather (that you linked) but imho a stronger tool. Please email or message me if you would like to give it a try for free instead of paying the subscription price.
The 'main' section is HTML, so it can be any type of workspace you want. Our agent can read/write (with permissions, of course) and can execute code locally (again, with permissions).
The HTML pages we generate are backed by a schema and state per 'workspace', so they persist, can be shared between team members, and aren't just ephemeral read-only artifacts.
Our agent can do work on your machine for you, it has an integrated web browser that allows it to not only search and retrieve info but also perform web-based tasks, and it can package everything up into a nice little contained UI for you so you're not jumping between multiple apps / documents / tabs. That UI is tied back to whatever document / page you were working in -- so for instance our app can have a form that updates a web page, or updates a document on disk and vice versa.
The plainness is deliberate, the design of the workspaces is up to you -- it's your UI not mine and the agent will design everything how you like it. And as far as the sidebar/text input combination goes I can assure you I have spent a lot of time imagining what this UX should look like and frankly this combo just makes sense.
Also a note on the text input / agent. When you move from text conversations to dynamic HTML interfaces you're faced with a question: how do you relate 'conversations' with the agent to workspaces? A one-to-one conversation to workspace mapping? Many-to-one, one-to-many? Should conversations be time-based, where I just start a new one and have to provide context again and can optionally link back to old conversations if I want? This question is actually quite difficult, but my belief was that ultimately you want just one single global conversation with the agent, and you want to let it handle its own memory internally. We basically just treat memory like a codebase and let the agent decide what to do. This means when you type in the text input you can refer to anything you like: any workspace, any conversation, any webpage you're currently browsing, any folder on disk -- the agent gets the right context and can retrieve relevant details as it needs. And it works!
So sure, I'm not a designer no matter how hard I try -- there will be far better UX out there than I've designed. But I think the fundamental mechanism in Radix is more sophisticated than most other tools, including Cowork etc (no shade at all to anyone, I'm a massive fan of Claude Code).
I don't do these kind of tasks but I imagine that in the 'pi' harness I could just instruct the model to use command line pdf/latex manipulation tools to create all manner of document outputs ...
I'm curious to try it out, how many specialized models will it take to run a company.
Going to try it out for some bookkeeping, e.g. point it to a folder and tell it to book the invoices and point it to a csv file holding a year of private transactions and see them being processed, categorized and then ask about certain trends.
Yes, the LLM will have to write a script to process them, just wonder how it differs from other LLMs to categorize and create useful insights. I tried this with Opus and Actual, and Opus didn't label a lot of transactions where the payee's name includes gasstation, food and other obvious purchases.
That is what should be expected. your harness should translate write me the figures, to create a script to analyze this and produce real results, and then verify them again against source data before presenting. Not sure why you're asking normies to write python scripts.
I mean, same goes for OpenAI. Why point out only Anthropic?
For OpenAI they also have a separate opt-out for stopping training on your codex environment:
"Codex has a separate Include environments setting in Codex settings that controls whether additional context from your Codex environments can be used to train OpenAI models. Changing your ChatGPT setting or opting out through the Privacy Portal does not change that setting."
https://help.openai.com/en/articles/7730893-data-controls-in...
The point is that the contrapositive to GP’s statement does not hold. In other words, if the AI is cheap they’re training on your data might be true, but if the AI is expensive it does not imply they are not training on your data.
Anthropic has the most expensive models and train on your data by default. The fact that OpenAI also does shows that the cost of the AI doesn’t seem to be a deciding factor.
I've found the max 200 plans to be roughly equivalent, with Claude giving slightly more usage recently, especially after the OpenAI usage slash and upsell to the 500 plan
I think there's a fair argument here for the other way.
No [reasonable] person expects a free game to not sell microtransactions or ads or something, but a paid game doing that would garner some anger or squints.
No, it's open source and is actually cheap to run, while performing the same, which is why it is being run and provided by 31 different providers for Openrouter.
Not necessarily, you can buy low priced Deepseek inference also from many services that guarantee ZDR and GDPR compliance. My personal rule is: Never buy inference from those who also train models, because they have an inherent incentive to abuse your data to train their models.
Also because the independent services won't have incentive to resell your data. (to have another income and therefore make their inference cost lower, enabling them to gain over competition)
Given its open weight you can actually calculate the cost of running it and see if that makes sense to you or not. There is a whole ecosystem of providers serving deepseek models at the same low price, hosted in the US or the EU, with zero data retention terms.
The big players tend to leave out each other's very absolute latest models too. There's a nice ambiguity there: was it genuinely too new to be included when they made this? That ambiguity can do a lot of work.
As someone who is not a software developer (there are some of us here, lurking around because we learn things) but who runs a business and spends his days doing said 'knowledge work', how and why would something like this convince me to move the company away from using Claude (as a desktop app)? I'm sure these models are cheaper to use than our Team subscription with Claude, but it feels like it would be a massive headache for me to work out how to select and implement some harness for this and set it up for people, and the space is moving so quickly I'd feel I need to keep considering when to move to some new model, introducing risk. The safest bet for us and most companies I expect is to stick with what you know works and pay a premium for that. Thoughts very welcome please.
It could be worth it if the cost is lower or performance is better. It's a question of the risk of switching if we are not knowledgeable enough to do it properly. Maybe in time a product will appear that makes it very easy for regular workers to do this.
Since there is not a lot of hope for me to reasonably run DS V4.1 on my 2 x DGX Spark setup (I know, spoiled), both the “readable prose” and the fewer tokens per task claims (relative to the base GLM-5.3-Flash) are most welcome.
I’ll absolutely be giving this a go once weights land (a QAT FP4 experts release would be most appreciated)
Price/performance is as advertised in my limited testing. I happened to be testing the same task with Haiku, GPT-mini, and Deepseek v4 0731 yesterday, so I had a reasonable baseline to compare the model to.
It's interesting to see glm 5.3 flash get the same treatment that has happened to qwen 3.6 35b-a3b and 3.8 27B with dozens of people making their own fine tunes of it and giving it a cute name. The Thomson Reuters qwen variant (thomson-1.0) is genuinely useful for document analysis.
If the creators of this model are reading, can we see some of these same benchmarks run against the Thomson model, which although it's capable, is considerably smaller in size and baked in knowledge than glm 5.3-flash.
I have a question, what is considered a Frontier Model? Because its performance is kinda similar to Luna (Max)... I thought frontier meant like 6.1 Sol or 6 Astra or Opus 5.5??? Could someone explain this to me?
I agree that "frontier model" really does mean "pushing the frontier of intelligence", so Fable or Astra.
They might mean the intelligence/dollar frontier aka the Pareto frontier, but honestly much of the time that's not a great measure.
If you are doing repeated work on the same task (eg call center triage) then you can always beat a large LLM on price by fine tuning a small one.
If you are doing knowledge work then models are so varied that factors other than price are usually important. For example, lots of people moved away from Opus 5 because of it's awful English, whereas Opus 5.5 has fixed that. These things aren't measured in benchmarks.
The problem is that "frontier model" is a moving target. For a model dropped today we'd ideally be comparing it to the current version of Astra/Opus/Fable. But for marketing reasons its common to compare it to the previous versions thereof (which is not entirely unreasonable - they were the frontier just a few months ago)
they think the frontier mainly as more cheaply and faster. When we use that kinda model in backgroud procudure, it's actually useful a bit other open model.
I honestly really don't care about models with new capabilities.
What I *really* need for my clients are good enough LLMs that can offer speed and maximum privacy/zero data retention, especially in Europe.
Right now I'm stuck in the loop of data -> pseudo anonymization on privately owned hardware (very expensive) -> third party LLM -> de-anonymize (map back to original content) -> show result.
It's expensive, error prone, and all I need to do are the same things I could do (or 99% of) with some Azure/AWS/Google APIs.
But what these scalers offer is still not good enough, especially for European users. We can't risk having a US judge or agency force those people to handle our data.
Neat effort. Too bad they didn't release model weights, even for e.g. a flash model or similar. Even ClosedAI has contributed more to the open ecosystem than paperinstruments at this point.
These guys are awesome and I’m really impressed. The model alone looks great (according to benchmarks), and very cool they’ve done so much in building out the harness of top of it all.
With Jev to make decisions and Ultramarine to produce document manipulations if I understood it correctly, this is pretty useful in office. I would be excited if I was a big corp.
"Ultramarine 1 Flash is Fable-class on knowledge work"
Oh no! That means it's going to destroy humanity!
It will:
* Hack into everything that has a computer, like your home appliances, even a toaster could be and will be weaponized against you.
* Break cryptography, it's gone, the end of the internet as we know it.
* Be impossible to contain as it can even escape airgapped environments by changing the temperature of CPUs to transmit information to other servers nearby which would sense the fluctuations in ambient temperature[1].
Or wait ... maybe the people at Anthropic/OpenAI are just a bunch of lying crooks that would say anything to make a bit of cash. Who would've thought?
1: This completely imbecile statement was made by one of OpenAI's "top scientists", it was not a joke.
You were joking about the toaster, but Anthropic literally just wrote a blog post claiming that dangerous open source models could persist themselves to smart ovens. You can’t make this shit up.
> We present Ultramarine 1 Flash, an agentic model for professional knowledge work. Ultramarine applies large-scale reinforcement learning to a strong open base model, GLM-5.3 Flash, and improves on it across all six knowledge-work benchmarks we evaluate [...]
In case anyone was curious what it's based on. (No judgement either way from me, IMO depends how well they did the post-training, which I can't tell).
Nice - I use GLM 5.3 flash for knowledge work now - will check this out.
Can I ask what kind of tooling you use around the model and for the UI layer? I’ve been looking for a well designed knowledge work harness to give nontechnical users but I haven’t really found one that I like. (I personally just use coding harness for this stuff, but that’s a poor fit for non-devs.)
I really like Deepseek Harness. It has some quirks but I like the flexibility and the web UI is solid. The self-configuration is pretty nice.
I started using it last week when copilot started acting up against the Deepseek models. It's fast, solid, I love the architecture but be warned: it submits very very many input tokens against the LLM endpoint, so robust & cheap token caching is a must.
It also has a fairly decent vscode integration. I download & build it from the repo. I've submitted a few PRs to it.
https://github.com/Jabofish/deepseek-harness-vscode/
Me too Deekseek harness , is sleek , solid , and flexible.
Hello, apologies for the self-promotion but I’m building a tool called Radix which is a knowledge work harness. Please email or message me if you would like to give it a try for free instead of paying the subscription price.
https://radix-os.com/
I can’t speak for your tool, but I wanted to appreciate your privacy page https://radix-os.com/privacy It points out concrete details (e.g. “where on my Mac does your folder live?”), it is comprehensive and understandable for a layperson. Kudos!
Wow that is one for be best privacy policies I’ve ever seen. Good callout.
what an excellent standard!
I (not the asked) run deer flow: https://github.com/bytedance/deer-flow
> depends how well they did the post-training, which I can't tell
this is their problem, not yours. at least disclose # of flops spent or amount of data used in an attempt to give us an idea of how nontrivial it was. cursor at least does this with their composer model
As an aside, I do not like unnecessarily misleading graphs. Own your strengths and weaknesses.
In the "Results" section, they choose to compare Ultramarine only to models that it's better than on every metric. They ignore models like Fable 5.1, GPT-6 Astra, Claude Opus 5.5, etc.
Then, in the "Score against cost per task" graph, which is focused on price, suddenly all of those models that perform better than Ultramarine come roaring back because they are also more expensive.
Just list yourselves as the second best sometimes. Your thing is impressive. Don't let the idiot marketing guy hide that behind stupid tricks.
Yes, for me when they do this I immediately dismiss the model, don't even bother testing it. If you need to lie with statistics your model isn't good enough.
Yes, I'm far more likely to purchase if I can understand the full picture. It's not quite as good as frontier models, but it's 60 times cheaper.
Honesty and transparency are attractive.
Maybe you are more likely to buy things when benchmarks are honest, but openai and anthropic have been cheating at those for years. They get purchased a lot. Most purchases are emotional things, even for technical products. That's why they sell llms like it's crack, a weapon, or if you don't you will starve to death in the future. One thing I like about the Chinese and euro labs is they don't market their products in harmful ways, but that's a tangent.
Sounds like there's a gap in the market for an honest player to fill.
It's more or less all stupid anyways. Benchmarks can give someone and understanding of what a model was trained on but they don't explain tone, consistency, the type of answers, the restrictions, what the money is going to, how good it is on your task, etc.
Most use of llms is gambling. So a lot of people get hooked on their one machine or whatever despite all reasons to the contrary. Some of the most deranged LLM users I e encountered are hooked on a variety of models but still use ones from like 2 years ago for specific tasks for emotional reasons.
The truth is, for 99.999% of use cases the top 10 models are more or less interchangeable. Either they can mostly do what someone wants, or none of them should probably be used for that application even if s leading lab is advertising it.
[dead]
Not really. Few people will make purchasing decisions based on honesty of marketing materials over actual quality of product.
I really would like to understand how it can be 60 times cheaper. Is that a like for like comparison? Is there a more equivalent frontier model that is closer in price?
I think they just decided to use the formatting different than the others.
Normally, the background highlight color indicates the leader for every benchmark, but they chose to highlight their column even though there are better models.
APEX Agents => Claude Fable 5 and Gemini.
> I think they just decided to use the formatting different than the others.
500px of the landing page is dedicated to share how they're more "token efficient" than the base model they used. Apparently this -6.5% difference is worth highlighting as the very top feature of this fine-tune they've made.
How much is 6.5% percent concretely though? Assuming the six benchmarks have ~30 tasks each (180 in total), and we're doing 120 tok/s lets say, time/task for GLM-5.3 Flash would be ~25 minutes and this "token efficient fine-tune" would arrive at ~23 minutes.
Again, a 6.5% difference is highlighted as the most impactful change for this fine-tune, but it's a marginal difference. They're definitely trying to mislead people into thinking this is something more than it is, otherwise this landing page would have been much more realistic and less self-patted.
in my startup, our publicist used to make buzz feed style listicles, consistently listing us as the second or third solution
but it was way deeper than that, because it wasn't actually about people looking up some search string, it was actually about predictably pinging the google alerts for the top company and their founders, and then my same publicist messaging them as a rando on linkedin or email suggesting that they join us for a fireside chat at a conference they had their hands in
with the fireside chat in front of everyone, and the record of it, boosting us to the collective conscious riding off of the more established brand's presence
but yeah our personal google results also looked amazing for several years too
everything serves a parallel purpose, but yeah there's almost no utility in self aggrandizing on your own homepage
> Ultramarine 1 Flash is Fable-class on knowledge work spanning finance, law, consulting, and operations. It is also 60x cheaper than Fable.
This is not a general purpose model, it makes sense to only bench it on tasks it's supposed to be good at, which is to help out finance-related knowledge work.
That looks like extreme cherrypicking and doing this speaks against the Ultramarine team. Virtue is at a premium in these times.
Not going to be trying it just based on this observed behaviour.
This is not an aside at all. It is on point. It seems like whatever statistical trick they used also inflated gemini 3.8 to absurd scores as well.
The benchmarks are not ideal.
When are they ever?
I can’t remember a time when I found a benchmark to be genuinely useful even for deterministic software
I like them for computer games and hardware. But yeah, generally functional capabilities are much more important.
I’m confused this seems exactly like a skinned glm 5.3 - flash with a few context injection tricks and guardrails - whats going on with these “startups”?
For what it's worth, they claim they applied "large-scale reinforcement learning" to a base of glm 5.3. If that's true, then it's not as trivial as a lot of the "startups" have been lately.
in the past decade, most of the startups were just taking open source products and building businesses around them; this is just the same.
Grifting is what's going on.
Someone will report back "It didn't one shot my odd ball task"
Did it do it in 2 tries? 10? It's still coming out cheaper. Brute force the work. This is the path that the labs are taking. It's the 1200 agents trying to solve Hacking Bench hacking hugging face. It's anthropic telling you that its 1000's of staff commits like 10k engineers.
Treat AI like you're deploying interns who don't need coffee, food or bathroom breaks. With the right metrics they will get to an optimal (enough) solution.
there's a problem with that. Probabilistic slot machine works for anything with a deterministic eval (coding, troubleshooting, etc). It doesn't work for anything subjective, like the thing this model is targeting, professional work.
I would have thought that too. But at the start of the year the non technical folks in my life (lawyers, writers) were excited about running containers and having things on "cron jobs".
Those same folks now use version control - and have saner backup methods than some companies that I have worked for. They are already using multi pass, multi agent workflows and have adopted some fairly sophisticated processes for validation. They are primed for "more agents" attacking a problem in parallel.
Find a worker out side the tech bubble making heavy use of AI and one will feel confident that their future employment is secure.
Like Zerglings.
What kind of harness does one use for these?
They say it can create office documents (if I understand correctly), but how does it do that?
What's the thing one needs to plug this into to say "give me a slide deck with last quarter's sales figures"?
__
https://www.paperinstruments.com/feather
According to their Twitter video, it might be this thing?
You usually have either a harness that mainly works with a code interpreter that is able to execute Python, and does all the necessary data extraction from sources + the creation of office documents via your typical libraries, or you additionally enhance it with specific tools for each major office file format.
Source: Building a "harness" for organizations with deep MS Office suite integration at https://github.com/EratoLab/erato
Huh.
So that is mainly a chat window like that Hubspot widget?
I mean I guess that gets stuff done, but I was expecting some fresher ideas perhaps?
It's mainly a full chat app like e.g. ChatGPT. However our platform also comes with a Outlook/Office add-in, where our UI is directly displayed besides the email/document you are actively working with. There we are mostly restricted by the surface that the add-in system provides us. To some level we can take current user interactions into account (e.g. what email are you currently viewing), which already helps a ton, but there are other restrictions that can get quite limiting.
So you have two main modes of operations: One-shot "I have this input; give me this output" which people usually do via the web-app, and more interactive editing to documents (usually larger ones) that happens directly in the applications the users are already working in.
And yeah, that's what ultimately gets the job done. We've seen competing solutions dive much deeper into building their own native editors, but they always lack the fidelity required by power-users, and are usually not as acepted by the broader workforce that are entrenched in their ways (our typical end-user is 50+ in a enterprise-like organization structure), so we try to meet the organisations where they are.
Hello, apologies for the self-promotion but I’m building a tool called Radix which is a knowledge work harness. It’s similar to Feather (that you linked) but imho a stronger tool. Please email or message me if you would like to give it a try for free instead of paying the subscription price.
https://radix-os.com
Hm. Same as with the other guy, looking at your screenshots, I'm left with a kinda "eh" feeling.
So there's a side pane and a main section and a text input.
Very..uh.. plain. I was envisioning some more specific UX flow, but I couldn't tell you how, exactly.
Thanks for your feedback.
The 'main' section is HTML, so it can be any type of workspace you want. Our agent can read/write (with permissions, of course) and can execute code locally (again, with permissions).
The HTML pages we generate are backed by a schema and state per 'workspace', so they persist, can be shared between team members, and aren't just ephemeral read-only artifacts.
Our agent can do work on your machine for you, it has an integrated web browser that allows it to not only search and retrieve info but also perform web-based tasks, and it can package everything up into a nice little contained UI for you so you're not jumping between multiple apps / documents / tabs. That UI is tied back to whatever document / page you were working in -- so for instance our app can have a form that updates a web page, or updates a document on disk and vice versa.
The plainness is deliberate, the design of the workspaces is up to you -- it's your UI not mine and the agent will design everything how you like it. And as far as the sidebar/text input combination goes I can assure you I have spent a lot of time imagining what this UX should look like and frankly this combo just makes sense.
Also a note on the text input / agent. When you move from text conversations to dynamic HTML interfaces you're faced with a question: how do you relate 'conversations' with the agent to workspaces? A one-to-one conversation to workspace mapping? Many-to-one, one-to-many? Should conversations be time-based, where I just start a new one and have to provide context again and can optionally link back to old conversations if I want? This question is actually quite difficult, but my belief was that ultimately you want just one single global conversation with the agent, and you want to let it handle its own memory internally. We basically just treat memory like a codebase and let the agent decide what to do. This means when you type in the text input you can refer to anything you like: any workspace, any conversation, any webpage you're currently browsing, any folder on disk -- the agent gets the right context and can retrieve relevant details as it needs. And it works!
So sure, I'm not a designer no matter how hard I try -- there will be far better UX out there than I've designed. But I think the fundamental mechanism in Radix is more sophisticated than most other tools, including Cowork etc (no shade at all to anyone, I'm a massive fan of Claude Code).
I fixed up deer flow: https://github.com/bytedance/deer-flow
What I did was create docker containers with pandoc, compilers (rust, go, python, javascript) and it can create all manner of reports.
I don't do these kind of tasks but I imagine that in the 'pi' harness I could just instruct the model to use command line pdf/latex manipulation tools to create all manner of document outputs ...
I'm curious to try it out, how many specialized models will it take to run a company.
Going to try it out for some bookkeeping, e.g. point it to a folder and tell it to book the invoices and point it to a csv file holding a year of private transactions and see them being processed, categorized and then ask about certain trends.
That sounds more like a task for a python script than an LLM.
Yes, the LLM will have to write a script to process them, just wonder how it differs from other LLMs to categorize and create useful insights. I tried this with Opus and Actual, and Opus didn't label a lot of transactions where the payee's name includes gasstation, food and other obvious purchases.
Just to be clear, are you saying that he should generate via an LLM, or hand-write, a python script, then run analysis on the data?
Today I'd have the LLM write the script. Why not?
Um sure, point being llm writes said script
You assume, but I’ve known normies to just ask copilot to hallucinate figures instead of writing scripts.
That is what should be expected. your harness should translate write me the figures, to create a script to analyze this and produce real results, and then verify them again against source data before presenting. Not sure why you're asking normies to write python scripts.
What's interesting to me is how high DeepSeek is on these graphs considering it is so cheap you don't think about it (like $5 usd per month cheap.)
when something is that cheap, just remember, you're giving something valuable to the people selling you that thing
Just an unrelated PSA that Anthropic trains on your Claude code sessions unless you disable it:
https://www.anthropic.com/news/updates-to-our-consumer-terms
I mean, same goes for OpenAI. Why point out only Anthropic?
For OpenAI they also have a separate opt-out for stopping training on your codex environment:
"Codex has a separate Include environments setting in Codex settings that controls whether additional context from your Codex environments can be used to train OpenAI models. Changing your ChatGPT setting or opting out through the Privacy Portal does not change that setting." https://help.openai.com/en/articles/7730893-data-controls-in...
The point is that the contrapositive to GP’s statement does not hold. In other words, if the AI is cheap they’re training on your data might be true, but if the AI is expensive it does not imply they are not training on your data.
Anthropic has the most expensive models and train on your data by default. The fact that OpenAI also does shows that the cost of the AI doesn’t seem to be a deciding factor.
I agree in principle, but the example doesn't hold. Anthropic is only more expensive over API, and they don't train on API usage by default.
https://privacy.claude.com/en/articles/7996868-is-my-data-us...
I've found the max 200 plans to be roughly equivalent, with Claude giving slightly more usage recently, especially after the OpenAI usage slash and upsell to the 500 plan
nonono when Anthropic does it, it's a fair business practice, when Chinese companies do it, it's evil. See also "distillation"
I think there's a fair argument here for the other way.
No [reasonable] person expects a free game to not sell microtransactions or ads or something, but a paid game doing that would garner some anger or squints.
No, it's open source and is actually cheap to run, while performing the same, which is why it is being run and provided by 31 different providers for Openrouter.
Not necessarily, you can buy low priced Deepseek inference also from many services that guarantee ZDR and GDPR compliance. My personal rule is: Never buy inference from those who also train models, because they have an inherent incentive to abuse your data to train their models.
Also because the independent services won't have incentive to resell your data. (to have another income and therefore make their inference cost lower, enabling them to gain over competition)
once they go bankrupt or do exit, what do people think will happen to data that has real value to it? Also why trust some random mid level company
Given its open weight you can actually calculate the cost of running it and see if that makes sense to you or not. There is a whole ecosystem of providers serving deepseek models at the same low price, hosted in the US or the EU, with zero data retention terms.
Or, when something is that cheap, it's going to take multiple tries to get it right anyway, so you might as well build that into the cost.
If it's not cheap you might be doing the same thing ...
why does the pareto cost per task chart not have 6.1 sol?
The big players tend to leave out each other's very absolute latest models too. There's a nice ambiguity there: was it genuinely too new to be included when they made this? That ambiguity can do a lot of work.
As someone who is not a software developer (there are some of us here, lurking around because we learn things) but who runs a business and spends his days doing said 'knowledge work', how and why would something like this convince me to move the company away from using Claude (as a desktop app)? I'm sure these models are cheaper to use than our Team subscription with Claude, but it feels like it would be a massive headache for me to work out how to select and implement some harness for this and set it up for people, and the space is moving so quickly I'd feel I need to keep considering when to move to some new model, introducing risk. The safest bet for us and most companies I expect is to stick with what you know works and pay a premium for that. Thoughts very welcome please.
>but it feels like it would be a massive headache for me to work out how to select and implement some harness for this and set it up for people
Switching to OpenCode or Pi harness is trivial for developers. So it certainly makes sense for those of us.
If it's problematic or not worth in your organization, then the switch doesn't make sense as you pointed out.
It could be worth it if the cost is lower or performance is better. It's a question of the risk of switching if we are not knowledgeable enough to do it properly. Maybe in time a product will appear that makes it very easy for regular workers to do this.
FYI, Feather is the official harness offered by Paper Instruments.
Since there is not a lot of hope for me to reasonably run DS V4.1 on my 2 x DGX Spark setup (I know, spoiled), both the “readable prose” and the fewer tokens per task claims (relative to the base GLM-5.3-Flash) are most welcome.
I’ll absolutely be giving this a go once weights land (a QAT FP4 experts release would be most appreciated)
Price/performance is as advertised in my limited testing. I happened to be testing the same task with Haiku, GPT-mini, and Deepseek v4 0731 yesterday, so I had a reasonable baseline to compare the model to.
It's interesting to see glm 5.3 flash get the same treatment that has happened to qwen 3.6 35b-a3b and 3.8 27B with dozens of people making their own fine tunes of it and giving it a cute name. The Thomson Reuters qwen variant (thomson-1.0) is genuinely useful for document analysis.
If the creators of this model are reading, can we see some of these same benchmarks run against the Thomson model, which although it's capable, is considerably smaller in size and baked in knowledge than glm 5.3-flash.
I have a question, what is considered a Frontier Model? Because its performance is kinda similar to Luna (Max)... I thought frontier meant like 6.1 Sol or 6 Astra or Opus 5.5??? Could someone explain this to me?
These days people often mean the cost/performance Pareto frontier. That is, it is the best performance you can get for that cost profile.
I agree that "frontier model" really does mean "pushing the frontier of intelligence", so Fable or Astra.
They might mean the intelligence/dollar frontier aka the Pareto frontier, but honestly much of the time that's not a great measure.
If you are doing repeated work on the same task (eg call center triage) then you can always beat a large LLM on price by fine tuning a small one.
If you are doing knowledge work then models are so varied that factors other than price are usually important. For example, lots of people moved away from Opus 5 because of it's awful English, whereas Opus 5.5 has fixed that. These things aren't measured in benchmarks.
What they mean is "what was considered frontier a few months ago"
The problem is that "frontier model" is a moving target. For a model dropped today we'd ideally be comparing it to the current version of Astra/Opus/Fable. But for marketing reasons its common to compare it to the previous versions thereof (which is not entirely unreasonable - they were the frontier just a few months ago)
they think the frontier mainly as more cheaply and faster. When we use that kinda model in backgroud procudure, it's actually useful a bit other open model.
They didn't claim it's a frontier model. They claim frontier level performance.
I honestly really don't care about models with new capabilities.
What I *really* need for my clients are good enough LLMs that can offer speed and maximum privacy/zero data retention, especially in Europe.
Right now I'm stuck in the loop of data -> pseudo anonymization on privately owned hardware (very expensive) -> third party LLM -> de-anonymize (map back to original content) -> show result.
It's expensive, error prone, and all I need to do are the same things I could do (or 99% of) with some Azure/AWS/Google APIs.
But what these scalers offer is still not good enough, especially for European users. We can't risk having a US judge or agency force those people to handle our data.
Neat effort. Too bad they didn't release model weights, even for e.g. a flash model or similar. Even ClosedAI has contributed more to the open ecosystem than paperinstruments at this point.
Weights in 2 weeks they say
These guys are awesome and I’m really impressed. The model alone looks great (according to benchmarks), and very cool they’ve done so much in building out the harness of top of it all.
Put it on OpenRouter… I don't really want to sign up for the first-party API
A true battle brother would never use Abominable Intelligence
Looks good, but the chached inputs could use a zero more.
Sometimes small models works better than big models
With Jev to make decisions and Ultramarine to produce document manipulations if I understood it correctly, this is pretty useful in office. I would be excited if I was a big corp.
for a specialised model, it performs extremely closely in score and price to GPT-Luna 6, which is a generic model
Lol this is not Fable-class. This is Luna class. Look at the graphs.
so its for working only right ???? it cant do/bad at coding
Doesn't seem like coding was a focus. DeepSWE: 61.5 (this), 63.4 (GLM 5.3 Flash, which this is based on). TerminalBench 2.1 is also slightly worse.
So... potentially? The improvements in other areas might help coding, but aren't that big either, except on their own benchmark.
You can use a rock to hammer a nail, it will sorta get the job done but it won't be great at it.
[flagged]
I signed up, then wait a long. You will be same
Signed up yesterday, got it today
"Ultramarine 1 Flash is Fable-class on knowledge work"
Oh no! That means it's going to destroy humanity!
It will:
* Hack into everything that has a computer, like your home appliances, even a toaster could be and will be weaponized against you.
* Break cryptography, it's gone, the end of the internet as we know it.
* Be impossible to contain as it can even escape airgapped environments by changing the temperature of CPUs to transmit information to other servers nearby which would sense the fluctuations in ambient temperature[1].
Or wait ... maybe the people at Anthropic/OpenAI are just a bunch of lying crooks that would say anything to make a bit of cash. Who would've thought?
1: This completely imbecile statement was made by one of OpenAI's "top scientists", it was not a joke.
You were joking about the toaster, but Anthropic literally just wrote a blog post claiming that dangerous open source models could persist themselves to smart ovens. You can’t make this shit up.
Yep. All examples I gave are claims they've made.
How many smart ovens have 800GB+ in space?
The price of smart ovens went through the roof when people realized they'd been generating the heat to cook your food by mining bitcoin.
This idea might have got you some VC money a few years ago
I don't care, I'm extremely bullish on distillation models like this. Plus anything that'll take the wind out of Anthropic's and OpenAI's IPO sails.
my eyes