If you only need classification, and you can provide some training data, you can ask Codex/Claude to build an embeddings + logistic classifier model for you
For emails, I get 95% accuracy with this method, with only 50-100 examples for training
Training the model takes less than 5 minutes on a CPU
The resulting model is <1MB, and inference is sub 100ms
Some other cool things about this approach:
* the model doesn’t train on some “ideal” or general classification, instead it learns your preferences
* the model runs on pretty much any mobile device and can be retrained online on the device
* privacy, the whole training and inference is 100% local, no data goes anywhere (except whatever you feed codex/claude while building the model)
Note: to do a more general test, I made a classifier for the Banking77 dataset. The model is <10MB, trains in <30s on CPU and gets 94.5% accuracy, which puts it in the top 5?models by accuracy for that set (the best one is at 94.86%, but it’s 350MB in size and takes hours to train on a GPU).
Now that the models are smarter / know more than most of us, a new bottleneck is finding out what you don't know. If you don't know about logistic classifiers and how they could be applied to your problem, you will not ask for it.
These days, I tend to start my coding sessions by the high level problem I'm trying to solve vs the prescriptive, specific solution I may have in mind. It often surfaces ideas and approaches that I did not know about.
I built a whole thing that collects data, trains classifiers, exports models and dataset just for that. Claude writes me a terraform file that contains shape of the classifier and dataset. For images it can create datasets based of another dataset (crop this region from images that have these labels).
Originally it was so I can label data to fine-tune a VLM, but now a few tiny classifiers that run in milliseconds on cpu.
Now its collecting data to make a domain specific BERT and do what Jev does.
Very cool. What kinda of classifications are you running? How big are the models/training sets?
Also curious about if you plan on doing some sort of routing for the requests. Like detecting the type of task to decide which model to route the request to
Some classifiers are tiny - like 2k params, maybe even less.
This whole thing started because I wanted something to help me play Dune Imperium. Even relatively large models with vision encoders couldn't reliably extract the full state of the board. Now that I have ~2k labeled screenshots, I want to train heads on top of SigLIP2 to extract all of that data in one go.
That's how it started. Now the thing supports multiple kinds of datasets:
Images - currently the Dune Imperium and Bolatro screenshots, with SigLIP2 heads being the next step.
STT - my self-hosted Linux dictation tool feeds this dataset. I run Nemotron ASR tuned for my voice.
TTS - for Piper TTS, trained to speak like SHODAN. Trained from data generated by Qwen3-tts + original video games files.
Text pairs - for a 1.2B model that converts normal text into "what would SHODAN say?"
FastApply - a Qwen3.5-4B LoRA adapter for doing fast edits.
Chat threads - all agent/chat threads get saved too, so eventually I can turn the useful ones into a dataset and train a LoRA for a really good Rust-specialized version of Qwen3.8-27B.
Tool calls (extracted from chat threads) - this is where I want something Jev-like, mainly to add an auto-approval mode to my agent harness.
A model router isn't planned because I'm trying to gear everything toward self-hosting, and there just isn't that much to route between. I’ll probably build something Jev-like for smart-home control, though.
The FastApply dataset is already ~20k entries, with the majority of outputs being 8k–16k tokens. The STT dataset is roughly 30 hours and growing.
Basically, the whole thing has turned into a Collect -> Distill -> Train pipeline for whatever I happen to need.
I had a version that on validation dataset nearly 1:1 across entire spectrum, but I ended up linking this revision more.
Also had qwen3-tts version, but no matter what I do it sounded more like brat Cortana. The way piper-tts work under the hood make it sound more machine-like.
Missing the point that they marketing, but so far I saw it perform good at some tasks, and pretty bad on others. I don't think it's all that general purpose they claim it to be.
I did this yesterday and it works incredibly well. I finetuned ModernBERT to classify documents. With zeroshot it achieved around ~30% accuracy, which jumped to 98.2% with finetuning, and latency of around 150ms on my Macbook. Just incredible!
If you don’t see good performance with LRs, you may want to try RBF SVMs. We’ve found they work super well for our use cases with the embeddinggemma model as they can better separate classes in the non-linear embedding space.
Our resulting RBF models are tiny and fit in L1 cache, with microsecond inference latency.
I'm working on exactly this! I'm building a small model that classifies the correct DOM node containing an HTML's article content/title/date/author (given a raw html with a lot of noise/chrome). A fun learning exercise :)
30KB model, 40-50ms inference. Pretty happy with the results so far!
I can see an entire industry of tiny models like this, now that we have AI to help us do the grunt setup work (validation/training data creation, data cleaning, etc). Or just use a general classifier like Jev/Kev ha
I'm interested in the Banking77 example you gave which I want to reproduce. Can you give some details for this run? Also, does the <10 MB size include the embedding encoder? Would appreciate any configuration or code to look at!
The gist uses BAAI/bge-large-en-v1.5, which is 1.2GB approx. You can replace it for all-MiniLM-L6-v2 (91 MB @ fp32 or 45 MB quantized fp16) small enough for mobile/edge. With all-MiniLM-L6-v2 it still gets 93.0% on Banking77, only 1.3 points behind bge-large at 15x smaller
Yes, I ran some benchmarks. This architecture seems to match or beat Jev and Laya in all basic classification tasks (datasets tested: AG News, Emotion, MASSIVE Intent, Banking77)
The type of task in which it does really well, especially against Laya, is classification with >50 classes
But this architecture has no “reasoning”, so it performs rather poorly on tasks that require it, like the ones from the XLNI dataset (Jev/Laya do a lot better on this one)
For the latter cases, you could probably enhance the architecture with a lightweight LLM, something like a Gemma model. Or even some basic MLP
Do you realize people is using LLM's as classifiers, right? For lots of companies and developers reaching an API is feasible, while running a training pipeline, no matter how simple, is not. I know that they should still be gathering data for evaluation and they could use it to train a model instead. But they won't do it, for a variety of reasons.
This is the same route but WAAAY faster and cheaper. And you can modify it like you do with code or prompts. It's really appealing, TBH.
The one thing jev has going for it is a dedicated company focused entirely on making the product good and keeping it maintained. I haven't been willing to jump on board with all these jev-shaped projects because their releases feel driven mostly by opportunism. I'm fine waiting a bit for the opportunists to shake out so we can see who is genuinely committed to bringing something valuable to the open-weight community.
Jev is much better than the traditional ML crowd gives it credit for, but my enthusiasm hits a wall when it comes to their data policy. It is completely draconian. Whatever you feed into the system, they retain.
The jev team needs to release a ZDR product, or their platform is dead on arrival. An open, jev-shaped model will win out solely on that basis.
> I'm fine waiting a bit for the opportunists to shake out so we can see who is genuinely committed to bringing something valuable to the open-weight community.
that is generally a very healthy attitude in the AI space anyway in my opinion.
Some of our R&D departments haven't actually finished an interesting project in years because they keep jumping from trend to trend wanting to try out all the latest shit all the time.
We (1) will not train or fine tune any artificial intelligence or machine learning models on Input, and (2) will not disclose any Input to a third party other than our service providers.
Yeah but they reserve the right to retain the data virtually indefinitely.
These aren’t acceptable terms on a personal or corporate level. I’ve seen some fools brag about proxying their life through jev. Messages, emails, LLM calls, files.
Will they train or fine tune on derivatives of input?
I'd prefer if these companies would just enumerate what they will do with my data rather than these vague over-specific claims about what they will not do, which leave me with more questions than answers.
I was enthusiastic about the release of jev much more than I was openclaw because new generative primitive are fun to play around with. But this may be the fastest I’ve ever gotten to being sick and tired of the discussion cycle around it.
for other providers, e.g. openai and anthropic, zdr is offered by 3p hyperscaler like aws/azure/gcp, right? they have no reasons to risk their business and piss off their customers.
The open source ones- I downloaded a number and tried them and compared to Jev.
Anything that required knowledge / familiarity mmBERT and ModernBERT post-trains performed much worse.
So it seems like they did some kind of useful expansive pre-training.
Things that were Qwen or Gemma Diffusion did better at those kinds of tasks but were generally pretty inconsistent in terms of whether they could succeed repeatedly (and be stable + reliable) on the many types of tasks that are in the cookbook part of the Jev docs.
If you ask Jev similar input + questions, it's pretty stable. And does a reasonable job on a lot of questions.
This one public benchmark (the only I've seen) seems to give the open versions way too much credit. It wasn't my experience at all.
It gave my a false wrong sense of what might be required to get it working for something at work to avoid needing a new subprocessor as - at least on Cloudflare / OpenRouter Jev is third-party not hosted.
We are working on running our own benchmark of Jev and some of the other models.
Our use case is classification that runs in a UI. Currently LLMs have good accuracy, but are too slow (and expensive).
Jev not being available through a cloud provider (Bedrock or similar) makes it more challenging for us to start testing and rolling it out.
I think I found Jev is now available through OpenRouter? Oh but maybe you mean somethign like Bedrock with zero-data-retention offered? Anyway, it was on OpenRouter the other day, lest anyone be confused by above, as I did some testing with it for my pareto frontier tool https://github.com/bglusman/model_skyline (which, apologies, is full of slop because its 100% AI maintained but, may or may not have some utility for guiding automated or manual decisions... it definitely showed that Jev was performing MUCH better than some of the open alternatives to it we tested anyway)
One thing I'm still trying to figure out is how this compares to something like gliner. If you're just doing classification in what situations would you choose kev vs gliner?
If Jev is fundamentally trained using RLCD while you’re building on a Qwen model that was trained using RLHF, how can the resulting model be considered Jev-like?
People seem to turn their brain off when it comes to this type of cargo culting. This doesn’t mean much. Qwen often identifies itself as Claude. Does that make it Claude?
Unless specifically told in a system prompt, the pile of weights has absolutely no knowledge of itself. You could hypothetically train it to answer such questions, but nobody bothers to do this, and ALL "knowledge" embedded in the weights is probabalistic anyway.
(I feel like this should be common knowledge in LLM discussions on HN by now.)
This is less true for modern posttrained models. Model identity can be explicitly reinforced during posttraining. Qwen's own finetuning docs include identity training examples, and Qwen models have been trained with system prompts that explicitly say things like "You are Qwen, created by Alibaba Cloud."
So a model correctly identifying its family doesn't necessarily mean it inferred that from pretraining.
I think with Jev, they took a posttrained model and trained it further, so it did not forget about its earlier knowledge during Owen's own RL.
Right, if a model says it is Qwen there is no way to distinguish a ModernBert fine tuned with Qwen completion data from a Qwen model fine tuned with completion data.
It’s also entirely possible that they used completions from a pool of open weight models.
Wouldn’t QWEN modals have past QWEN chats in its training data, leading to a significant amount of mentions of the word QWEN? Just the question “what model are you” would have been answered deterministically multiple times and they’re now part of the weights.
$ ./build/bin/llama-cli -m Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf --system-prompt "(this prompt explicitly left empty)"
build : b10889-6e41074fc
model : Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf
ftype : Q4_K - Medium
modalities : text
using custom system prompt
> What model are you?
[Start thinking]
Here's a thinking process:
1. **Analyze User Input:** The user asks "What model are you?"
2. **Identify Core Question:** The user wants to know my identity/model name.
3. **Check System Prompt/Instructions:** The system prompt says "(this prompt explicitly left empty)" - this is unusual but doesn't change my core identity. I need to respond based on my actual identity as Qwen / 通义千问.
4. **Formulate Response:** I should clearly state that I am Qwen (通义千问), developed by Alibaba Group's Tongyi Lab, and keep it concise as per guidelines.
5. **Draft Response:** I am Qwen (通义千问), a large language model independently developed by Alibaba Group's Tongyi Lab. How can I assist you today?
6. **Refine according to guidelines:** The draft matches the required identity and tone. It's direct and helpful. Ready.
[End thinking]
I am Qwen (通义千问), a large language model independently developed by Alibaba Group's Tongyi Lab. How can I assist you today?
Okay, I stand corrected that _some_ models may include self-identifying metadata in their training. Although I believe I accounted for that in my comment.
Historically, many do not and there are lots of counter-examples proving this. They merely hallucinate an answer just like anything else. The SAME model may even give different answers to the same prompt when asked multiple times... sometimes they claim to be ChatGPT, sometimes Gemma, etc. The fact that the answer is delivered confidently fools people who don't understand this, and these people then run straight to social media with "proof" of their conspiracy theory that one AI lab "stole" another AI lab's model.
My point stands that unless specifically trained or told, big bags of weights do not possess any inherent introspection. LLMs have many fascinating emergent properties, but this is not one of them.
Can someone explain to me how such self-awareness can be forced into the model. I mean I guess the pre training data could contain all sorts of stuff. How reliable are those hacks. I know that a lot of open weight models answer that they are Claude in the absence of a system prompt. I find destillation not that much of a plausible explanation as typically claude would probably not mention that it is Claude all the time. I find it rather plausible that a foreig. system prompt made it into pre-training. But again: I have no clue how much care is given by models to leave traces for destillation (for closed weights) or post training (for open weights).
for some reason this is really funny to me. it's like the "black museum" black mirror episode where a consciousness in a toy animal can only communicate using very primitive predefined responses
Once the first letter is Q, the rest is probably pretty determined. Can you see the confidence for the first letter (don't want to accept the ToS to follow your link)?
I agree that once "Q" is selected, "Qwen" is by far the most likely choice. What I don't get is why it would start by picking "Q", one of the least-used letters in English, unless it already decided to say "Qwen". Now, as others have pointed out, saying "Qwen" and being Qwen are two separate things (though I don't get why they don't just filter model declarations out of the dataset, or carefully replace them with theirs, as that would easily bias the model to always say their name).
Technically, Q is picked because it has the highest probability of all letters. But it makes a difference whether the probability for Q is barely above a uniform 1/26~3.8% or whether that one letter concentrates >50%. What I remember from reading the docs is that Jev gives you the full probabilities (and the confidence, which is something like normalized entropy).
But in general, we might be reading too much into this. If I were to build something like this, a Qwen model would be among the first things I'd reach for too. Initially just prompted inside a little harness to guarantee you get the desired output. Next step would be finetuning, finally training your own foundation model, if you can muster the funding. In this fast-moving space, I think it's quite understandable that they'd go public with an MVP asap, so likely not much training on their own. And even if they're finetuning, Qwen's baked-in answer (through Alibaba's finetuning) seems likely to survive unless it was explicitly overridden.
TypeSafe is primarily a data research lab, which is how the biggest results in AI get made. We make all the data ourselves. We wouldn’t train on your data even if you asked us to (no offense). We do some pretty sophisticated stuff, but if you want to find out more, we’d have to hire you.
While it is painfully, abundantly clear that such wasn't my point, if we need to stick to the cake analogy, it's like buying a cake from the grocery store, cutting it into six slices, and then reassembling the slices and now claiming that you made the cake yourself. It is, you see, "100% synthetic".
This is a ridiculous conversation, but their claim is such laughable bullshit that it's amazing that anyone actually buys it.
Bit of a Jev explosion going on. Is it because it's taking us back to a simpler time we understand better? Classification models have been around for a while.
The way I see this (I havent played around with Jev or layla the OSS version) is that classifiers have always existed and a recognised tool in the ML world. But, the norm is that one needs to not only know what to classify as, but determine what weights to use to classify the input.
Jev came in, and added that magic of "you dont need to train your classifier or determine the weights" if you dont want to, and just get the classified answer out. I think that's what is making people see this with a glitter in their eyes.
I would be curious to see comparisons of jev and similar things with problem specific classifiers. I think layla suggested making problem specific versions anyway? There is a lot of demand for magic don't do any work solutions, which is kind of weird in an era where agents can really help you build a customised solution effectively.
Yes, I've been enjoying playing with Jev. But I do keep wondering: maybe I could just train my own classifier for each of these things I'm doing?
But it's still pretty nice to have an extremely low barrier to entry to prototype something involving a classifier. I keep being like "hmm, that seems useless actually, let's try something else". It's nice to be able to get to that point before needing to gather all the data and create a bespoke classifier.
>Classification models have been around for a while.
I'm still trying to catch-up on the Jev stuff, but my understanding is that it's basically just a more efficient LLM when all you want is the LLM to produce a classification.
There's more to it, of course, but it's not just "generic" classification ML because it accepts arbitrary inputs and can produce probabilities over arbitrary classes. Not saying this is the first time people have done this, but typically classification tasks are more static and limited.
In the same vein, it's also not just an LLM with structured outputs (which have been a thing for a while) specifically because that is a very inefficient way to approach classification using this kind of architecture. Jev models are much more performant because of how limited they are compared to a full LLM.
So when you want an LLM, but you only really need this kind of classification from the LLM, then Jev makes a ton of sense. This makes sense for me, since I've definitely used LLMs for this kind of classification work and, even then, it kind of felt like using a jackhammer to place some nails, etc.
But an LLM provider could very easily add a "Jev mode" to any existing model, right? LLMs already produce a probability distribution over arbitrary classes. Just tell e.g. 5.6 Luna “here is the user's question, you must respond ONLY with the words 'foo', 'bar', or 'baz',” run a single forward pass of the model, and report the normalized probabilities of 'foo' 'bar' and 'baz' tokens before the first output.
With such an approach you could even retain full reasoning capability
My hunch is that you would need some post training. On top of that; I don’t think the llm itself can read inside the transformer state although I can see how that could be enabled. (I feel that would open up yet another class of exfiltration opportunities)
It reminds me a bit of what Ansible got right: user communication. The underlying tech may have existed for a long time, but the genius is presenting it to a regular developer in a way that reads "yes, even you can understand ML, just using a little JSON". The contribution of that should not be understated, as has been clearly evident recently.
Arguably, it doesn't. Instead of a team of people doing break-fix on golden images you have that same size team of people doing break-fix on upstream playbooks. Lots of software doesn't achieve the goals but has a huge deployment story; not sure those things have ever been related.
It does "work", you can download ansible today and use it, it does what it says. Is it the greatest solution for all use cases in infrastructure? Of course not, nothing is. Do people misuse it? Of course too, we're all human.
Regardless of what tooling you use, we're all building houses of cards, and depending on the situation, try to hold down those cards as well as we can, balancing a ton of other needs and requirements.
The Jev model is economically, but also in terms of compute, a much more efficient model. A normal LLM goes token by token, each token in a separate step. Whereas Jev just returns all the results the first round. So it is much better at classification than LLMs.
Compared to traditional ML classification, Jev works without training, like a LLM.
Previous classification models need to be trained on the specific question/choices you are trying to output. Jev doesn't need to be retrained for every choice set provided.
LLMs can act as classifiers but they still have to generate text output in the form of a JSON object. This means they have to generate every single curly bracket, quote, command, etc. This turns out to be pretty expensive. On the other hand, Jev uses a different decision head so it doesn't generate text output at all, it outputs logits *only* for the choices provided. So it completely avoids the need to generate text at all, which means no malformed JSON and it's much faster as a result.
Finally, Jev also provides confidence scores that are actually reliable (not made up like LLMs).
Classifier models are extremely niche and trained for a singular purpose. A utility classifier that you can one-shot on almost any topic or need is a dramatically different beast.
Is it truly useful or accurate or beneficial? To be seen. But it's the idea that has everyone so captivated. An expert system that is an expert at most everything is a lot more useful than an expert system that is an expert at choosing a bar of soap, for instance.
> and enabled things that were impractical previously
I think that there are not _that_ many use-cases that have been opened up by this that tool-calling on other models didn't solve already. Really depends what benchmark you're looking at. This one against BANKING77[0] has many issues, but suggests it's really not far off DeepSeek 4.1 Flash. This one against BoolQ[1] shows marginal improvement over Qwen3.6. This one against MMLU-Pro[2] (same author as the previous) shows significant improvements over two Qwen models.
So there's definitely _some_ alpha there, but I don't think it's the sea-change that the hype would suggest; that is to say, yes, some things that weren't practical before are now, but many things were already very practical with the existing tools.
The part about "tool-calling on other models didn't solve already" is what gets you, sure I could tool call deepseek, glm or any other model, but the latency is huge and you get no confidence score. I gave JEV a shot via OpenRouter and it has a reply in less than 400ms, it's fast enough and cheap enough that you can hook it up to a game loop for example (so highly state dependant) and it can do decisions in real time.
We know how useful classification models are, but massive pretraining is even more useful. So useful that it's tempting to just use a generic autoregressive model for everything, rather than trying to train a specialized classifier.
I think specialized models of all sorts, if it were possible to train them on equally much data, would outperform general models. In cases where we can have the data, like in self-play in games, they clearly do and have done so for a long time.
But it's a lot of effort to train specialized models. I think it's too early to tell if Jev delivers on its promises to give the best of both worlds, but if it doesn't, we'll have to make it work. It's just the right thing to try.
Jev is creating a sort of identity crisis for me, because the number of absolutely clueless folks parroting the classifier thing is the first time I've seen this sort of mass psychosis in CS upfront.
Like even 5 minutes of tinkering captures why this isn't anymore like BERT or any past classification model than ChatGPT is like those old Markov Chain generators, yet folks cannot shut up about how this is nothing new.
Absolutely scary and makes me wonder how much of the field is just people super confidently discrediting otherwise promising/interesting directions for development for a cheap dunk!
Show me a single example how is this jev thing better than a modern Bert solution?
Or even llm if you claim about versatility. You can easily modify the llm inference code to make it predict a single token represent the classification choice and extract the probability that way.
Sure jev will still be faster, but a local deployed Bert model is way faster than both.
And to get the most out of it you still need to fine tune the models anyway, unless your classification task is just one of those mainstream ones.
I did a GRPO run (multiple now actually) with a per sample rubric that leans heavily on subjective preference judgements that BERT wouldn't have the learning capacity for: not to metion you'd need to finetune hundreds of instances and host them somewhere.
And even if BERT wasn't woefully underintelligent for the task... have 100+ instances of BERT running locally faster than Jev API response times? Sweet rig you must have...
LLMs would not be fast enough without constrained decoding tricks that people fundamentally don't seem to understand make the models much dumber, and sure wouldn't be cheaper or faster.
Again I feel this deep discomfort because presumably you're somewhat intelligent but your opening salvo made it hard not to scream DO YOU EVEN HAVE A SINGLE CLUE WHAT IT DOES instead of giving you my actual answer... yet you're speaking from the chest! If I didn't try it for myself I would have been 100% sucked into you and this ocean of clueless negativity.
-
I apologize if that sounds harsh but it angers me because why should I have to deal with this kind of noise in an already insanely noisy environment? What do you gain from being cluelessly pessimistic?
And dwelling a but more I think it breaks one of my most used filters which was assuming people who know the "old world" of AI/ML are better at judging the "new world" full of hype and noise. Maybe my frustration is also just fear that things moved so quickly that the "old world" is becoming increasingly irrelevant. That'd be really disappointing.
> And even if BERT wasn't woefully underintelligent for the task... have 100+ instances of BERT running locally faster than Jev API response times? Sweet rig you must have...
why the heck do you need 100+ instances of bert. do you even attempt to research about this before?
Finetune and infer: One instance ModernBERT didn't have the learning capacity for a single problem in the shape of my subjective preference task with finetuning, do you not have the basic research taste to realize no conceivable post-training recipe will result in an instance that can zero-shot hundred plus similar questions that vary with each sample?!
You really need to try that to find out?
And again have you actually tried Jev? It has a ton of world knowledge: it's able to infer user personas based on TV show watch histories using fairly recent titles... where the hell do you think that capability is emerging in 395M params?
The irony is if you really want to die on this hill, there are much better angles by focusing on LLMs that've had diffusion heads attached for fast inference with as much of a constrained decoding intelligence penalty: at least that'd put you in the ballpark.
I was being charitable that you know the field and are clueless about Jev, mea culpa for giving you the space to think I'm the one that's missing something.
Please see my other reply, I was not exaggerating when I said this feels like asking why ChatGPT is different than a Markov Chain.
You're going to post-train 100s of instances of BERT? Traditional ML had world knowledge more than a fart?
The closest/fairest comparison is still an LLM, but no one has actually chucked enough compute at post-training to make a better Jev yet.
I'm sure in more time that'll happen, and so my excitement is expanded to Jev-like things... but so far most Jev like things are this weirdly reactionary attempts to steal thunder: is it so bad if we have some team actually invest in a quality post-training receipe to compete?
You already wrote the prompt, no? What I'd do, if I were you, is run the question through a LLM and then come back with targeted questions that it didn't answer.
I did the first part yesterday, jumped down the rabbit hole, and have 3 product ideas in my head now.
"Why is Jev fundamentally better than classification models like BERT or traditional ML?"
I think the reason is in the general ballpark of people throwing LLMs at a huge variety of problems and being too slow disqualifies them from a bunch of things.
Now there's a new training-free thing that is fast enough to be useful on a new class of problem.
If you have a little data and can ask a frontier LLM to train a model, you can probably beat it on average for a specific task.
I do think "I already have an intuition for how to use and compose this!" has been an exciting thing for me. I've already built a few toy systems that I enjoyed doing, in a way I haven't with "hey there's a new version of grok" or whatever, in a very long time.
It’s simply not understanding. In terms of latency, LLM processing for large fact-based decision-making was incredibly slow. The real unlock here was speed in decision-making.
I think it’s timing. So many devs trying to squeeze their subscriptions, build more tooling to throughputMaxx. 6 months ago, i speculate it launches pretty flat.
Thanks to these projects, what was an innovative-but-closed piece of technology one week ago is now much more accessible. Whether they're in it for fame or not, I couldn't care less!
There already was an alternative a year ago, with a published paper and open weight lmao... all the other projects are literal slop shat out by script kiddies 2 hours after the release of jev, it reminds me of the flappy bird era, depressing
I think a great use case for these will be when they have large context windows and are able to enforce styling rules for frontend development, and component creation rules for react. You can then ditch the styles guides and styling skills and create a decision tree for enforcing styling, so that you can't run into drift issues or duplication issues. That's where I'm wasting most of my time right now, constantly correcting all of the UX/UI issues that are created for every single feature.
Back in the good old days we would prevent these ux/ui issues by rigorously enforcing the use of our own stylesheets and classes. Later that grew to only using the company ux components. This was very successful in keeping everything neat and tidy. The only drawback was creating and curating new elements and getting consensus. But otherwise it works wonders.
Try constructing reusable components out of what you are doing instead of building everything up from basic building blocks. This also allows more concrete testing of individual parts and then if you want to change the look you can change it in one place and have it apply everywhere.
Agentic development doesn’t mean “throw all what we learned out of the window”, the same practices that helped speed up and improve quality of work of humans also helps agents. In fact, the multiplier is even bigger. You will notice it in development speed and reduced cost due to avoiding churn.
I use reusable components but the issue is that the model rarely checks to see if a component already exists. Or it will use the wrong one. Or if the component strays from the reusable component it doesn't extend it in a generalized way and will special case inline styles, or it'll create a new component. Rarely does it intelligently figure out the correct course of action.
I've also built up a suite of linter rules to catch the same mistakes the model makes over and over. Still, there are a lot of gaps. I think it's mostly because my codebase is massive at this point. It was easy when the codebase was small and didn't require context gathering to make good decisions.
If you have reusable components it should be quite easy for you to add it yourself. Its okay to code by hand, especially when doing frontend.
When comes to frontend work, the handmade UX designed by a professional will always outperform whats made by an llm.
Sure its 10x slower, but you'll get 20x more usere because it will obvious it was made with care (if done well, and you should learn to do it well). Its the same reason handmade clothing, watches and other luxery items speak to us more, sometimes its marketing but often times its the craft.
There's a reason why all the companies selling the idea that web dev is dead, like lovable, still hire designers with 50k retainers to do their landing pages.
I think this is a good example of "AI tools are not as good as users think they are". If you're having problems with AI tools not checking to see if a component already exists, then they're not ready for prime-time on your codebase. The solution isn't to stop caring whether or not you're using reusable components, the solution is just to acknowledge (regretfully!) that AI coding is not good enough to meet an appropriate quality standard without more supervision than you've been able to use it with in the past.
Start by adding more descriptions to the help and documentation docs of the components. Create an index. Forbid the llm to use any other component than on that list. Let another llm review the work. Forbid it to make custom html.
Then every time it tries to do so ask it why it didn’t use one of the ones from the list. Update the list with more ifthenusethis texts. Also negatives, if such then use y instead.
Hi, I'm currently planning on launching a product like this "Grammarly for Design" in a few weeks. Would you be interested in being part of the alpha group ?
I probably wouldn't be able to for data privacy reasons unfortunately, unless it's using an open-weight model (if it is, let me know). But I wish you the best of luck!
Quite impressed by the energy people are putting into making OSS Jev-like models.
I understand the hype but I wonder: what are the use cases for this kind of model? Could it be used in the context of coding agents, or is it more relevant in totally different situations?
Consider every situation where you "force" an LLM to output only a choice / category, or a set of them. If you have workflows like that, you're now being promised significant cost- and latency reduction.
For coding agents it'd only be useful in a subset of situations. E.g. you could imagine using one to classify bash tool calls into safe and unsafe for example.
To develop a smart ai system for my 2d roguelike platformer?
game has way too many moving system for classic state-machine ai + i cant spare the time to develop it.
its low latency entices me.
> what are the use cases for this kind of model? Could it be used in the context of coding agents
Yeah, it could. The most obvious usage would be to have local fast cheap "feedback" / "control" over a slower more expensive agent (i.e. cc / codex / opencode). Things like "goals" could now be split from a long prompt into "actions" and "verifiers". Where for each action you also produce a verifier. Then after each action you run the verifier w/ this kind of "universal classifier" and decide if the step was done correctly, if it needs follow-up and so on.
Example: implement auth in this repo -> llm_plan() -> for item in plan generate_verifier() -> for item in plan implement() ; verify() ; accept() / followup().
Verifiers could be something like this. take a plan item as input, generate classification questions that might verify the task "is this following project conventions?" | "is this touching files from other tasks?", etc.
You can do that with LLMs, but some things might become cheaper / faster. And you can pretty much use it to check against an ever growing list of conventions. Yours or project specific.
The git repo linked at top has some good examples for email classification for automatic email forwarding to specific departments, along with judging email tone and severity / priority, like for customer service emails.
Edit: The Flipper One is planning to have an LLM acceleration co-processor, and be able to host up to a 4GB VRAM size LLM. One use case they envision in their planning is using the microphone along with text to speech to be able to say, "Create an .ini file for this system with these specs" and the small LLM can do that on-device (its a handheld device) and then the user can use/send/upload that file.
Second Edit: I would love a mini LLM in KiCad or Altium that could take a component datasheet and produce a good footprint and schematic symbol for it.
The big problem is that Jev is only useful when you both:
- need fast response
- can tolerate Jev's mediocrity compared to real frontier models
(I explicitly ignore cost, because if you desperately need a cost-optimized classifier you would just build one)
Outside of fun demos these two rarely come together: if it's critical enough to require sub second speed, then it can't be mediocre.
The reason so many OSS models are being built is that Typesafe team made a ton bombastic claims about Jev being a huge breakthrough, and ML folks are realizing they can build Jev-like model in 2 days instead of 2 years.
Classifiers can be very useful for guiding the agentic loop which is basically a state machine. You have the agent propose a task, write some tests, write code, run tests, tests fail, write more code, go to acceptance, etc. So, a classifier can judge state transitions and decide what the agent should do next for example.
There is an enormous amount of time that goes into ticket and other triaging admin across industries. Its use cases go beyond that, but that alone would save a huge number of person-hours.
I hope these get small and good enough to create “pet like” AIs for games. You know, like scream “follow me” at an NPC, STT stack translates it and feeds it to a local Jev-like model that then picks a number of things for the NPC to do.
I tried to use jev for this. I'll share what I learned for the interested.
--
The setup was a simple map with different rooms. Each room had 1-3 doors.
For the inputs: The AI had an array of "known places" empty at the start, the current position, the current doors with no information about where they lead to, and the list of past actions
The goal / task for the AI was to explore all rooms and save them to known places. The AI needed to decide if to move or save the place at every turn.
--
So I wasn't able to make the AI explore all of the rooms. The AI kinda always wanted to move to the first option when moving. Out of 6 rooms it was able to save 3. My hypothesis is that jev as it is now is really bad at making connections and understanding it's input. So for example, even if it had a list of previous actions, it wasn't able to reason about it and know where to go. For this to work I'd need to explicitly tell it where it did not go. So you could say that the model is also not good with uncertainty / ambiguous scenarios.
edit: one last thing is that i replaced jev with an standard llm and it finished the goal no problem with the same information given
edit 2: it also felt like the same tradeoffs between small model vs large model. With small models you need to be very conscious and careful with the input while large models are more forgiving. Maybe jev is a small model, and we just need a larger one.
Yeah the model has to be small, so need to do as much pre processing for it as possible I guess. It essentially doesn’t have any reasoning, so the reasoning would need to be input context (list of things that need to be done, resource shortfall or items needed or whatever) and assuming that you might run this once per second per NPC, it would need to be teeny tiny, maybe 0.3B or smaller.
I wonder how these would do filtering my spam. I have been using 27B-class models for a while now, and they are nearly perfect at determining what is spam and what isn't. The only disadvantage is computational cost.
Take a look at Thomson 1.0-small, which is a variant of qwen 3.6 35b post trained by Thomson Reuters for text analysis. It classifies text content very well.
Are you on the foundation research team for Thomson? (If so, hiya from B!)
Why would you expect Thomson to be particularly good at spam clf? I figured your additional corpus was all news and legal?
I have no connection with Thomson Reuters other than as an end user of a GGUF of the LLM I mentioned. That said, from my personal experience with this specific LLM, it's a decent improvement over a "base" Qwen 3.6 35B A3B Q8, and it does a good job of analyzing and categorizing documents on relatively small resources. It'll run fine in llama-server in pure CPU only on a 64GB RAM system with plenty of room to spare, takes something like 47GB with RAM reserved in llama-server for cache and full context size.
On Gemma 4 12B, I am getting 220 ms per move or QS. I used it to play the Snake game locally:
prompt_eval=244 ms wall=245 ms schema_cache=hit generated=0
Move limit reached after 200 moves: score=16, length=19.
So, if a 12B dense model can offer this latency on a local old PC, then definitely you can scale it up with more powerful machines and get even lower latency.
I wrote a small proxy that points points to a jev api and a frontier api.
My harness connects to the proxy and only sees the frontier api, models, commands, etc. When I send prompts with tool calls, proxy routes to jev, jev narrows the tools, proxy cleans/sends to the frontier api.
So far in my tests, about 60% less tool calls. I'm also going to implement model switching, so it can use cheaper models. I think my workbench harness needs its prompts cleaned up.
Can someone do a ELI5A of how they achieve classification over any user defined list of items? Normal neural networks do a softmax over a known output set to get probabilities
You can achieve open-vocabulary classification by making the final weights in the softmax come from a category encoder instead of being fixed learned weights. So instead of
softmax(encode(input)*learned_weights)
You have
softmax(encode(input)*encode(categories))
I'm not sure if Jev does it this way, but it's how you get open-vocabulary zero-shot image classification with models like CLIP [1].
I can think of two possible approaches
1. Jev limits to 255 distinct options. So they can preprocess your set of options and “tell” the LLM via input tokens 1 = red, 2 = blue, etc then jev need only output softmax over 255 states while benefiting from pretrain of other LLMs
2. You allow the forward pass to output over the total token state but mask over the logits to limit to the user options. Less plausible? bc tricky when input is multi token which they clearly support.
My guess would be option 1. Didn’t read the kev repo here which would also explain
All these Jev projects… great. But Jev was only just released a week ago. That’s the hard limit on how much effort has gone into all these OSS extensions and derivatives: one week. I don’t therefore see any value in adopting any of them, versus just vibe-coding my own if needed.
Because these decision models do not have tool calling, the knowledge cutoff might become a problem. We'll either have to keep training continuously if we run locally or switch to the newer version every month or so when using a closed one like Jev
even with knowledge cutoff set a second from now, you still want to provide as much info as you can if you’re using such tools for delegating decisions
All the people that are just writing an Jev-like API on top of a normal LLM are missing the point. What makes Jev special is the training data; it's how it's trained. The architecture is probably nothing special. Just a text encoder with parallel prediction branches.
I have tried many of these open-source Jev-like models on some linguistic tasks and they are so bad compared to Jev.
I hope so. And I would really like to try an actual Jev open source model. But it will make it more difficult to market it when someone releases something like that because of so many of these "open source Jev-like model".
I'm just sitting back for a few weeks / a couple months to let it shake out, let others put in all the work, and then see if people are still interested and finding use cases that this access model fits better than the usual chat completions endpoint people are used to.
We are indeed in an era of riches (thanks to the $1 Trillion a year being spent on this tech) that it is improving so fast I can just sit back with my 3 year old hardware and newer, better, more amazing workflows keep becoming possible for me just due to model compression / optimization + workflow developments. If I can't get something working this season, I just wait until 3 months from now and there will be an easier to set up, less resource demanding, better working version I can have my local AI install for me. Pretty wild times.
Ever since Jev blew up, it's wild seeing so many models pop up right away. It looks like there is another new trend in models. It doesn't look like I'd have much use for that in my project(Cortrix), since what I need to solve is semantic accuracy.
The bright side of Jev being so popular could be that many companies and individuals realize that their applications might work well with a System One model, and they decide to run an open-source (or fine-tuned) version on their own
Is Jev a decoder (e.g., BERT) or is it some kind of encoder (e.g., GPT) that just happens to be trimmed down to only outputting a handful of tokens for the answers and their probability?
You've got encoder and decoder reversed. BERTs and other models that primarily convert text (or other input) into latent representations are encoders. Models that convert their internal representations back into outputs are the decoders (in the case of GPT et al., autoregressive decoders because they perform this decoding based on past tokens).
> "[I don't understand what it's actually doing under the hood, or why we should be excited.] Jev feels more and more like a glorified if/else block"
You should be excited because there's so many low-hanging classification problems (e.g. "Is this email spam?", "Is this yelp review happy or sad or neutral?") that a "cheap" LLM is overkill for. It really should be this cheap, and now Jev is the first to do it well and do it at scale.
If I was explaining it to my mom, I'd say "Classifying 1000 yelp reviews used to cost $50. Now it costs 5¢, at similar accuracy"
As there have been a lot of Jev related submissions, can someone point me to a simple guide on how I can use it? For example, say I have a script/workflow where I use OpenRouter for LLM calls, and at some point I want to do a simple classification. Can I still use OpenRouter with some Jev model...?
If you only need classification, and you can provide some training data, you can ask Codex/Claude to build an embeddings + logistic classifier model for you
For emails, I get 95% accuracy with this method, with only 50-100 examples for training
Training the model takes less than 5 minutes on a CPU
The resulting model is <1MB, and inference is sub 100ms
Some other cool things about this approach:
* the model doesn’t train on some “ideal” or general classification, instead it learns your preferences
* the model runs on pretty much any mobile device and can be retrained online on the device
* privacy, the whole training and inference is 100% local, no data goes anywhere (except whatever you feed codex/claude while building the model)
Note: to do a more general test, I made a classifier for the Banking77 dataset. The model is <10MB, trains in <30s on CPU and gets 94.5% accuracy, which puts it in the top 5?models by accuracy for that set (the best one is at 94.86%, but it’s 350MB in size and takes hours to train on a GPU).
Now that the models are smarter / know more than most of us, a new bottleneck is finding out what you don't know. If you don't know about logistic classifiers and how they could be applied to your problem, you will not ask for it.
These days, I tend to start my coding sessions by the high level problem I'm trying to solve vs the prescriptive, specific solution I may have in mind. It often surfaces ideas and approaches that I did not know about.
I built a whole thing that collects data, trains classifiers, exports models and dataset just for that. Claude writes me a terraform file that contains shape of the classifier and dataset. For images it can create datasets based of another dataset (crop this region from images that have these labels).
Originally it was so I can label data to fine-tune a VLM, but now a few tiny classifiers that run in milliseconds on cpu.
Now its collecting data to make a domain specific BERT and do what Jev does.
Very cool. What kinda of classifications are you running? How big are the models/training sets?
Also curious about if you plan on doing some sort of routing for the requests. Like detecting the type of task to decide which model to route the request to
Some classifiers are tiny - like 2k params, maybe even less.
This whole thing started because I wanted something to help me play Dune Imperium. Even relatively large models with vision encoders couldn't reliably extract the full state of the board. Now that I have ~2k labeled screenshots, I want to train heads on top of SigLIP2 to extract all of that data in one go.
That's how it started. Now the thing supports multiple kinds of datasets:
A model router isn't planned because I'm trying to gear everything toward self-hosting, and there just isn't that much to route between. I’ll probably build something Jev-like for smart-home control, though.The FastApply dataset is already ~20k entries, with the majority of outputs being 8k–16k tokens. The STT dataset is roughly 30 hours and growing.
Basically, the whole thing has turned into a Collect -> Distill -> Train pipeline for whatever I happen to need.
> Basically, the whole thing has turned into a Collect -> Distill -> Train pipeline for whatever I happen to need
Amazing, thank you for sharing your setup. Very cool applications
> TTS - for Piper TTS, trained to speak like SHODAN
I didn't know I wanted this! Can just imagine asking how to make a Trinidad Sour and being called an insect lmao
It still requires some post-processing to add glitches and metal sound, final result is like this this: https://drive.google.com/file/d/1CJkK9nHQNBPMr6sIxVmlq8gaxDG...
I had a version that on validation dataset nearly 1:1 across entire spectrum, but I ended up linking this revision more.
Also had qwen3-tts version, but no matter what I do it sounded more like brat Cortana. The way piper-tts work under the hood make it sound more machine-like.
> Now its collecting data to make a domain specific BERT and do what Jev does.
this misses the point of jev somewhat - the point is that this is a foundational, general purpose classifier model - see some good sources https://x.com/mparakhin/status/2101683565520199887?s=12
Missing the point that they marketing, but so far I saw it perform good at some tasks, and pretty bad on others. I don't think it's all that general purpose they claim it to be.
I did this yesterday and it works incredibly well. I finetuned ModernBERT to classify documents. With zeroshot it achieved around ~30% accuracy, which jumped to 98.2% with finetuning, and latency of around 150ms on my Macbook. Just incredible!
If you don’t see good performance with LRs, you may want to try RBF SVMs. We’ve found they work super well for our use cases with the embeddinggemma model as they can better separate classes in the non-linear embedding space.
Our resulting RBF models are tiny and fit in L1 cache, with microsecond inference latency.
I'm working on exactly this! I'm building a small model that classifies the correct DOM node containing an HTML's article content/title/date/author (given a raw html with a lot of noise/chrome). A fun learning exercise :)
30KB model, 40-50ms inference. Pretty happy with the results so far!
I can see an entire industry of tiny models like this, now that we have AI to help us do the grunt setup work (validation/training data creation, data cleaning, etc). Or just use a general classifier like Jev/Kev ha
Is this like for cleaning html data from say common crawl ?
30kb model is super impressive.
What’s the model architecture?
I'm interested in the Banking77 example you gave which I want to reproduce. Can you give some details for this run? Also, does the <10 MB size include the embedding encoder? Would appreciate any configuration or code to look at!
Here's a gist with code you can use to test the Banking77 dataset:
https://gist.github.com/nicobrenner/056a5aaff5d0119c0032ecda...
The <10MB does not include the embeddings encoder
The gist uses BAAI/bge-large-en-v1.5, which is 1.2GB approx. You can replace it for all-MiniLM-L6-v2 (91 MB @ fp32 or 45 MB quantized fp16) small enough for mobile/edge. With all-MiniLM-L6-v2 it still gets 93.0% on Banking77, only 1.3 points behind bge-large at 15x smaller
> For emails, I get 95% accuracy with this method, with only 50-100 examples for training
I'm assuming you only need to consider a single language for your emails?
Mostly 2 languages
Haven’t tried with more
Do you have a specific use case?
I would need at least two languages, probably three. I was guessing this approach might be less multilingual than LLMs.
In my case Norwegian, English and Japanese.
love this idea. did you try comparing to jev?
Yes, I ran some benchmarks. This architecture seems to match or beat Jev and Laya in all basic classification tasks (datasets tested: AG News, Emotion, MASSIVE Intent, Banking77)
The type of task in which it does really well, especially against Laya, is classification with >50 classes
But this architecture has no “reasoning”, so it performs rather poorly on tasks that require it, like the ones from the XLNI dataset (Jev/Laya do a lot better on this one)
For the latter cases, you could probably enhance the architecture with a lightweight LLM, something like a Gemma model. Or even some basic MLP
Do you realize people is using LLM's as classifiers, right? For lots of companies and developers reaching an API is feasible, while running a training pipeline, no matter how simple, is not. I know that they should still be gathering data for evaluation and they could use it to train a model instead. But they won't do it, for a variety of reasons.
This is the same route but WAAAY faster and cheaper. And you can modify it like you do with code or prompts. It's really appealing, TBH.
[flagged]
Man, I'm already burnt out on all this jev talk.
The one thing jev has going for it is a dedicated company focused entirely on making the product good and keeping it maintained. I haven't been willing to jump on board with all these jev-shaped projects because their releases feel driven mostly by opportunism. I'm fine waiting a bit for the opportunists to shake out so we can see who is genuinely committed to bringing something valuable to the open-weight community.
Jev is much better than the traditional ML crowd gives it credit for, but my enthusiasm hits a wall when it comes to their data policy. It is completely draconian. Whatever you feed into the system, they retain.
The jev team needs to release a ZDR product, or their platform is dead on arrival. An open, jev-shaped model will win out solely on that basis.
> I'm fine waiting a bit for the opportunists to shake out so we can see who is genuinely committed to bringing something valuable to the open-weight community.
that is generally a very healthy attitude in the AI space anyway in my opinion.
Some of our R&D departments haven't actually finished an interesting project in years because they keep jumping from trend to trend wanting to try out all the latest shit all the time.
Agentic has now replaced javascript for the "It has been 0 weeks since the last Y framework" meme.
https://typesafe.ai/legal/privacy-policy
In their privacy policy they say
We (1) will not train or fine tune any artificial intelligence or machine learning models on Input, and (2) will not disclose any Input to a third party other than our service providers.
Yeah but they reserve the right to retain the data virtually indefinitely.
These aren’t acceptable terms on a personal or corporate level. I’ve seen some fools brag about proxying their life through jev. Messages, emails, LLM calls, files.
According to https://docs.typesafe.ai/legal they do offer ZDR for enterprise customers.
I missed that. Typesafe, why is this hidden in your docs?!
Will they train or fine tune on derivatives of input?
I'd prefer if these companies would just enumerate what they will do with my data rather than these vague over-specific claims about what they will not do, which leave me with more questions than answers.
They leave the door open for future, maybe novel, uses of your data... horrors beyond their (current) comprehension, I always assume.
What about throughput (reasoning) and output? Or can we reasonably assume those are downstream of input and therefore covered by this policy as well?
They offer ZDR. It was effortless to get.
It's Openclaw all over again.
I was enthusiastic about the release of jev much more than I was openclaw because new generative primitive are fun to play around with. But this may be the fastest I’ve ever gotten to being sick and tired of the discussion cycle around it.
> The one thing jev has going for it is a dedicated company focused entirely on making the product good and keeping it maintained.
by not allowing to benchmark it, they make it user-hostile, you don't know for what kind of quality you pay
> ZDR
Does anyone actually believe this?
for other providers, e.g. openai and anthropic, zdr is offered by 3p hyperscaler like aws/azure/gcp, right? they have no reasons to risk their business and piss off their customers.
Found this benchmark for Jev-class models: https://benchmarkheaven.com/jev-models
There are already many Jev-like models in there.
Edit: No affiliation. Just found it and thought others might find it interesting.
The open source ones- I downloaded a number and tried them and compared to Jev.
Anything that required knowledge / familiarity mmBERT and ModernBERT post-trains performed much worse.
So it seems like they did some kind of useful expansive pre-training.
Things that were Qwen or Gemma Diffusion did better at those kinds of tasks but were generally pretty inconsistent in terms of whether they could succeed repeatedly (and be stable + reliable) on the many types of tasks that are in the cookbook part of the Jev docs.
If you ask Jev similar input + questions, it's pretty stable. And does a reasonable job on a lot of questions.
This one public benchmark (the only I've seen) seems to give the open versions way too much credit. It wasn't my experience at all.
It gave my a false wrong sense of what might be required to get it working for something at work to avoid needing a new subprocessor as - at least on Cloudflare / OpenRouter Jev is third-party not hosted.
Thanks for sharing your findings!
We are working on running our own benchmark of Jev and some of the other models. Our use case is classification that runs in a UI. Currently LLMs have good accuracy, but are too slow (and expensive).
Jev not being available through a cloud provider (Bedrock or similar) makes it more challenging for us to start testing and rolling it out.
I think I found Jev is now available through OpenRouter? Oh but maybe you mean somethign like Bedrock with zero-data-retention offered? Anyway, it was on OpenRouter the other day, lest anyone be confused by above, as I did some testing with it for my pareto frontier tool https://github.com/bglusman/model_skyline (which, apologies, is full of slop because its 100% AI maintained but, may or may not have some utility for guiding automated or manual decisions... it definitely showed that Jev was performing MUCH better than some of the open alternatives to it we tested anyway)
It's for data processing, privacy, and data retention. And not having to onboard a new vendor and data processor.
TypeSafe is still the provider. It's just proxied.
So it's a new subprocessor. Which can often be painful to onboard, especially if not compliant according to your needs.
One thing I'm still trying to figure out is how this compares to something like gliner. If you're just doing classification in what situations would you choose kev vs gliner?
totally, considering gliner2 (and especially 2.5) can do _most_ of things that jev can, minus the flashy demos
If Jev is fundamentally trained using RLCD while you’re building on a Qwen model that was trained using RLHF, how can the resulting model be considered Jev-like?
I can't find it but saw that if you give Jev English alphabet as choices and ask it in a loop what model it is, it would say Qwen
also tried myself: https://console.typesafe.ai/playground?share=shr_1690a3160f1...
People seem to turn their brain off when it comes to this type of cargo culting. This doesn’t mean much. Qwen often identifies itself as Claude. Does that make it Claude?
And Claude often identifies as Qwen or Deepseek when prompted in Chinese.
see sister comment's response https://news.ycombinator.com/item?id=49786548
Sort of implies its basis, doesn’t it?
Claude has identified itself as DeepSeek before if prompted in Chinese, is it DeepSeek?
evidence? I can't get it to do that.
Sure, here's a few reports: https://news.ycombinator.com/item?id=47145081 https://www.reddit.com/r/DeepSeek/comments/1rd5jw7/claude_so...
Then why doesn’t jev identify as Claude?
No, it does not.
Guys, if it turns out to be based on Qwen, you are going to come apologize for the downvotes, right?
That is not how models work.
Unless specifically told in a system prompt, the pile of weights has absolutely no knowledge of itself. You could hypothetically train it to answer such questions, but nobody bothers to do this, and ALL "knowledge" embedded in the weights is probabalistic anyway.
(I feel like this should be common knowledge in LLM discussions on HN by now.)
This is less true for modern posttrained models. Model identity can be explicitly reinforced during posttraining. Qwen's own finetuning docs include identity training examples, and Qwen models have been trained with system prompts that explicitly say things like "You are Qwen, created by Alibaba Cloud."
So a model correctly identifying its family doesn't necessarily mean it inferred that from pretraining.
I think with Jev, they took a posttrained model and trained it further, so it did not forget about its earlier knowledge during Owen's own RL.
Right, if a model says it is Qwen there is no way to distinguish a ModernBert fine tuned with Qwen completion data from a Qwen model fine tuned with completion data.
It’s also entirely possible that they used completions from a pool of open weight models.
"<Q>What model are you?<A>Qwen." is surely in Qwen's training data. It's quite standard to include such meta knowledge during instruction tuning.
Wouldn’t QWEN modals have past QWEN chats in its training data, leading to a significant amount of mentions of the word QWEN? Just the question “what model are you” would have been answered deterministically multiple times and they’re now part of the weights.
This is, in fact, not true.
Okay, I stand corrected that _some_ models may include self-identifying metadata in their training. Although I believe I accounted for that in my comment.
Historically, many do not and there are lots of counter-examples proving this. They merely hallucinate an answer just like anything else. The SAME model may even give different answers to the same prompt when asked multiple times... sometimes they claim to be ChatGPT, sometimes Gemma, etc. The fact that the answer is delivered confidently fools people who don't understand this, and these people then run straight to social media with "proof" of their conspiracy theory that one AI lab "stole" another AI lab's model.
My point stands that unless specifically trained or told, big bags of weights do not possess any inherent introspection. LLMs have many fascinating emergent properties, but this is not one of them.
> The SAME model may even give different answers to the same prompt when asked multiple times
temperature?
Can someone explain to me how such self-awareness can be forced into the model. I mean I guess the pre training data could contain all sorts of stuff. How reliable are those hacks. I know that a lot of open weight models answer that they are Claude in the absence of a system prompt. I find destillation not that much of a plausible explanation as typically claude would probably not mention that it is Claude all the time. I find it rather plausible that a foreig. system prompt made it into pre-training. But again: I have no clue how much care is given by models to leave traces for destillation (for closed weights) or post training (for open weights).
for some reason this is really funny to me. it's like the "black museum" black mirror episode where a consciousness in a toy animal can only communicate using very primitive predefined responses
Once the first letter is Q, the rest is probably pretty determined. Can you see the confidence for the first letter (don't want to accept the ToS to follow your link)?
I agree that once "Q" is selected, "Qwen" is by far the most likely choice. What I don't get is why it would start by picking "Q", one of the least-used letters in English, unless it already decided to say "Qwen". Now, as others have pointed out, saying "Qwen" and being Qwen are two separate things (though I don't get why they don't just filter model declarations out of the dataset, or carefully replace them with theirs, as that would easily bias the model to always say their name).
Technically, Q is picked because it has the highest probability of all letters. But it makes a difference whether the probability for Q is barely above a uniform 1/26~3.8% or whether that one letter concentrates >50%. What I remember from reading the docs is that Jev gives you the full probabilities (and the confidence, which is something like normalized entropy).
But in general, we might be reading too much into this. If I were to build something like this, a Qwen model would be among the first things I'd reach for too. Initially just prompted inside a little harness to guarantee you get the desired output. Next step would be finetuning, finally training your own foundation model, if you can muster the funding. In this fast-moving space, I think it's quite understandable that they'd go public with an MVP asap, so likely not much training on their own. And even if they're finetuning, Qwen's baked-in answer (through Alibaba's finetuning) seems likely to survive unless it was explicitly overridden.
lol, this is hilarious
> If Jev is fundamentally trained using RLCD
Big if. More likely, it seems, is they started with an open LLM model and fine-tuned and repurposed it via their "RLCD" process.
The FAQ on the Jet announcement (https://typesafe.ai/blog/introducing-system-one-models-and-j...) claims it was trained with their own data:
Where Does Our Training Data Come From?
TypeSafe is primarily a data research lab, which is how the biggest results in AI get made. We make all the data ourselves. We wouldn’t train on your data even if you asked us to (no offense). We do some pretty sophisticated stuff, but if you want to find out more, we’d have to hire you.
> We make all the data ourselves.
I mean, this claim is simply preposterous, and is discountable as ridiculous nonsense on its face.
Really? I read it as saying they use 100% synthetic data, which is made in house. Why is that preposterous?
To create a cake from scratch, first you must create the universe.
How do you create "100% synthetic data" that is filled with countless facts, coding patterns, medicine, law, philosophy, etc? The notion is farcical.
Sure, if your point is that nobody bakes cakes, you’re right.
While it is painfully, abundantly clear that such wasn't my point, if we need to stick to the cake analogy, it's like buying a cake from the grocery store, cutting it into six slices, and then reassembling the slices and now claiming that you made the cake yourself. It is, you see, "100% synthetic".
This is a ridiculous conversation, but their claim is such laughable bullshit that it's amazing that anyone actually buys it.
Yeah, and synthetic data is pretty darn good nowadays especially with access to frontier models.
> especially with access to frontier models
... so not made in house.
Also my question.
Bit of a Jev explosion going on. Is it because it's taking us back to a simpler time we understand better? Classification models have been around for a while.
The way I see this (I havent played around with Jev or layla the OSS version) is that classifiers have always existed and a recognised tool in the ML world. But, the norm is that one needs to not only know what to classify as, but determine what weights to use to classify the input.
Jev came in, and added that magic of "you dont need to train your classifier or determine the weights" if you dont want to, and just get the classified answer out. I think that's what is making people see this with a glitter in their eyes.
I would be curious to see comparisons of jev and similar things with problem specific classifiers. I think layla suggested making problem specific versions anyway? There is a lot of demand for magic don't do any work solutions, which is kind of weird in an era where agents can really help you build a customised solution effectively.
Yes, I've been enjoying playing with Jev. But I do keep wondering: maybe I could just train my own classifier for each of these things I'm doing?
But it's still pretty nice to have an extremely low barrier to entry to prototype something involving a classifier. I keep being like "hmm, that seems useless actually, let's try something else". It's nice to be able to get to that point before needing to gather all the data and create a bespoke classifier.
Agreed.
Just to be helpful if anyone is searching for layla, it's laya.
>Classification models have been around for a while.
I'm still trying to catch-up on the Jev stuff, but my understanding is that it's basically just a more efficient LLM when all you want is the LLM to produce a classification.
There's more to it, of course, but it's not just "generic" classification ML because it accepts arbitrary inputs and can produce probabilities over arbitrary classes. Not saying this is the first time people have done this, but typically classification tasks are more static and limited.
In the same vein, it's also not just an LLM with structured outputs (which have been a thing for a while) specifically because that is a very inefficient way to approach classification using this kind of architecture. Jev models are much more performant because of how limited they are compared to a full LLM.
So when you want an LLM, but you only really need this kind of classification from the LLM, then Jev makes a ton of sense. This makes sense for me, since I've definitely used LLMs for this kind of classification work and, even then, it kind of felt like using a jackhammer to place some nails, etc.
Happy to be correct, though.
But an LLM provider could very easily add a "Jev mode" to any existing model, right? LLMs already produce a probability distribution over arbitrary classes. Just tell e.g. 5.6 Luna “here is the user's question, you must respond ONLY with the words 'foo', 'bar', or 'baz',” run a single forward pass of the model, and report the normalized probabilities of 'foo' 'bar' and 'baz' tokens before the first output.
With such an approach you could even retain full reasoning capability
My hunch is that you would need some post training. On top of that; I don’t think the llm itself can read inside the transformer state although I can see how that could be enabled. (I feel that would open up yet another class of exfiltration opportunities)
If you did you still wouldn't match Jev on price and speed though. it's orders of magnitude
> Happy to be correct, though.
Not normally one to point out a typo but this one made me smile
It reminds me a bit of what Ansible got right: user communication. The underlying tech may have existed for a long time, but the genius is presenting it to a regular developer in a way that reads "yes, even you can understand ML, just using a little JSON". The contribution of that should not be understated, as has been clearly evident recently.
Yeah except it doesn't really work. It constantly breaks underneath you. The whole system has to be managed, e.g NixOS, or else it's a house of cards.
I'm also not personally a fan of Ansible, but to claim it doesn't really work is quite breathtaking given the size of the installed base.
Arguably, it doesn't. Instead of a team of people doing break-fix on golden images you have that same size team of people doing break-fix on upstream playbooks. Lots of software doesn't achieve the goals but has a huge deployment story; not sure those things have ever been related.
> Yeah except it doesn't really work
It does "work", you can download ansible today and use it, it does what it says. Is it the greatest solution for all use cases in infrastructure? Of course not, nothing is. Do people misuse it? Of course too, we're all human.
Regardless of what tooling you use, we're all building houses of cards, and depending on the situation, try to hold down those cards as well as we can, balancing a ton of other needs and requirements.
The Jev model is economically, but also in terms of compute, a much more efficient model. A normal LLM goes token by token, each token in a separate step. Whereas Jev just returns all the results the first round. So it is much better at classification than LLMs.
Compared to traditional ML classification, Jev works without training, like a LLM.
Previous classification models need to be trained on the specific question/choices you are trying to output. Jev doesn't need to be retrained for every choice set provided.
LLMs can act as classifiers but they still have to generate text output in the form of a JSON object. This means they have to generate every single curly bracket, quote, command, etc. This turns out to be pretty expensive. On the other hand, Jev uses a different decision head so it doesn't generate text output at all, it outputs logits *only* for the choices provided. So it completely avoids the need to generate text at all, which means no malformed JSON and it's much faster as a result.
Finally, Jev also provides confidence scores that are actually reliable (not made up like LLMs).
It's appealing not having to fine-tune separate model for each use case
So you have more flexibility to get on with building, evolve your business logic etc
Classifier models are extremely niche and trained for a singular purpose. A utility classifier that you can one-shot on almost any topic or need is a dramatically different beast.
Is it truly useful or accurate or beneficial? To be seen. But it's the idea that has everyone so captivated. An expert system that is an expert at most everything is a lot more useful than an expert system that is an expert at choosing a bar of soap, for instance.
It’s because it’s practically useful and enabled things that were impractical previously.
> and enabled things that were impractical previously
I think that there are not _that_ many use-cases that have been opened up by this that tool-calling on other models didn't solve already. Really depends what benchmark you're looking at. This one against BANKING77[0] has many issues, but suggests it's really not far off DeepSeek 4.1 Flash. This one against BoolQ[1] shows marginal improvement over Qwen3.6. This one against MMLU-Pro[2] (same author as the previous) shows significant improvements over two Qwen models.
So there's definitely _some_ alpha there, but I don't think it's the sea-change that the hype would suggest; that is to say, yes, some things that weren't practical before are now, but many things were already very practical with the existing tools.
0: https://sanand0.github.io/llmevals/jev/
1: https://github.com/ekzhang/openjev-sglang/blob/a3554ed9e9c26...
2: https://github.com/ekzhang/openjev-sglang/blob/a3554ed9e9c26...
The part about "tool-calling on other models didn't solve already" is what gets you, sure I could tool call deepseek, glm or any other model, but the latency is huge and you get no confidence score. I gave JEV a shot via OpenRouter and it has a reply in less than 400ms, it's fast enough and cheap enough that you can hook it up to a game loop for example (so highly state dependant) and it can do decisions in real time.
Please refer to the latency and price figures for DeepSeek V4.1 Flash in the first link I shared.
For a while is the keyword. It’s just vibe coders have just discovered the classifiers
We know how useful classification models are, but massive pretraining is even more useful. So useful that it's tempting to just use a generic autoregressive model for everything, rather than trying to train a specialized classifier.
I think specialized models of all sorts, if it were possible to train them on equally much data, would outperform general models. In cases where we can have the data, like in self-play in games, they clearly do and have done so for a long time.
But it's a lot of effort to train specialized models. I think it's too early to tell if Jev delivers on its promises to give the best of both worlds, but if it doesn't, we'll have to make it work. It's just the right thing to try.
Jev is creating a sort of identity crisis for me, because the number of absolutely clueless folks parroting the classifier thing is the first time I've seen this sort of mass psychosis in CS upfront.
Like even 5 minutes of tinkering captures why this isn't anymore like BERT or any past classification model than ChatGPT is like those old Markov Chain generators, yet folks cannot shut up about how this is nothing new.
Absolutely scary and makes me wonder how much of the field is just people super confidently discrediting otherwise promising/interesting directions for development for a cheap dunk!
Show me a single example how is this jev thing better than a modern Bert solution?
Or even llm if you claim about versatility. You can easily modify the llm inference code to make it predict a single token represent the classification choice and extract the probability that way.
Sure jev will still be faster, but a local deployed Bert model is way faster than both.
And to get the most out of it you still need to fine tune the models anyway, unless your classification task is just one of those mainstream ones.
I did a GRPO run (multiple now actually) with a per sample rubric that leans heavily on subjective preference judgements that BERT wouldn't have the learning capacity for: not to metion you'd need to finetune hundreds of instances and host them somewhere.
And even if BERT wasn't woefully underintelligent for the task... have 100+ instances of BERT running locally faster than Jev API response times? Sweet rig you must have...
LLMs would not be fast enough without constrained decoding tricks that people fundamentally don't seem to understand make the models much dumber, and sure wouldn't be cheaper or faster.
Again I feel this deep discomfort because presumably you're somewhat intelligent but your opening salvo made it hard not to scream DO YOU EVEN HAVE A SINGLE CLUE WHAT IT DOES instead of giving you my actual answer... yet you're speaking from the chest! If I didn't try it for myself I would have been 100% sucked into you and this ocean of clueless negativity.
-
I apologize if that sounds harsh but it angers me because why should I have to deal with this kind of noise in an already insanely noisy environment? What do you gain from being cluelessly pessimistic?
And dwelling a but more I think it breaks one of my most used filters which was assuming people who know the "old world" of AI/ML are better at judging the "new world" full of hype and noise. Maybe my frustration is also just fear that things moved so quickly that the "old world" is becoming increasingly irrelevant. That'd be really disappointing.
> And even if BERT wasn't woefully underintelligent for the task... have 100+ instances of BERT running locally faster than Jev API response times? Sweet rig you must have...
why the heck do you need 100+ instances of bert. do you even attempt to research about this before?
the laya paper show that you can do the similar stuff with jev using modern bert only: https://laya.convaiinnovations.com/
and even without the newer wave of applying llm techniques to the older bert models, even flan-t5 was trained for handling 1800+ tasks.
Finetune and infer: One instance ModernBERT didn't have the learning capacity for a single problem in the shape of my subjective preference task with finetuning, do you not have the basic research taste to realize no conceivable post-training recipe will result in an instance that can zero-shot hundred plus similar questions that vary with each sample?!
You really need to try that to find out?
And again have you actually tried Jev? It has a ton of world knowledge: it's able to infer user personas based on TV show watch histories using fairly recent titles... where the hell do you think that capability is emerging in 395M params?
The irony is if you really want to die on this hill, there are much better angles by focusing on LLMs that've had diffusion heads attached for fast inference with as much of a constrained decoding intelligence penalty: at least that'd put you in the ballpark.
I was being charitable that you know the field and are clueless about Jev, mea culpa for giving you the space to think I'm the one that's missing something.
Hey I am clueless, how do I learn more?
Why is Jev fundamentally better than classification models like BERT or traditional ML?
Happy to read a written response or if you suggest a prompt to put into my LLM to get it to research and explain the relevant details.
Please see my other reply, I was not exaggerating when I said this feels like asking why ChatGPT is different than a Markov Chain.
You're going to post-train 100s of instances of BERT? Traditional ML had world knowledge more than a fart?
The closest/fairest comparison is still an LLM, but no one has actually chucked enough compute at post-training to make a better Jev yet.
I'm sure in more time that'll happen, and so my excitement is expanded to Jev-like things... but so far most Jev like things are this weirdly reactionary attempts to steal thunder: is it so bad if we have some team actually invest in a quality post-training receipe to compete?
You already wrote the prompt, no? What I'd do, if I were you, is run the question through a LLM and then come back with targeted questions that it didn't answer.
I did the first part yesterday, jumped down the rabbit hole, and have 3 product ideas in my head now.
"Why is Jev fundamentally better than classification models like BERT or traditional ML?"
I think the reason is in the general ballpark of people throwing LLMs at a huge variety of problems and being too slow disqualifies them from a bunch of things.
Now there's a new training-free thing that is fast enough to be useful on a new class of problem.
If you have a little data and can ask a frontier LLM to train a model, you can probably beat it on average for a specific task.
But... This is the case with LLMs too.
I do think "I already have an intuition for how to use and compose this!" has been an exciting thing for me. I've already built a few toy systems that I enjoyed doing, in a way I haven't with "hey there's a new version of grok" or whatever, in a very long time.
It’s simply not understanding. In terms of latency, LLM processing for large fact-based decision-making was incredibly slow. The real unlock here was speed in decision-making.
Feels like astroturfing.
I think it’s timing. So many devs trying to squeeze their subscriptions, build more tooling to throughputMaxx. 6 months ago, i speculate it launches pretty flat.
Could you link to some of these classification models that can be used as versatile and perform with similar quality, speed and cost?
[flagged]
Thanks to these projects, what was an innovative-but-closed piece of technology one week ago is now much more accessible. Whether they're in it for fame or not, I couldn't care less!
There already was an alternative a year ago, with a published paper and open weight lmao... all the other projects are literal slop shat out by script kiddies 2 hours after the release of jev, it reminds me of the flappy bird era, depressing
I think a great use case for these will be when they have large context windows and are able to enforce styling rules for frontend development, and component creation rules for react. You can then ditch the styles guides and styling skills and create a decision tree for enforcing styling, so that you can't run into drift issues or duplication issues. That's where I'm wasting most of my time right now, constantly correcting all of the UX/UI issues that are created for every single feature.
Back in the good old days we would prevent these ux/ui issues by rigorously enforcing the use of our own stylesheets and classes. Later that grew to only using the company ux components. This was very successful in keeping everything neat and tidy. The only drawback was creating and curating new elements and getting consensus. But otherwise it works wonders.
Try constructing reusable components out of what you are doing instead of building everything up from basic building blocks. This also allows more concrete testing of individual parts and then if you want to change the look you can change it in one place and have it apply everywhere.
Agentic development doesn’t mean “throw all what we learned out of the window”, the same practices that helped speed up and improve quality of work of humans also helps agents. In fact, the multiplier is even bigger. You will notice it in development speed and reduced cost due to avoiding churn.
I use reusable components but the issue is that the model rarely checks to see if a component already exists. Or it will use the wrong one. Or if the component strays from the reusable component it doesn't extend it in a generalized way and will special case inline styles, or it'll create a new component. Rarely does it intelligently figure out the correct course of action.
I've also built up a suite of linter rules to catch the same mistakes the model makes over and over. Still, there are a lot of gaps. I think it's mostly because my codebase is massive at this point. It was easy when the codebase was small and didn't require context gathering to make good decisions.
If you have reusable components it should be quite easy for you to add it yourself. Its okay to code by hand, especially when doing frontend.
When comes to frontend work, the handmade UX designed by a professional will always outperform whats made by an llm.
Sure its 10x slower, but you'll get 20x more usere because it will obvious it was made with care (if done well, and you should learn to do it well). Its the same reason handmade clothing, watches and other luxery items speak to us more, sometimes its marketing but often times its the craft.
There's a reason why all the companies selling the idea that web dev is dead, like lovable, still hire designers with 50k retainers to do their landing pages.
I think this is a good example of "AI tools are not as good as users think they are". If you're having problems with AI tools not checking to see if a component already exists, then they're not ready for prime-time on your codebase. The solution isn't to stop caring whether or not you're using reusable components, the solution is just to acknowledge (regretfully!) that AI coding is not good enough to meet an appropriate quality standard without more supervision than you've been able to use it with in the past.
Start by adding more descriptions to the help and documentation docs of the components. Create an index. Forbid the llm to use any other component than on that list. Let another llm review the work. Forbid it to make custom html.
Then every time it tries to do so ask it why it didn’t use one of the ones from the list. Update the list with more ifthenusethis texts. Also negatives, if such then use y instead.
Hi, I'm currently planning on launching a product like this "Grammarly for Design" in a few weeks. Would you be interested in being part of the alpha group ?
I probably wouldn't be able to for data privacy reasons unfortunately, unless it's using an open-weight model (if it is, let me know). But I wish you the best of luck!
Quite impressed by the energy people are putting into making OSS Jev-like models.
I understand the hype but I wonder: what are the use cases for this kind of model? Could it be used in the context of coding agents, or is it more relevant in totally different situations?
Consider every situation where you "force" an LLM to output only a choice / category, or a set of them. If you have workflows like that, you're now being promised significant cost- and latency reduction.
For coding agents it'd only be useful in a subset of situations. E.g. you could imagine using one to classify bash tool calls into safe and unsafe for example.
To develop a smart ai system for my 2d roguelike platformer? game has way too many moving system for classic state-machine ai + i cant spare the time to develop it. its low latency entices me.
[dead]
> what are the use cases for this kind of model? Could it be used in the context of coding agents
Yeah, it could. The most obvious usage would be to have local fast cheap "feedback" / "control" over a slower more expensive agent (i.e. cc / codex / opencode). Things like "goals" could now be split from a long prompt into "actions" and "verifiers". Where for each action you also produce a verifier. Then after each action you run the verifier w/ this kind of "universal classifier" and decide if the step was done correctly, if it needs follow-up and so on.
Example: implement auth in this repo -> llm_plan() -> for item in plan generate_verifier() -> for item in plan implement() ; verify() ; accept() / followup().
Verifiers could be something like this. take a plan item as input, generate classification questions that might verify the task "is this following project conventions?" | "is this touching files from other tasks?", etc.
You can do that with LLMs, but some things might become cheaper / faster. And you can pretty much use it to check against an ever growing list of conventions. Yours or project specific.
I don’t think this is a very good use case. You could do it better with a strong LLM and structured outputs.
The problem is that you want the model to carefully reason about the goal and code.
Zero shot classification with an approach like this isn’t going to do that. It’ll answer on first pass vibes.
Why not just let the LLM write a test instead of a "verifier"?
The git repo linked at top has some good examples for email classification for automatic email forwarding to specific departments, along with judging email tone and severity / priority, like for customer service emails.
Edit: The Flipper One is planning to have an LLM acceleration co-processor, and be able to host up to a 4GB VRAM size LLM. One use case they envision in their planning is using the microphone along with text to speech to be able to say, "Create an .ini file for this system with these specs" and the small LLM can do that on-device (its a handheld device) and then the user can use/send/upload that file.
Second Edit: I would love a mini LLM in KiCad or Altium that could take a component datasheet and produce a good footprint and schematic symbol for it.
The big problem is that Jev is only useful when you both:
- need fast response
- can tolerate Jev's mediocrity compared to real frontier models
(I explicitly ignore cost, because if you desperately need a cost-optimized classifier you would just build one)
Outside of fun demos these two rarely come together: if it's critical enough to require sub second speed, then it can't be mediocre.
The reason so many OSS models are being built is that Typesafe team made a ton bombastic claims about Jev being a huge breakthrough, and ML folks are realizing they can build Jev-like model in 2 days instead of 2 years.
You should call Jev-like models when you give it a JSON-like structure to produce, it is useful when you need _some_ intelligence in your code.
Edit: I want to add that you can see Jev like a smart if-statement.
Classifiers can be very useful for guiding the agentic loop which is basically a state machine. You have the agent propose a task, write some tests, write code, run tests, tests fail, write more code, go to acceptance, etc. So, a classifier can judge state transitions and decide what the agent should do next for example.
There is an enormous amount of time that goes into ticket and other triaging admin across industries. Its use cases go beyond that, but that alone would save a huge number of person-hours.
Why would you use Jev over Sol/Sonnet though?
Speed/cost benefits of Jev are mostly useless for triage.
Feels like you would be using a dumber model to achieve worse results.
If all you have is a Jev, everything looks like it can be solved with a classifier?
Yeah same. Got access to their API and then realised I don’t really have an immediate use case
I hope these get small and good enough to create “pet like” AIs for games. You know, like scream “follow me” at an NPC, STT stack translates it and feeds it to a local Jev-like model that then picks a number of things for the NPC to do.
I tried to use jev for this. I'll share what I learned for the interested.
--
The setup was a simple map with different rooms. Each room had 1-3 doors.
For the inputs: The AI had an array of "known places" empty at the start, the current position, the current doors with no information about where they lead to, and the list of past actions
The goal / task for the AI was to explore all rooms and save them to known places. The AI needed to decide if to move or save the place at every turn.
--
So I wasn't able to make the AI explore all of the rooms. The AI kinda always wanted to move to the first option when moving. Out of 6 rooms it was able to save 3. My hypothesis is that jev as it is now is really bad at making connections and understanding it's input. So for example, even if it had a list of previous actions, it wasn't able to reason about it and know where to go. For this to work I'd need to explicitly tell it where it did not go. So you could say that the model is also not good with uncertainty / ambiguous scenarios.
edit: one last thing is that i replaced jev with an standard llm and it finished the goal no problem with the same information given
edit 2: it also felt like the same tradeoffs between small model vs large model. With small models you need to be very conscious and careful with the input while large models are more forgiving. Maybe jev is a small model, and we just need a larger one.
Yeah the model has to be small, so need to do as much pre processing for it as possible I guess. It essentially doesn’t have any reasoning, so the reasoning would need to be input context (list of things that need to be done, resource shortfall or items needed or whatever) and assuming that you might run this once per second per NPC, it would need to be teeny tiny, maybe 0.3B or smaller.
I wonder how these would do filtering my spam. I have been using 27B-class models for a while now, and they are nearly perfect at determining what is spam and what isn't. The only disadvantage is computational cost.
Take a look at Thomson 1.0-small, which is a variant of qwen 3.6 35b post trained by Thomson Reuters for text analysis. It classifies text content very well.
Are you on the foundation research team for Thomson? (If so, hiya from B!) Why would you expect Thomson to be particularly good at spam clf? I figured your additional corpus was all news and legal?
I have no connection with Thomson Reuters other than as an end user of a GGUF of the LLM I mentioned. That said, from my personal experience with this specific LLM, it's a decent improvement over a "base" Qwen 3.6 35B A3B Q8, and it does a good job of analyzing and categorizing documents on relatively small resources. It'll run fine in llama-server in pure CPU only on a 64GB RAM system with plenty of room to spare, takes something like 47GB with RAM reserved in llama-server for cache and full context size.
Shouldn't Jev-like models be built on top of diffusion models ? like GSAI-ML/iLLaDA-8B-Instruct ?
That showed us the best results at least
On Gemma 4 12B, I am getting 220 ms per move or QS. I used it to play the Snake game locally:
prompt_eval=244 ms wall=245 ms schema_cache=hit generated=0
Move limit reached after 200 moves: score=16, length=19.
So, if a 12B dense model can offer this latency on a local old PC, then definitely you can scale it up with more powerful machines and get even lower latency.
I wrote a small proxy that points points to a jev api and a frontier api. My harness connects to the proxy and only sees the frontier api, models, commands, etc. When I send prompts with tool calls, proxy routes to jev, jev narrows the tools, proxy cleans/sends to the frontier api.
So far in my tests, about 60% less tool calls. I'm also going to implement model switching, so it can use cheaper models. I think my workbench harness needs its prompts cleaned up.
[dead]
Can someone tell me what is the difference between Jev and a normal neural network that does classification ?
My understanding is: it takes text input and it does one shot classification (no training data)
Yes, this is essentially it.
As a corollary, the output classes can be any set, rather than needing to be set before training.
Can someone do a ELI5A of how they achieve classification over any user defined list of items? Normal neural networks do a softmax over a known output set to get probabilities
You can achieve open-vocabulary classification by making the final weights in the softmax come from a category encoder instead of being fixed learned weights. So instead of
softmax(encode(input)*learned_weights)
You have
softmax(encode(input)*encode(categories))
I'm not sure if Jev does it this way, but it's how you get open-vocabulary zero-shot image classification with models like CLIP [1].
[1] https://openai.com/index/clip/
which amounts to nearest-neighbor
I can think of two possible approaches 1. Jev limits to 255 distinct options. So they can preprocess your set of options and “tell” the LLM via input tokens 1 = red, 2 = blue, etc then jev need only output softmax over 255 states while benefiting from pretrain of other LLMs 2. You allow the forward pass to output over the total token state but mask over the logits to limit to the user options. Less plausible? bc tricky when input is multi token which they clearly support.
My guess would be option 1. Didn’t read the kev repo here which would also explain
All these Jev projects… great. But Jev was only just released a week ago. That’s the hard limit on how much effort has gone into all these OSS extensions and derivatives: one week. I don’t therefore see any value in adopting any of them, versus just vibe-coding my own if needed.
How are you forcing Qwen to answer in a structured way? I like this one better - https://github.com/deepanwadhwa/OpenDecision
Because these decision models do not have tool calling, the knowledge cutoff might become a problem. We'll either have to keep training continuously if we run locally or switch to the newer version every month or so when using a closed one like Jev
even with knowledge cutoff set a second from now, you still want to provide as much info as you can if you’re using such tools for delegating decisions
I noticed it has a pretty small context window of only 32k. For most tasks I guess it would be enough with ample context.
All the people that are just writing an Jev-like API on top of a normal LLM are missing the point. What makes Jev special is the training data; it's how it's trained. The architecture is probably nothing special. Just a text encoder with parallel prediction branches.
I have tried many of these open-source Jev-like models on some linguistic tasks and they are so bad compared to Jev.
It won't be long until people produce a decent training data set generation pipeline.
The number of people working on this is crazy. Something will coalesce.
I hope so. And I would really like to try an actual Jev open source model. But it will make it more difficult to market it when someone releases something like that because of so many of these "open source Jev-like model".
I'm just sitting back for a few weeks / a couple months to let it shake out, let others put in all the work, and then see if people are still interested and finding use cases that this access model fits better than the usual chat completions endpoint people are used to.
We are indeed in an era of riches (thanks to the $1 Trillion a year being spent on this tech) that it is improving so fast I can just sit back with my 3 year old hardware and newer, better, more amazing workflows keep becoming possible for me just due to model compression / optimization + workflow developments. If I can't get something working this season, I just wait until 3 months from now and there will be an easier to set up, less resource demanding, better working version I can have my local AI install for me. Pretty wild times.
I love that, within days of Jev being released, we have a flurry of open alternatives. This is kinda the open-source dream :)
Ever since Jev blew up, it's wild seeing so many models pop up right away. It looks like there is another new trend in models. It doesn't look like I'd have much use for that in my project(Cortrix), since what I need to solve is semantic accuracy.
Been hoping for something in this space. Jev-like decision models on Qwen3.5 could really simplify some of our internal routing logic.
- https://github.com/logan-markewich/jeff - https://github.com/bespokelabsai/nimble
Are these forks? Different orgs doing the same thing as the OP?
Why does nobody ever ship these as a docker image?
I guess you have AI to write your docker files and push your images now.
Why not name it Qev?
The bright side of Jev being so popular could be that many companies and individuals realize that their applications might work well with a System One model, and they decide to run an open-source (or fine-tuned) version on their own
Half-OT:
Is Jev a decoder (e.g., BERT) or is it some kind of encoder (e.g., GPT) that just happens to be trimmed down to only outputting a handful of tokens for the answers and their probability?
You've got encoder and decoder reversed. BERTs and other models that primarily convert text (or other input) into latent representations are encoders. Models that convert their internal representations back into outputs are the decoders (in the case of GPT et al., autoregressive decoders because they perform this decoding based on past tokens).
You got it reversed. Bert is encoder only and gpt is decoder only.
Jev feels more and more like a glorified if/else if block
> "[I don't understand what it's actually doing under the hood, or why we should be excited.] Jev feels more and more like a glorified if/else block"
You should be excited because there's so many low-hanging classification problems (e.g. "Is this email spam?", "Is this yelp review happy or sad or neutral?") that a "cheap" LLM is overkill for. It really should be this cheap, and now Jev is the first to do it well and do it at scale.
If I was explaining it to my mom, I'd say "Classifying 1000 yelp reviews used to cost $50. Now it costs 5¢, at similar accuracy"
so, it IS a glorified if/else block, just in an actual awesome sense and not in a derogatory sense :D
related, vLLM PR nearing merge readiness for Jev like capabilities with diffusion models
https://github.com/vllm-project/vllm/pull/57250
How long does it typically take for something like this to become available on openrouter?
Haven't tried it yet, but it looks like it's already live?
https://openrouter.ai/typesafe/jev-1.13
Using jev for data labeling would be interesting. I wonder how kev compares
Apply it to this: https://minimallysufficient.com/posts/llm-classification-is-...
How is calibration of Jev or Jev-inspired models being evaluated?
Distilling Jev should be super easy and cheap.
what kinda of specs would it need to run?
Interesting approach with Qwen3.5 for decision models. Curious how "tiny" they've made them while keeping LLM reliability for critical paths.
Interesting to see a Jev-like approach applied to Qwen3.5. Always appreciated Jev's simplicity for quick decisions.
Always?
OK - I guess I'll ask here.
As there have been a lot of Jev related submissions, can someone point me to a simple guide on how I can use it? For example, say I have a script/workflow where I use OpenRouter for LLM calls, and at some point I want to do a simple classification. Can I still use OpenRouter with some Jev model...?
Best just to use a skill.
Here is typesafe’s skills.md https://docs.typesafe.ai/agent-skill (which I don’t think is well written, but it’s a start).
Also read their docs, I think they’re interesting.
I need it for running in standalone scripts. How do I get a skill to work there?
I mean, I guess I can have my script call pi and offload it to that, but I just want everything contained in one script.
Oh Jared is cool - he made After and Razzle - nice
looks high lev
[flagged]
[flagged]
[dead]
[dead]