Jev reading notes: what people found, and what we think

Jev is an AI that picks instead of writes. You give it a situation and a fixed set of options. It returns one choice, a probability for each option and a confidence score. In its first week, many people measured it, drew it and tried it. We collected the pieces we could open and check ourselves. Every number below comes from the linked source. We did not re-measure them.

Our view in short: a judge like this does not replace large AI. It fits in front of it, taking the small, repeated decisions. What decides its value is less its speed and more whether its confidence can be trusted, and who owns the threshold.

1. The launch post · TypeSafe

Introducing System One Models & Jev

In short: "unstructured state in, typed probabilistic decisions out." Responses in 70 to 500 ms, free output tokens, up to 255 options, and training aimed at honest probabilities (RLCD).

Our take: the 70 to 500 ms figure is the company's own and includes a round trip to its servers. Before comparing it with any other number, match how it was measured. That is why we never put it side by side with our 2.4 ms as a contest.

2. One call on one whiteboard · Stanislav Beliaev

One Jev call, start to finish (LinkedIn)

In short: one request goes in and comes out as a choice, a score or a yes/no, all on one whiteboard. It also lists what not to use it for: writing, summaries, code, arithmetic and chained reasoning.

Our take: the best line is "the threshold is a line in your code, not in the model." That is where a person stays responsible.

3. An independent test on 791 public items · Adel Dahani

Jev vs GPT and Claude: Independent Benchmark

In short: two intent-routing tasks and prompt-injection detection, 791 items in all. Jev's median was 0.33 s against 1.17 s for a large general model, about 3.6 times faster. Accuracy matched small general models and trailed the large one by 5 to 6 points on intent routing. When Jev answered only items at 0.80 confidence or higher and passed the rest to the large model, the result matched the large model's accuracy at 26 to 28 percent of its cost.

Our take: the key result here is not speed. It is the split at 0.80. That is the same shape as our "fast judge before the gate." But 0.80 was set on this data, and your work needs its own value. The author also notes confidence intervals 6 to 13 points wide.

4. 240 routing calls · LiteLLM

JEV Classifier: 5.43x as Fast as Haiku, 96% Lower Cost

In short: classifying which model should handle a request, 80 cases three times, 240 calls. Median 126.81 ms against 688.40 ms, about 5.4 times faster, 96% cheaper, and 95% against 73.75% agreement with the expected tiers.

Our take: a router with a few fixed options is exactly where a judge like this fits best. But the author says one person wrote the expected tiers. Re-measure on your own data before borrowing the numbers.

5. Swapping out a model router · Taishi Morinaga

I tried replacing model routing with TypeSafe (Jev)

In short: 40 calls sorting conversations into four difficulty tiers. 0.64 to 0.67 s, about three times faster or more than the earlier setups. On the "medium" tier, confidence came out low, 0.57 to 0.67.

Our take: low confidence on the middle tier is a useful signal in itself. If a judge says "this one is unclear," send only those cases to a person or a larger model.

6. A small test of 17 cases · Souvik R.

jev-test (GitHub)

In short: yes/no, choice and score questions, 17 cases and 23 checks. Every model passed every check. Jev's median was 452 ms, and it was about 35 and 190 times cheaper than two large models. The author calls it one run on clear-cut cases.

Our take: on easy cases everyone is right, so only speed and cost differ. The real test is whether confidence drops honestly on the hard ones.

7. Is Jev BERT in a mask? · Denis Timonin

Jev: an LLM for decisions, not chat (LinkedIn)

In short: Jev does work that BERT-style classifiers did for years. The difference is that a fine-tuned classifier needs retraining for new labels, while Jev takes the options in each request. The piece also repeats the company's own warnings: arithmetic, dates and indirect reasoning are weak spots, and long irrelevant context hurts accuracy.

Our take: we saw the other side of that trade. A small judge narrowed to one job is very fast, but it only knows that job. So choose by how often your options change.

8. A threshold is a budget for human review · Laurie Voss

Will TypeSafe's Jev change how we build AI applications? (LinkedIn)

In short: with a cheap, fast judge you can score every output instead of a sample. With honest confidence, you "set your threshold and how much human review you're buying." The cost is rebuilding apps around many small typed questions instead of one big call.

Our take: the threshold sentence is the one to keep. A threshold is a budget for human attention, so it needs an owner, not just a default.

9. Wiring Jev into a gateway, from Korea · Seulbi Lee

TypeSafe Jev connects to LiteLLM (LinkedIn)

In short: in the LiteLLM gateway, Jev attaches only as a route, not as a listed model, but every token and dollar is still recorded. 177 ms direct, about 400 ms through the gateway. It was not always cheaper than small models; the gain was calibrated confidence.

Our take: an honest field report, and it matches what we found. Use it because the confidence is honest, not because it is cheap.

10. What we built ourselves · MSSJ

• We built a tiny Jev on one PC (video)
• What would a tiny, Jev-like judge look like?

In short: we borrowed the input and output shape, narrowed it to one job (a shooting game) and ran it on one PC. A typical decision took 2.4 ms, and the average hit rate over 58 games was 98.2%.

Our take: narrow the scope and it gets small and fast. But we end where the tests above end. Before speed, test on known answers, and set a rule for handing low-confidence cases to a person.

What's next

If enough people are interested, we'll open-source our small judge on GitHub. If not, we'll keep improving it and share it when there is something worth sharing. If you're interested, drop us a line through contact.

Suggest a source

Seen a good test or post about Jev or decision-only AI? Tell us through contact. We will check the source, add our take and list it here.