Skip to main content
AI & Technology

Meet Jev: AI for Classifying and Evaluating Information

Jev focuses on structured judgments such as which category a document belongs to or whether a person should review it. This introduction explains Choice, Score, Noul and how to read uncertainty without handing every decision to a model.

JevTypeSafe AIAI EvaluationAI ClassificationHuman Judgment
AI-generated image of a person sorting paper notes beside a laptop in a naturally lit office
AI-generated editorial illustration

When people hear AI, they often picture a chat window that returns a written answer, or a tool that creates an image or drafts an article. Much of the software around us needs something shorter. It may need to know whether a document is an invoice or meeting note, which queue should receive a request, or whether a person should review a message.

Jev is a TypeSafe AI model built for this kind of evaluation. You send source text with questions whose answer shapes are already defined. Jev returns structured results that software can use. Its main job is making judgments against stated options and criteria, while conversational and generative AI generally focuses on producing text for a person to read.

Many general language models can also return structured results. Jev is another option designed specifically for evaluation, so a team should compare it with the tools it already uses on the same task.

Picture a shared folder full of documents

Suppose a team receives text files from several departments. Each file belongs in one of four categories: invoice, expense claim, meeting note or other. The team does not need AI to rewrite the documents. It wants a suggested category after the model reads each one.

Jev receives the text and those four options. It returns the option with the greatest weight, a probability distribution across every option and confidence that reflects whether the weight is concentrated on one answer or divided among several. The Choice documentation defines these three parts of the result.

A document that starts with an invoice number and lists purchased items may place nearly all its weight on invoice. A file that contains a supplier quote and a request to make a payment could split the weight between categories. That ambiguity is useful. The workflow can send the file to a person instead of forcing a category and continuing as though the answer were settled.

An other option also matters. If the list does not cover the material people actually submit, the model must otherwise pick the nearest answer even when none fits. The Choice guidance recommends other or none of the above when the option set may be incomplete.

Choice, Score and Noul answer different questions

TypeSafe defines three Jev question types. Its introduction recommends keeping each question narrow. When a decision depends on several factors, ask about them separately and combine the results in software.

Choice applies when the answer is one item from a defined set, such as a document category or the team that should receive a request. It returns the selected option, probabilities for every option and confidence.

Score applies when the answer lies on ordered, described levels. A team might rate document completeness from "insufficient information" to "all required details present." The descriptions must separate the levels clearly. A Score can sit between levels because it is calculated from the weight across all levels, rather than selected as an arbitrary fixed rating.

Noul evaluates a yes-or-no statement, such as "This document states a payment due date." Its result is a value between 0 and 1. Noul has no separate confidence field, so a rule written for Choice or Score confidence should not be copied over to it.

All three question types can use the same source text, but Jev evaluates each question independently. Asking which category a document belongs to should not change the answer to a separate question about whether it includes a due date.

Probability does not remove human judgment

Choice and Score return probability and confidence. The terms are related but not interchangeable. Probability shows how the weight is distributed across the options or levels. Confidence is calculated from the shape of that distribution. Weight concentrated on one answer tends to produce higher confidence. Weight spread across several answers produces lower confidence. TypeSafe's confidence documentation explains how software can use this to continue, ask for more information or route work for review.

Neither value guarantees that one answer is correct. A workflow still needs a response to mistakes based on their consequences. A meeting note placed in the wrong folder may be easy to repair. A missed invoice may not be. Review thresholds should come from the organisation's own material and error costs, rather than a number copied from a documentation example.

People still define the categories, describe each option, examine ambiguous examples and decide what the software may do next. Model output makes uncertainty visible. It does not transfer accountability away from the process owner.

What the evaluation cannot establish

Jev evaluates the text it receives against the question it is given. A result should not be stretched beyond that question. Asking whether a passage resembles AI-written material produces a model judgment about textual characteristics. It does not prove who wrote the passage.

The same applies to a question such as "Will this post go viral?" Actual reach depends on the audience, timing, channel, distribution and events outside the text. Jev could evaluate a criterion the team defines, such as whether the topic is clear, but that answer is not a future view or share count.

The model also has practical boundaries. TypeSafe's Jev 1.13 notes describe limitations with exact arithmetic, counting, date comparisons and multi-step indirection. Its model reference says Jev accepts text and works best in English. A Thai use case therefore needs tests with the Thai abbreviations, misspellings and mixed-language text people actually submit. English results do not establish Thai performance.

Why generation and evaluation are separate jobs

Consider an article workflow. A generative model might draft the copy. Evaluation then asks different questions: which category fits, which claims need source review, or which editor should receive it. Separating the jobs makes it easier to see what the system is creating and what it is judging.

The boundary also lets a team change its categories without replacing the tool that created or received the document. When an evaluation is unclear, the workflow can route only that case to a person.

For teams already using the Vercel AI SDK, the AI Gateway Evaluation documentation shows typesafe-ai/jev through experimental_evaluate. The function name marks the interface as experimental, so an operational use should test version changes and retain a rollback path.

A useful first trial can stay small. Choose one collection of documents, define categories that have actual owners and keep examples that do not fit. Compare Jev's suggested categories with human classification, inspect disagreements and set review rules from the impact of each error. That is enough to discover where Jev reduces repeated reading without handing the whole decision to the model on day one.

Primary sources

"Empowering Innovation,
Transforming Futures."

Contact us to make your project a reality.