Can Jev Answer Trust and Safety's Hardest Questions

Everyone in the AI space has been talking about one thing and one thing only for the last week: a new model called Jev, developed by TypeSafe AI. Unlike traditional chat models, Jev is designed to make fast, focused decisions. You give it information and a question with defined possible answers, and it returns a choice, score, or probability. Its output is highly structured and easily consumable by software systems. It could sort customer requests by topic, choose the next move for a character in a game, or judge how likely a sales lead is to convert. Its appeal is that it can handle these small decisions quickly and at low cost.

The use cases for a model of this type are so intriguing that many AI companies, including NVIDIA, OpenAI, and Cloudflare, have already begun to develop their own competitors.

In the Trust and Safety space, Jev appears promising. Its ability to make quick decisions and output prediction values could allow platforms to make fast, inexpensive judgments on content moderation and live-filter what their users see.

However, what makes Jev promising for some use cases is also what limits it for others. While it excels at making structured decisions with high accuracy, Trust and Safety is, and always will be, a more nuanced space.

Jev performs poorly in the gray areas. And in the Trust and Safety space, the gray areas are the hard part.

A good way to frame it is that Jev can quickly and accurately make decisions that an average human could make given a few seconds. Decisions like "does this chat message contain sexually explicit language" are usually relatively easy calls. In this regard, Jev looks well suited to make the clear, repeatable calls that constitute a large share of moderation work.

The harder question is what happens when a decision depends on a platform's particular rules, the surrounding context, and how similar cases have been handled before. Judging whether the confederate flag is considered hate speech or whether a painting of someone in a bikini is considered nudity requires much more subtlety and cultural context than Jev's quick decisions can currently provide. These are the calls that define Trust and Safety, and in these scenarios, speed is a poor substitute for subtlety, consistency, and accuracy.

In the future, it's entirely possible that other System One models like Jev will develop to manage the granularity of this complex decision making. For now, this can only be achieved by training ML models with the specifics of the exact type of question in mind. That's why Cinder's value proposition is customizable, trained models that can use multiple factors to make complex decisions. While Jev might succeed with straightforward questions, Cinder's approach builds models to handle the gray area.