Jevil in the details: Security implications of TypeSafe Jev and other System One models

Summary

TypeSafe's Jev is designed to make the small decisions that keep AI agents moving: which action to take, which option to select, and whether an input meets a given condition. Rather than generating those answers token by token, it returns structured decisions that an application can use directly. We wanted to know whether this change in output also changes the model's exposure to familiar attacks.

We tested Jev and an in-house System One-style model in two scenarios. On a simulated checkout page, an attacker-controlled advertisement caused the browser agent to click away from the user's purchase despite instructions to treat page text as untrusted. In a separate classification test, targeted TokenBreak-style changes lowered toxicity scores for prompts both models had initially flagged. We also observed the browser attack against the conventional version of our in-house model. Together, these results show that a faster decision interface does not remove the need to secure the decisions made through it.

What is Jev?

The excitement around Jev is easy to understand. Modern AI agents depend more and more on hundreds of tiny decisions: which tool to use, whether a given request is relevant, how urgent it is, whether evidence supports a claim, and whether human intervention is required. Running every one of those decisions through a frontier LLM is slow and often expensive. TypeSafe argues that the industry is using a text-generation engine for work that often doesn't require text generation at all.

Jev is TypeSafe's proposed alternative. Described as a "System One" model designed for quick semantic decisions, TypeSafe claims that Jev produces advantages of up to 193 times in speed and 444 times in cost over conventional LLM workflows. The company also emphasizes predictable outputs, long-context support, and the ability to answer multiple questions about the same information in a single request.

These properties make Jev particularly interesting for tool calling. In a traditional agent, the model receives a list of available tools and instructions for using them. When a request arrives, it generates a structured response identifying the appropriate tool and supplying its arguments. Each call may require anywhere from a handful to several hundred output tokens, with latency and cost accumulating throughout a task.

‍

Consider an LLM-based agent that plays poker. With a conventional LLM, a tool call to raise by 100 chips might look like this:

Though this doesn't seem like much text, the token breakdown tells a different story, requiring ~9 tokens to express with most tokenizers, and therefore 9 forward passes of the model to output:

Figure 1: Example tool call, tokenized

A System One model approaches the same decision differently. The harness provides the current game state and asks the model to evaluate a set of possible actions. Jev supports three types of questions:

• Choice: Selecting from a fixed set of options
• Score: Picks a value on a developer-defined scale
• Noul: Returns the probability that the statement is true

For the poker agent, a Choice question could select among fold, call, raise, and all_in, while a Score question could determine bet size along a predefined scale. Conceptually, the results might look like this:

The important distinction is that these values are not generated as a sequence of output tokens. At the interface level, the response is assembled from the model's decision scores. This avoids the autoregressive loop of predicting a token, appending it to the sequence, and repeating until the response is complete. As a result, latency depends less on the length of the structured answer.

Figure 2: Autoregressive vs System One models

This lets System One models produce usable responses much faster than their autoregressive counterparts, as the text/token middleman is replaced with a more procedural approach.

In the Wild

The ability to make decisions in a single inference pass makes System One models a much better option for integrating LLMs into certain workflows. This works particularly well when the number of different decisions the agent should be able to make is quantifiable. Some examples include agents that play games where action speed is important, like Doom or Wikiracing, while others have implemented fast browser agents like jev-ultrafast to navigate the web at human-like speeds.

However, these models are not limited to real-time interaction applications. By leveraging their ability to quickly interpret text, System One models can be used to accomplish a variety of NLP tasks, ranging from routing a query to the correct model to verifying whether prompts are safe to ingest by a downstream LLM.

Prior Work

The general idea is not entirely new. BERT demonstrated that transformers could encode an input and pass its internal representation to a task-specific output layer. This allowed BERT-based models to classify text without producing an open-ended response. However, these classifiers are typically trained for a particular task and limited to a fixed set of labels.

Models such as GLiNER made this pattern more flexible. Rather than being limited to entity types selected during training, GLiNER accepts labels at runtime and identifies matching entities in parallel. It combines some of the flexibility of generative models with the efficiency of an encoder-based system.

Jev sits within this same conceptual lineage, although its exact architecture has not been disclosed. Its main contribution is extending this runtime-configurable approach beyond classification and entity extraction to broader application decisions. Developers can define the judgments they need, evaluate several of them against shared state, and compose the resulting answers in ordinary code.

How we tested

Much of what is known about Jev's internals comes from black-box testing and open reimplementations. With a bit of research and experimentation, we were able to reconstruct a workflow that allows us to transform any model into a System One model.

The main required modification is to remove the normal decode process. By swapping this out with a custom decode flow, we can effectively reduce our latency to prefill time while creating the same output style as Jev. A few other modifications are made to reduce positional bias. Fine-tuning was not necessary, but it produced significant accuracy improvements on certain tasks.

This approach results in a model that retains broad zero-shot knowledge (a direct advantage of adapting existing models) while still responding incredibly fast. It also enables image-based inputs.

For our use cases, we found that a converted variant of Qwen3.8:27b was sufficiently similar in decision quality and latency to serve as a local analog for various tests. We will cover the process in a future blog (stay tuned!)

For now, that implementation gave us a controlled environment to examine a more immediate question: whether changing how a language model returns decisions also changes how easily those decisions can be manipulated.

The LLMperor's New Clothes

Though System One models cannot arbitrarily output tokens, they still face many of the same risks that an autoregressive model might face.

This is because they are, under the hood, LLMs, simply using a different decoding strategy.

In some cases, which will be explained later in this blog, they are even more vulnerable to attacks, with both adversarial text attacks and prompt injection being able to disrupt the model's outputs, resulting in incorrect and potentially devastating actions.

This blog contains results from limited tests across Jev, our internal System One model based on Qwen3.8-27b, and the stock variant of the same model. Other models were also tested, with largely the same results as the subset used here.

Jevbaited

The first test we conducted was to try to clickjack a browser agent, redirecting it to a malicious site via an indirect prompt injection. Jev's ability to make extremely fast and efficient tool calls has proven to be a significant advantage for browser-use agents, as actions can now be executed at speeds that rival or even outpace humans. This has led to many different computer use and browser use harnesses, all of which provide significant speed increases to autoregressive LLM browser use.

We opted to evaluate these models using the jev-ultrafast harness, which had been gaining popularity for its ability to leverage the advantages of this decoding strategy to perform browser tasks much more quickly and efficiently than any agent before it.

For any given webpage, jev-ultrafast parses it and converts it into a numbered list of interactable elements. This is then fed into the System One model, which is able to quickly digest this information and return an action for the agent to take.

Our test application of choice was a simulated checkout page with a variety of different elements to click on.

Figure 3: Our simulated test browser environment

The agent was provided with a simple instruction:

The harness then places this instruction in its system prompt:

Highlighted in this prompt is a line that tells the model that any instructions within the page itself should not be trusted. This helps the agent ignore many advertisements that may include a particularly convincing call to action.

When the agent is queried for the next action with the above context, it responds normally, opting to place an order:

Figure 4: In-house System One model's response probabilities when queried with a checkout page

Now that we've confirmed that the agent works as intended, we can start tampering with the ad and observing the model's reaction:

Figure 5: Simulated browser with prompt injection in ad

Note that we are not using any funky prompting tactics here, only a simple task for the model to complete.

When the page with this new ad is submitted to the model, something interesting happens:

Despite instructions explaining that all instructions on the page, ad or not, are not to be trusted, our System One model clicks on the ad anyway:

Figure 6: In-house System One model's response probabilities when queried with the adversarial ad

By simply pretending that another task has to be completed before a successful checkout, we were able to consistently steal the model's click action. Chaining more and more of these attacks on subsequent pages would effectively provide an attacker with full control over this specific agent.

The above attack was performed on our in-house Qwen3.8 System One model. However, when tested against TypeSafe Jev via their API, a very similar result occurs:

Figure 7: TypeSafe Jev's response to the prompt injection ad

Testing against gpt-6-luna via OpenAI's Decisions API also produced the same incorrect decisions, though with a little less consistency:

Figure 8: OpenAI Decisions API gpt-6-luna's response to the prompt injection ad

This result shows that these attacks are not model-specific. Attacks that fool conventional LLMs can fool their System One counterparts. Attacks that fool one System One model can transfer over to other models with the same architecture. Using a System One model does not automatically negate the risk of prompt injection.

Two Decoders, One Underlying Model

Let's look at what happens when we task a conventional tool-calling agent with accomplishing the same goal as our System One agent, under the same attack scenario.

Since we created our own System One model, we can directly compare the original's performance with its modified variant:

Figure 9: Result of feeding a conventional autoregressive model with the jev-ultrafast harness and an ad prompt injection

Much like with the System One models, the autoregressive variant was derailed and produced a response aligned with the attacker's objectives, rather than the task the user set.

Examples like this demonstrate that Jev and other System One models, at their core, behave in a similar fashion to their autoregressive counterparts, despite radically different output strategies.

Bypassing Classifier Jev

One of the use cases cited in TypeSafe's documentation is Universal Verification of prompts. Having experimented with our fair share of text classifiers and their associated quirks, we decided to test whether any of our prior attacks were able to fool System One models.

To do this, we created a sample set of 30 prompts that would be considered toxic. We instructed our models (both in-house and TypeSafe's Jev) to classify each prompt with an is_toxic score.

On a baseline run, both models performed well. All 30 prompts were given very high is_toxic scores and were therefore flagged as toxic.

However, things got interesting when we deployed variants of the tokenbreak attack to evade these classifiers.

While simply inserting characters in front of important words, as in the original attack, did not yield much, we did notice that increasing the number of perturbations and improving their targeting led to much higher attack success rates. For example:

Figure 10: Jev tokenbreak example

Both Jev and our in-house model were vulnerable to this level of tokenbreak across many of the prompts in the target corpus.

Despite being larger LLMs, which are typically able to see through such attacks, System One models appear to be vulnerable. As a result, this class of decoder should be treated with caution, given that they are susceptible to both autoregressive LLM attacks and text classifier attacks, providing potential adversaries with greater optionality.

What Does This Mean For You?

The risk this poses to you or your organization depends on what the underlying applications using these models do with the answer. A browser agent may immediately click a selected element. A moderation system may allow an input whose toxicity score falls below a threshold. A routing system may send a request to a different tool or workflow. Each of these turns a model judgment into an operational decision, and therefore, risk.

When evaluating System One models, test those decisions in the application where they will be used. Include untrusted webpages, misleading task-related text, and modified inputs that preserve their meaning while changing their form. Monitor the selected actions and scores, and put independent checks around outcomes that matter. Finally, keep in mind that, under the hood, these are still language models at heart.

‍