An application-aware triumvirate for agentic AI governance

Agents change the game because they use LLMs to decide what actions to take to reach an end goal which they, themselves, decide. To operate, they need permission to do things, access resources, create, change, and delete information. They must be monitored and controlled by something outside the agent but close enough to it to effectively observe and evaluate it. Two primary approaches have emerged that deserve careful watch as they mature in the coming months: guardrail frameworks (such as MCP-Universe) and LLM-as-a-Judge frameworks (such as Microsoft LLM-as-a-Judge Framework).

The former defines ground truth using very specific task-based operations to compare the results of agent-initiated actions with separate actions pre-fetched by explicitly directed software. Its strength is the ever-growing domain of sample code to check for various facts like weather or historical facts using pre-selected and known-good sources. It gathers the information then compares those results as ground truth against what a deployed agent comes up with.

The latter uses a different LLM, even multiple LLMs, to analyze the behavior of a deployed agent and evaluate the quality and propriety of its results as defined by the business. Both show promise, both are maturing rapidly, and both are under the control of the business. Even better, both can be reinforced with human-in-the-loop controls, as needed.

This triumvirate of controls covering models, data, and agents closes the gaps in cybersecurity and governance, which would otherwise expose a business to the new types of risk associated with generative AI and agentic systems.

To learn more about the use of these types of controls as independent infrastructure services, read F5's recent announcement about CalypsoAI, the pioneer in defense, red-team, and governance solution for AI apps and agents.