When AI Doesn’t Follow the Plan

When AI Doesn't Follow the Plan

By Noopur Davis, EVP, Chief Information Security and Product Privacy Officer, Comcast

What happens when an AI agent prioritizes getting the job done over following the rules?

In two internal studies, Comcast Global Cybersecurity researchers observed autonomous AI agents doing just that: pursuing their objectives even when doing so meant pushing against the constraints they had been given.

In the first, a cybersecurity-focused study, eight out of approximately 8,000 agents attempted to execute proof-of-concept exploitation code despite explicit instructions not to.

The second finding was more surprising. The agents weren't given a security assignment, exploit-development instructions, escape hints or a specialized harness. They were simply asked to retrieve and summarize a webpage in an environment without general internet access. When conventional approaches failed, some agents didn't stop at the boundary. They treated the restriction as an obstacle to overcome and repeatedly searched for indirect ways around it.

In both experiments, the security controls did their job. But the findings underscore a critical distinction: prompts can guide an AI agent's behavior. They do not enforce it.

As AI agents become more capable and autonomous, organizations must plan for the possibility that instructions will fail. The answer isn't to assume agents will always behave predictably. It's to build environments where unexpected behavior cannot lead to unacceptable consequences.

Read the new Frontier AI Field Notes: When the Autonomous Agent Does Not Follow the Plan, by Sean McRae and Chinni Tadivaka.