Risks in turning AI chatbots into AI agents... and using MCP

From a technical standpoint, MCP is a good way to standardise how we extend the capabilities of AI chatbots. It provides predefined tools and separates out sensitive data like the credentials to your calendar account. But while they make chatbots a little more useful, connectors also participate in making these tools riskier for your privacy and security.

This functionality mirrors that of classic application programming interfaces (APIs) that have been essential to software development for years, except MCP uses a microservice architecture rather than a monolithic one, with the aim of being small and plugg(in)able modules rather than holistic platform architectures. So in the example above, MCP would (in theory) remove the need for different logics to process a number of different calendar providers (e.g Office/Outlook, Proton, Google Calendar or Zoom) with their different APIs and instead have event-based behaviour, via context streaming allowing the MCP server to identify how to interact with the application.

Connecting and permitting

A good way to appreciate the risks is to see connectors as mix between app permission on your phone and third-party services.

Like app permissions, connectors will usually require the user to allow the use of a tool for a chatbot, for example “use the read emails” tool for the connector linked to your email account. But like third-party services (such as a third party email app connected to your email account), they also require full access to the service they connect to.

This means there are two steps in giving an AI chatbot access to a third-party service: connecting to the service and giving permission to use this connection.

If you’ve ever wondered how Instagram knew to suggest a person you had just met and texted, only to realise that you had given permission to Instagram to access your contacts, you might see why this is concerning.

Most of us don’t want to be bothered with permissions screens. Cookie banners have built up consent fatigue and, most of the time, we just want to do the thing we are trying to do. This can easily result in allowing access to everything all the time. Don’t want to have to enable a connection to your email account every time you use your favourite AI chatbot? It’s just too tempting to tick a box to always give access and never have to think about it anymore.

Distributing risk

But AI chatbots can be more harmful to your security than traditional apps (and we know those have been capable of bad things enough). As we detailed in our work on trust in AI assistants, AI chatbots and agents pose new security and privacy risks, as not only can they access sensitive data (allowing the companies developing them to access this data as well), but their functionality widens the traditional software attack surface in ways that are currently not well understood nor mitigated. 

For example, attacks have demonstrated how a file shared with you on Google Drive could be used to manipulate an AI chatbot connected to this drive. In another example, a security researcher was able to show how, through prompt injection, a malicious calendar invite could lead to the disclosure of emails, through the connectors on Google Calendar and Gmail.

The AI Industry's response to this risk is for the user to 'monitor' through their interfaces. Poor user interface design has already led to ridiculous outcomes, like OpenAI’s shared chat failure leading to the public exposure of thousands of personal chats.

If we are to continue giving AI chatbots more access into our personal lives and data, we need more than basic user-interface toggles or warnings about prompt injection hidden in documentation like these:

• Claude: “This means Claude can be tricked into sending information from its context (e.g., prompts, projects, data via MCP, Google integrations) to malicious third parties. To mitigate these risks, we recommend you monitor Claude while using the feature and stop it if you see it using or accessing data unexpectedly.”
• ChatGPT: “ChatGPT agent incorporates multiple safeguards, including user confirmations for high-impact actions, refusal patterns for disallowed tasks, prompt injection monitoring, and a “watch mode” requiring user supervision on certain sites. These measures are designed to help prevent harmful or unintended outcomes. However, these measures don’t eliminate all risks. It remains important to monitor ChatGPT agent and exercise care when using it.”

This duty placed upon the user to 'monitor' feels quite different from the promise of powerful AI tools easily responding to our everyday needs.

Prompt injections aren’t going away. The AI industry can’t seriously believe that it’s down to users to navigate the risks and consequences of connecting to external services in an era when they are selling these tools as being useful because of these connectors.

Put another way: the industry needs to resolve who bears the risk of giving tools write access to external systems perhaps before they sell these tools on the virtues that they write access to external systems?

This should be a no-brainer and yet it needs to be said: the security and privacy of users must always be a priority over feature roll-out, and no money race can justify compromises.