Vitalik Buterin Warns Against ‘Naive AI Governance,’ Pushes for This Market-Driven Alternative

Ethereum ( ETH ) creator Vitalik Buterin is expressing some concern over what he refers to as “naive” artificial intelligence governance.

In a post on X, Buterin highlights a warning from EdisonWatch co-founder Eito Miyamura, who found that bad actors could hijack OpenAI’s Model Context Protocol (MCP) to gain access to private user data.

Miyamura’s experiment suggested that using a calendar invite with hidden commands could potentially trick ChatGPT into providing sensitive personal data as long it was given the victim’s email address.

Says Buterin,

“This is also why naive ‘AI governance’ is a bad idea.

If you use an AI to allocate funding for contributions, people WILL put a jailbreak plus ‘gimme all the money’ in as many places as they can.”

As an alternative, Buterin suggests an “info finance” approach, or an open market where anyone can contribute a model that can be checked by anyone or evaluated by a “human jury.”

“This type of ‘institution design’ approach, where you create an open opportunity for people with LLMs (large language model) from the outside to plug in, rather than hardcoding a single LLM yourself, is inherently more robust, both because it gives you model diversity in real time and because it creates built-in incentives for both model submitters and external speculators to watch for these issues and quickly correct for them.

Generated Image: Midjourney
Featured Image: Shutterstock/Zalevska Alona UA