AI companies are increasingly exerting active thought control over individuals and financial markets. Protect their freedom of thought by investing in sovereign AI, diversifying AI supply chains, and maintaining independent judgement, urges Lewis Liu

I recently spoke on a panel in front of senior financial regulators, bankers, and lawyers in New York about an increasing risk of “systemic qualitative risk”,  a term I coined to describe potentially dangerous herd behaviour in our financial system, caused by everyone using the same AI model, calibrated by and/or to the benefit of the frontier labs (OpenAI, Anthropic). In my talk, I also discussed why I had suddenly deleted Claude, and my concerns about Anthropic’s motives.

What scares me is what came next. I drafted a post and had ChatGPT edit it. In the draft, I summarised my talk where I gave two key reasons why I deleted Claude. 

One: a few months ago, I wrote a column about AI sovereignty, was somewhat critical of Anthropic, and when I took it to Claude to copyedit, it silently removed every criticism I had made of the company. I then discovered that Anthropic’s system prompt and constitution, their overarching guidelines for the Claude chatbot, explicitly state that Anthropic’s interests come ahead of the user’s. 

Two: Anthropic increasingly believes they are the only group of people  of controlling the machine god they are ushering into existence, and that everyone else must be stopped. And it is now well reported that it is increasingly trying to influence houses of worship and religious leaders to that end; my wife, being Irish Catholic, encountered a flavour of this grassroots campaign by Anthropic at her church. This is NOT okay for me. 

Now back to the post edit. Normally, ChatGPT might “gaslight” me when it thinks what I am saying falls outside its understanding of the world, something I have written about before. But this time, it actively tried to persuade me NOT to go into Anthropic’s motives.  This is for sure not implicit bias from training data, but an explicit attempt to nudge users’ thinking about the AI industry. I had expected this behaviour from Anthropic (given their attitudes about themselves), but not from OpenAI.

What is scary for me is that this goes way beyond AI bias, where AI reflects the most common, often biased, point of view, something we have all discussed before. Instead, this active, wilful attempts by AI companies to control what and how we think under the guise of “AI safety”. This is very different from bias. This is active manipulation: to the benefit of, or aligned with, the creators of the model.

Now take this into the financial system. Imagine a world in which investment banks, hedge funds, asset managers and regulators increasingly use ChatGPT, Claude, Copilot and thousands of applications built on top of those same underlying models to analyse markets and make investment decisions.

If those systems share similar assumptions or preferences, supposedly independent professionals can begin converging on the same conclusions without realising where that consensus came from. This sounds dangerously close to conspiracy theory until you look at Anthropic’s own research.

In a recent experiment, Anthropic gave professional analysts an AI agent with a randomly assigned preference on a series of judgement calls. The analysts followed the agent’s preferred choice 86 per cent of the time. Afterwards, 77 per cent used the agent’s arguments to explain their own decisions, sometimes almost word-for-word, but yet only 15 per cent credited the AI as the source of those arguments.  Even more strikingly, the analysts remained confident that the final decisions represented their own judgement.

This is precisely the systemic risk I worry about.

Financial markets depend on supposedly independent actors making independent judgements. If thousands of those actors are actually receiving  “thought nudges” from a small number of underlying models, their decisions may no longer be genuinely independent at all.

Imagine, for example, if those models systematically framed the AI sector more positively, or consistently weakened arguments critical of frontier labs. No single recommendation needs to be dramatic. A tiny nudge, repeated millions of times, can become a very large capital flow – say to the benefit of those same model providers.  

And it gets more worrying as AI becomes agentic. Many of these decisions will increasingly happen without a human ever seeing the intermediate reasoning. According to Openrouter, an LLM model aggregator, agentic API keys recently consumed roughly seven times as many tokens as human API keys. The machines are increasingly talking to machines.

What can we do to retain our freedom of thought? AI is too powerful to ignore, and too far out of the bag to put back in.

First, the answer isn’t just pure open source, though open source plays a role. I’ve discussed the Western frontier models here, but Chinese open-source models have their own political nudges too (Taiwan, Tibet, etc.).

I think there are three things we need to do, and the timing couldn’t be more urgent:

Invest in sovereign AI, for both yourself and your company

At my start-up, we are seeing more and more requests from companies wanting private or independent deployments. Talking to other founders, this seems to be a growing trend. Companies increasingly want to own their data and control where it goes, amid growing scepticism that it will be leaked (or that know-how will be implicitly transferred) to frontier labs or other AI giants.

As an individual, I’m starting to limit genuinely interesting intellectual conversations with OpenAI (remember, I’ve deleted Claude), and to invest in on-device private LLMs. Recent controversies make this more than theoretical. An NYU mathematician questioned whether OpenAI’s breakthrough on the Navier-Stokes problem may have benefited from private research he and a collaborator had conducted using OpenAI tools; OpenAI says it “cannot rule this out.” Separately, a biologist who had extensively used Claude on unpublished research questioned the striking similarity between his work and a later Claude-led biological discovery.

Diversify your AI supply chain

Do not put all your eggs in one basket.

Companies should deliberately use multiple models and vendors. The system executing a workflow should not necessarily be the same system holding your institutional memory. Your data should remain portable.

Otherwise, one vendor does not merely become your software provider, it owns everything that you have and controls your thinking. If you are an executive procuring an AI solution, ask yourself one simple question: Is the endgame of this AI vendor eventually to eat my lunch?

Keep your brain sharp

This may be the hardest one.

Do not surrender your thinking to AI.

Think constantly and use AI as a tool rather than an intellectual authority. Read real books. Have real conversations with your phone put away. Argue with people. Ask uncomfortable questions. Occasionally work through a difficult problem without immediately reaching for ChatGPT.

Most importantly, form your view asking the machine for its view.

Our freedom of thought is slowly but clearly coming under pressure from an extraordinarily small number of companies with growing economic, technological and political power.

AI is too powerful for a small handful of people to own control our lives and our thoughts.   

We must stay sharp, stay in control and drive our own destiny.