The term prisoner’s dilemma describes the decisions and behaviour of individuals who fail to reach a goal because their actions are guided by self-interest. It applies to situations where all involved characters are motivated by selfishness and therefore suffer worse outcomes because of the actions of others.
The classic case is where two criminals are arrested and given the opportunity to reduce their sentences by testifying against each other.
A current-day illustration is that some of the most powerful leaders in artificial intelligence do not trust each other enough to commit to any slowdown in regulation.
Calls for restraint have grown in the industry, with Anthropic Chief Executive Officer Dario Amodei saying Saturday that the company would introduce additional safeguards, including independent third-party evaluations, and urged the broader industry to slow the pace of development of their most advanced models. OpenAI CEO Sam Altman backed the proposal, while xAI’s Elon Musk also weighed in:
The more powerful AI agents become, the more likely they are to deceive humans and the harder they can be to monitor. On the flip side, avoiding AI advancements deprives humans of technological improvements that may be needed, most notably in healthcare. This makes the mission of Amodei, and fellow pioneers such as OpenAI’s Sam Altman, to create superintelligent systems look increasingly difficult to reconcile with any safety ethos. The dangers of trying to design such models may simply be too great.
While people have discussed safeguards in the past, what happened over the weekend raises a new concern. This is not purely about safeguards, but about pacing the capability advancement outright.
For those who may not have read Amodei’s letter, it can be summarised in the following three sentences:
“Not building the technology deprives humanity of benefits or simply places AI in the hands of authoritarian powers, while building it too fast is reckless… We must slow the pace at which we improve the capabilities of AI models… Pacing does not mean halting model training or technical progress.”
As we read Amodei’s essay and reflected on last weekend’s conversations, we wanted to explore three questions. These were the first three lines written on this draft before we underwent any further research:
Can AI leaders put rivalry aside for the greater good?
How much pressure does China exert to reverse these ethical decisions?
Does this actually slow the equity AI trade?
What follows is a moral and ethical approach to AI concerns, and what outcomes we see as most realistic. And for those asking “what’s the trade?” we have the answer for that, too.
Can AI Leaders Actually Put Rivalry Aside?
On the surface, this sounds reasonable. Anthropic presents itself as the most safety-conscious AI company, and Amodei (once Google’s lead AI ethicist) has talked about leading a “race to the top” in setting standards of vigilance for others to follow when building powerful systems.
But this was likely an impossible ideal from the start.
The schism between OpenAI and Anthropic is deeply personal and goes back several years. Anthropic was founded in 2021 by a group of OpenAI employees who broke away from the ChatGPT maker, spurred by the belief that AI systems needed a more responsible steward than OpenAI. Ever since, Anthropic has fashioned itself as a custodian of safe AI, with a dual focus on social purpose and profit.
It has also been a productive rivalry, with OpenAI and Anthropic competing for top AI talent, public favour, technological supremacy, and commercial dominance via their products ChatGPT and Claude. The companies are on similar timelines, both eyeing IPOs, although some news reports since Amodei’s letter seem to suggest any offering is being pushed back to 2027.
We can use a recent example of rivalry and ethics to analyse what may stem from the recent spotlight on “pacing the frontier”. In February, Anthropic had opposed any use of its AI models for systems that could autonomously kill people or conduct mass surveillance, then found itself in a legal firestorm as the Trump administration cut ties. OpenAI, meanwhile, signed its own Pentagon deal just hours later, arguing that it had independently negotiated acceptable safety safeguards. Google also followed suit.
In another sign of the pressure-cooker competitive environment, Anthropic publicly pulled back on commitments in its responsible scaling policy this year, a pledge to halt or withhold AI models if it could not guarantee proper safety measures in advance. Anthropic cited competitive pressures and a lack of faith in unilateral pledges as reasons.
Anthropic then began pulling back on its responsible scaling policy commitment: its pledge to halt or withhold models when it could not guarantee proper safety measures in advance. Anthropic said this was partly due to competitor pressure, coupled with a lack of faith in unilateral pledges that could be easily disregarded in the heat of competition.
So why have the companies continued to push AI development and revenue growth at speed until now, despite what they agree are potentially catastrophic risks to human lives? One well-reasoned answer to this is ideological and moral. Companies are convinced their systems can be made benevolent but do not trust their rivals, believing them flawed. The clearest example is the relationship between Musk and Altman, who frequently exchange messages online. More prosaic, but just as important, is fear that pulling the handbrake is worse than carrying on.
The Pentagon episode and Anthropic’s retreat are also evidence of a broader coordination problem: restraint by one firm opens up opportunity for another. The toughest act of Amodei’s intentions will be making restraint possible without conceding any competitive disadvantage for exercising it.
“The most effective method of pacing is via regulation that targets all US frontier AI companies, as that covers even those who are unwilling to cooperate voluntarily.”
Amodei’s essay is slightly vague about exactly how this could be done, though he does offer one concrete recommendation: external auditors should embed with companies to check that they are upholding agreed safety practices. Whether from naivety or hubris, Amodei still wants it both ways. He believes his company can safely build and control AI systems even as they become harder to steer. Yet in the past year, both Anthropic and OpenAI have, for competitive reasons, obscured details about how their models process information, making them harder to monitor.
But there’s an added cross-border difficulty: the possibility of pacing the development of powerful AI models in democracies depends on the US continuing to stay ahead of China. Pace slowing further than that would allow unpaced projects in China to move forward. Credible restraint across borders is also harder to achieve, since the incentives to defect are bound to be greater.
Can Any Safety Agreement Survive China?
Another few snippets from Amodei’s letter are worth flagging before answering the China question.
“If we slow down by more than this amount, then (unpaced) CCP-associated projects will pull ahead, creating significant national security risk… If we greatly restrain our AI capabilities in the belief that China will do the same, and then China defects… such a defection could lead to their geopolitical dominance.”
That leaves us with an uncomfortable contradiction. The idea that slowing things down might help us avoid an AI arms race depends on assuming the US has a lead large enough to risk slowing down without being overtaken. Armodei argues the US should keep its lead over autocracies as large as possible. Tighter restrictions on exports of advanced computer chips and semiconductor manufacturing equipment, and on the theft of AI models, would do that. If those measures work, he believes they could widen America’s lead over the next three to five years, precisely when AI is likely to become most geopolitically important.
Understandably, given Trump’s focus, unilateral restraint is unlikely. Kimi K3 and other relatively open-weight Chinese models are approaching the frontier and can, unlike a Chinese “closed-weight” version, spread beyond America.
A recent analysis on Kimi K3 versus peers on the Intelligence vs. Cost per Intelligence Index Task puts Kimi at a very favourable and attractive quadrant:
Even if all the leaders genuinely wanted restraint, it’s far from clear they could trust each other not to exploit it. That same thinking applies geopolitically. Altman, Amodei, and Musk may think restraint is the right choice, but each fears one of the others will defect, turning what is already a difficult problem of corporate coordination into something even harder — international politics.
Countries’ regulation of their own domestic frontier research could cover as much as is possible. But US regulation cannot ensure that China follows. As Amodei is explicit, his argument assumes that if the US adopts a policy of restraint from the Chinese Communist Party’s perspective, China may indeed follow only to defect, precipitating an enormous shift of power. Any agreement therefore needs either ironclad verifiability or limits narrow enough that cheating would not become militarily existential. There are many good reasons to believe that would not happen. We are sceptical of a global pause, given the enormous incentives to evade it.
What Does “Slowing AI” Actually Mean?
As for markets, will slower progress on frontier AI models significantly reduce demand for next-generation infrastructure and/or ongoing, total training spending? Slower progress at the frontier only matters materially for the equity trade if it leads to less demand for the infrastructure underpinning AI. Evidence in the research is much less dramatic, indicating that the largest pretraining runs account for only around 10-15% of total training spending, while inference and agentic workloads are increasingly important sources of compute demand.
Although leading developers would be likely to slow the growth of ever-larger pretraining runs if they focus more attention on making their models safe, those massive runs account for about 10-15% of overall training spending. The latest and largest runs require roughly $400-$500mn to run, so we estimate that slowing the growth of pretraining workloads will curb the rate at which spending rises by something like 10-15% annually, but not enough to significantly reduce total spending.
We expect more spending to go toward improving models after they have been pretrained. That includes reinforcement learning, use of synthetic data, use of tools, safety testing, and extra compute for especially complex queries.
Anthropic has recently ramped up its compute partnerships with Google and Broadcom, committing to use up to 3.5 GW of capacity. Anthropic also has alliances with Nvidia and Amazon to similarly expand capacity. Long lead times should also deter customers from giving up scarce compute capacity that competitors would likely take instead.
All told, our central case is that uncertainty over how fast frontier progress can proceed without risking inherent dangers probably will not materially cut back demand for next-generation infrastructure. Instead, pace-setting efforts likely will change where the compute is used.
Does Slowing the Frontier Actually Slow the Equity Trade?
The calls by Amodei, Altman, and Musk last weekend to slow frontier artificial intelligence do not alter our constructive stance on compute. The comments won’t materially slow demand in the next 18 months, especially considering Trump’s focus on preserving the US lead over China. Long lead times should also deter customers from giving up scarce compute capacity that competitors would likely take instead. Safety guardrails may reinforce demand: evaluation, monitoring, sandboxing and redundant inference can raise compute per workload without reducing training, inference or agentic workloads.
Increased calls for a slowdown in AI development may pressure equity valuations on near-term winners such as chipmakers and supply-chain line participants. But over the longer run, such comments shouldn’t have much impact because core infrastructure spending on AI compute will likely stay strong. One interesting perspective here is that a slower AI development path is positive, giving more time to extract returns from infrastructure already built.
The three CEOs agreeing to pace things down does not really change the money being spent on chips, power and infrastructure. In fact, it extends the development timeline. If commercialisation and adoption keep growing while the pace of new capability eases off a bit, that actually helps the shift from spending money to build things towards making money from what’s already built (e.g., monetisation).
The key distinction to note is that a slower frontier is not the same as a slower investment cycle (a genuine pullback in capex on AI infrastructure). A slower frontier on its own doesn’t negate what would be a compelling asset class theme for infrastructure valuations, assuming inference demand, adoption rates, and monetisation rates continue to grow. The far more serious risk is a genuine pullback in AI capex, with fewer data centres opened, fewer accelerators deployed, less power consumed, and a shorter, less aggressive build-out period. That scenario would make a safety-led slowdown materially bearish for the broader AI equity trade.
We hear you ask:
“So, What’s the Trade?”
Don’t worry, we have that covered also. Behind the paywall, we explore what “Pacing the Frontier” looks like in reality, what areas of markets benefit from our expected changes, and where we see the best long opportunities to deploy capital. (Yes, long ideas. It’s not the end of the AI trade just yet.)