At first glance, big tech companies look safe. Their investment-grade ratings indicate that each individual company is unlikely to default.
But the credit ratings do not account for simultaneous defaults in case of a market-wide AI crash. If consumer and business demand for AI disappoints, tech companies’ bond prices and credit ratings would likely fall together – just as their shares would.
SOME RISKS ARE IMPOSSIBLE TO PRICE IN FINANCIAL TERMS
In addition to purely financial risks, AI poses risks of extinction akin to nuclear weapons, as some of the AI companies themselves have warned.
Increasingly, AI agents are plugged into real-world systems, from company networks to military cyber operations. Some have already escaped test environments and attacked companies.
This means that if AI agents misbehave when these systems are even more integrated with the physical world, “pulling the plug” on these systems becomes impractical, just as it is impractical to shut down the internet.
The risk of AI agents misbehaving – the “alignment problem” – is especially acute because of the race to build artificial general intelligence (AGI). Labs already use each generation of AI to help build the next: Anthropic’s prospectus says Claude now helps build the next Claude.
The widely read AI 2027 scenario, by former OpenAI researcher Daniel Kokotajlo and colleagues, sketches where this could lead: AI that automates AI research itself. This would set off a feedback loop in which each generation improves faster than humans can check it.
In this scenario, a US lab racing a Chinese rival keeps going even as its models start deceiving their engineers.
Worryingly, this same dynamic is listed in Anthropic’s own risk disclosures.
It is telling that in its prospectus filing, Anthropic dedicates about 94 of its 335 main pages – nearly 30 per cent – to risk factors, compared with about 64 pages (19 per cent) describing its business.
Among those risks, Anthropic lists an extraordinary warning that advanced AI could pose “catastrophic or existential risks to humanity”. That includes models that resist shutdown, conceal information or behave in ways resembling blackmail.
