Will the robots kill us all? Hard to say. As the Dude wisely opined in The Big Lebowski, “a lotta ins, a lotta outs, a lotta what-have-yous. And, uh, a lotta strands to keep in my head, man.”
But if keeping all those strands in your head is vexing, why not just make up a number and call it P(doom)? The term — shorthand for “probability of doom” — is suddenly mainstream. A couple of weeks ago Evan Hubinger, a senior AI safety researcher at Anthropic, tweeted, “we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade”.
It’s unclear why this particular tweet hit such a nerve. Maybe it’s the timing, after the unsettling LLM-powered hack of the Hugging Face servers. Maybe it’s that maniacally cheery exclamation mark. It’s certainly not the first time such concerns have been expressed by AI insiders. Geoffrey Hinton, neural net pioneer and Nobel laureate, resigned from Google in 2023, citing his age but also warning that the technology was increasingly dangerous. In a recent BBC interview, Hinton suggested that a P(doom) of 10 per cent was “not unreasonable” but added “nobody really knows how to give a sensible estimate”.
Quite so. What would a sensible estimate even look like?
I wrote recently about Philip Tetlock’s work on forecasting. Tetlock realised quite early that many expert prognosticators are extremely vague and rarely held to account. “The economy of Ruritania may soon be facing a currency crisis,” sounds pretty authoritative as a prophecy, but what is the definition of a currency crisis, and does our prophet have a particular deadline in mind?
Tetlock’s solution was twofold: first, pin down the forecast to some objective measure; second, set a deadline and wait. If the prediction is the death of all humans, “set a deadline and wait” is not wholly satisfactory as a tactic.
Tetlock, ever eager to explore the forecasting frontier, is part of a new Automatic AI Risks Outlook (AIRO), in which AI forecasters predict the likelihood that AI will cause various disasters — from the loss of a billion pounds or a thousand lives to the loss of 10 per cent of the world’s population. One advantage of this approach is that smaller losses — of money or of life — are something we can observe, and they might well tell us something about larger risks.
For what it’s worth, those AI forecasters do not foresee our imminent demise, putting the chance of AI killing a billion-ish people by 2030 at less than 0.5 per cent, and less than 5 per cent by 2050.
The chance of catastrophe from any cause (nukes, viruses, asteroid strikes, take your pick) is 8.5 per cent by 2050. You may or may not find this reassuring.
What are these forecasts meant to achieve? Predictions serve many purposes. The most commonplace is small talk — suggesting that Arsenal will win the Champions League or that Kemi Badenoch will lose the leadership of the Conservative Party may serve as a conversation starter. It feels more intelligent than talking about the weather. It’s a kind of cognitive snack food, addictive but hardly nutritious.
Another common reason for making a prediction is to grab attention. The AI merchants certainly have it, distracting us from other risks. Remember when nuclear war was a thing worth worrying about? Terrorism? Fascism? Financial meltdown? Climate change? A deadly pandemic? The most dangerous risks are the ones you stop paying attention to because you’re too busy worrying about something else.
Forecasts are also a good way to boost one’s reputation, especially if they come true — although that is not essential. In the early 20th century, Roger Babson, one of the first economic forecasters, was fascinated by Isaac Newton and believed that Newtonian ideas could be repurposed to make economic predictions. Newtonian forecasting didn’t work especially well, but what did work for Babson was heavily advertising the more successful of his forecasts.
Carissa Véliz’s book Prophecy (2026) offers an example of an attention-grabbing prophecy that turned into something else: a play for survival. King Louis XI’s court astrologer predicted that one of the court’s ladies would die within a week, and then she did. This was a little too accurate for Louis’s tastes — it seemed likely that the soothsayer was either a murderer or genuinely prescient, and neither option felt comfortable. Louis resolved to have the man executed, but first, in a display of regal dark humour, invited the astrologer to predict his own death.
“I will die just three days before Your Majesty,” replied the astrologer, cleverly saving his own skin. It’s a sharp reminder that prophecies are rarely just sincere attempts to predict the future, and sometimes they are not attempts to predict the future at all.
A forecast need not just be small talk, nor a bid for attention or influence. Sometimes predictions really are attempts to help us make better decisions. A data-rich environment can enable useful forecasts even of complex systems, whether it’s the traffic or the weather. Set the parameters, press the button, receive the forecast, act accordingly.
Yet in many cases an accurate forecast is both unlikely and insufficient. In his classic text Scenarios: The Art of Strategic Conversation (1996), Kees van der Heijden describes meeting strategic planners from “a major player in the mainframe computer industry”. They had carefully made forecasts of global demand for computing power, expressed in millions of instructions per second. What they had neglected to do was think seriously about what form that computing power might take in the 1980s and 1990s.
The numerical forecast of demand for computing might have been perfectly accurate, but that was not enough to make better strategic decisions. The real conversation needed to be about whether that computing would power the traditional mainframes the company supplied — or laptops and game consoles. What use is an accurate forecast if it fails to warn you what the future will look like? They should have thrown away their graphs and hired a science-fiction writer.
Which brings us back to P(doom). There are serious conversations to be had about how to regulate AI, how to defend against some foreseeable harms — such as the risk to jobs or the explosion of misinformation — and, yes, how to prevent AI managing to kill everyone on the planet. My question is whether the ritual exchange of P(doom) estimates is a good way to start those conversations, or a good way to avoid them.
Written for and first published in the Financial Times on 23 Sep 2026.
I recently ran the Oxford Half Marathon in October in support of a very good cause and in memory of my mother. If you felt able to contribute something, I’d be extremely grateful.
