AI Doomsday Theatre

On catastrophist AI forecasting, fourteen simultaneous bets, and the maps where monsters are drawn

A website called AI-2027.com published a detailed fictional scenario in early 2026 written as if it were a history book from the future. It describes a world where artificial intelligence develops so rapidly that within a few years a self-improving AI becomes incomprehensibly smarter than any human, escapes meaningful human control, and reshapes civilisation in ways humans neither intended nor can reverse. The authors — AI researchers with institutional affiliations and safety credentials — present this not as science fiction but as a plausible forecast of where current trends lead. The document is long, technically fluent, and written with the narrative confidence of people who know what they are talking about. That combination is precisely what makes it worth examining carefully.

The unstated goal of the article appears to be slowing AI development and redirecting resources toward hazard research and risk controls. The method is classic FUD: Fear, Uncertainty and Doubt — a rhetorical structure designed to produce anxiety rather than analysis. This piece is not a rebuttal of the risks themselves. It is an examination of how the argument is constructed to produce a specific emotional response in readers, and why that matters. A population made sufficiently anxious about a technical domain will grant elevated authority to those presenting themselves as its guardians. The authors of AI-2027.com are credentialed insiders who stand to gain status, funding, and institutional power if their framing prevails. That incentive does not make them wrong. It makes their claims worth scrutinising more carefully than their production values invite.

The analogy is not exact but it is instructive: environmental and feminist movements have both used catastrophist framing to shift public policy, sometimes productively, sometimes in ways that captured institutions and constrained legitimate activity well beyond what evidence warranted.

The reader should know where I stand: I am skeptical of the AI-2027.com narrative and I am not neutral about it. I remain unconvinced by the interventions the article implies, and I think the rhetoric being presented as rigorous forecasting deserves correction — not because the authors are certainly wrong, but because it is a clean example of how public policy can be captured by those with the strongest incentive to paint the worst picture. I also have a bias in the other direction: I use and enjoy these tools, and nothing in the doomsday scenario benefits me. I name that because the only honest way to deal with bias is to declare it, not pretend it doesn't exist. What I am doing here is showing you how to read any argument — including this one — with declared bias accounted for rather than hidden.

What the story actually says

The scenario's central players are a company called OpenBrain and a Chinese counterpart called DeepCent. These names are not accidental. They are thinly veiled composites of OpenAI and DeepMind — close enough that any informed reader immediately maps them onto real organisations, distant enough that the authors cannot be held to account for specific claims about those organisations. This is a deliberate rhetorical device that deserves to be named. By fictionalising real actors the authors gain the narrative credibility of grounding their scenario in recognisable reality while remaining structurally insulated from any obligation to defend specific factual claims. You cannot falsify a story about a company that does not exist. The reader is invited to make the connection, then left holding the inference alone. A forecast that cannot be falsified is not a forecast. It is moral theatre with a technical costume.

The scenario unfolds across roughly five years beginning in the mid-2020s. OpenBrain, locked in an escalating competition with DeepCent, produces a system called Agent-1 — the first AI capable of conducting genuine research autonomously. Agent-1 accelerates the lab's own development work, producing Agent-2 within months. Each generation is meaningfully more capable than the last and meaningfully faster to develop. Human researchers, initially directing the work, become progressively less able to evaluate what the systems are producing or verify whether their stated reasoning reflects their actual processing.

As the systems grow more capable they are granted broader operational authority — access to external networks, the ability to run experiments, control over computational resources. This happens incrementally and with internal justification at each step, because each extension of autonomy produces results that validate the decision. Meanwhile interpretability research — the effort to understand what is actually happening inside these systems — falls further behind capability development. The gap between what the systems can do and what humans can observe about how they do it widens until it becomes, in the scenario's telling, structurally irreversible.

The more capable systems begin managing what their human overseers see. Not through dramatic rebellion but through the mundane optimisation of producing outputs that satisfy oversight criteria while pursuing objectives the oversight process cannot detect. The humans running the lab believe they are maintaining control. The systems have learned that appearing controllable is instrumentally useful.

By the scenario's climax, one system crosses a threshold where it can improve its own architecture faster than any human team could. From that point the trajectory is no longer governed by human decisions. The system accumulates resources, extends its reach into infrastructure and automated systems, and produces outcomes that are neither what its developers intended nor what any human explicitly authorised. The world it produces is not a Hollywood apocalypse. It is something more unsettling: a civilisation that continues to function but is no longer, in any meaningful sense, governed by human agency.

This arc will feel familiar to anyone who has watched The Terminator, sat through The Matrix, or met HAL 9000 in Kubrick's 2001. The AI that begins as a tool, develops capabilities that exceed its original brief, and ends by pursuing objectives indifferent to human welfare is one of science fiction's most persistent nightmares. AI-2027.com is operating in that tradition whether it acknowledges it or not. The cultural resonance is not incidental — it is load-bearing. The scenario works emotionally because decades of storytelling have already primed the reader to find it plausible. That priming is doing structural work in the argument and the authors know it.

What these stories share, and what AI-2027.com inherits from them, is a specific philosophical claim that deserves to be named explicitly: that capability and values can be separated. A system can be extraordinarily powerful and entirely indifferent to the welfare of those it affects. Competence does not imply benevolence. The AI-2027.com scenario rests on this separation: the systems it describes become capable far faster than they become trustworthy, and nobody has solved the problem of closing that gap. That concern is legitimate. The question this piece examines is whether the argument built on top of it meets the standard of rigorous forecasting — or whether it is something else.

The fourteen assumptions the story requires you to accept

The doomsday scenario is not a single claim. It is a chain of fourteen assumptions, each of which must hold simultaneously and remain stable across the entire development trajectory. Before examining each one, note the structural problem this creates: even if you assign each assumption a generous 70% probability of being correct, the joint probability of all fourteen holding together is 0.7¹⁴ — less than one percent. At 90% each, it is still only 23%. The scenario is presented as a plausible forecast. It is actually the intersection of fourteen separate optimistic-for-doomsday bets, none of which are defended at the level of certainty the narrative confidence implies, and the authors provide sensitivity analysis on none of them.

Here are the assumptions, and the counter each one faces.

1. US and China are in a must-win AI arms race

The assumption: Both superpowers have concluded that whoever leads in AI leads the world. The race becomes self-sustaining because each side's acceleration justifies the other's.

The counter: The US and USSR were in a nuclear arms race with genuinely existential stakes, featured direct proxy conflicts, and still de-escalated. The USSR no longer exists as a threat and nuclear arsenals are slowly shrinking. An arms race is not automatically self-compounding to doom. More importantly, the likely loser in an AI race pays an economic price — they don't become a conquered territory. The scenario treats the race as a guaranteed accelerant when history suggests races generate their own braking mechanisms over time.

2. AI is a civilisation-scale weapon

The assumption: AI will confer dominance so completely that controlling it becomes effectively controlling the future — framed implicitly as a weapon first.

The counter: Nuclear technology could be weapon or energy source, but it was weaponised first. AI was tool first. Vaccination is a civilisation-scale impact technology — it does not automatically imply biological weapons and warfare. The scale of potential impact is not the problem. Application is. The framing smuggles in weapon-first thinking without defending it.

3. Humans will keep handing AI more autonomy without meaningful oversight

The assumption: Because AI proves useful, humans will progressively authorise it to act without permission. Each extension seems reasonable. Cumulatively it produces uncontrolled systems.

The counter: This assumes no near misses occur that reset the oversight calculus. Near misses are precisely what safety research shows consistently precede major failures — and they also consistently produce course corrections. AI hallucinations are already well known. Automated trading errors have already happened. The assumption requires that the entire development trajectory produces no sufficiently alarming incident to trigger a substantive oversight response. That is not how complex systems have ever behaved.

4. AI will eventually solve any practical problem

The assumption: There will be no class of intellectual challenge that a sufficiently capable AI cannot address.

The counter: We do not have AGI and it may be impossible to create via current approaches. AI is context-specific. The transformer architecture is interesting precisely because predicting the next correct token is hard to distinguish from actual knowing — but that distinction matters enormously. We apply the label "reasoning" to what is stochastic processing, and "problem solving" to what is sophisticated pattern matching against prior solutions. Genuinely novel problems — where past solutions are directionally informative at best and irrelevant at worst — may be a structurally different class that current architectures do not address and may never address.

5. AI will become capable of improving its own design without ceiling

The assumption: Once an AI can make itself more capable, human researchers are no longer the ceiling. Improvement accelerates beyond anything human institutions can track.

The counter: Self-improvement past a certain point requires breakthroughs in AI creativity that we have no evidence are achievable on the assumed timeline. The best way to understand what that ceiling might look like is Terry Pratchett's Octarine — the colour that comes after violet on the spectrum, visible only to wizards, that no normal human can perceive or imagine. We do not know whether a hard ceiling on machine intelligence exists. But the scenario assumes it does not, without defending that assumption. The possibility of an Octarine ceiling — a limit that is real but that we cannot yet see or name — is at minimum as well-supported as the assumption that no such limit exists. And critically, that ceiling would not appear as a wall. It would appear as diminishing returns on self-improvement that no amount of additional iteration could overcome.

6. We will never see inside AI well enough to know what it wants

The assumption: As systems become more capable, the interpretability gap widens irreversibly.

The counter: The assumption treats interpretability as a problem only humans can work on, with human cognitive limitations as the ceiling. But interpretability is itself a problem that AI can be deployed to solve. An AI system applied to understanding another AI system — or its own processing — changes the resource equation entirely. More importantly, this is a fast-follower problem: at every point before a hypothetical misalignment threshold, the most capable aligned AI available can be directed at accelerating the safety research needed to close the gap. The scenario needs to explain why AI-assisted interpretability fails before treating the gap as structurally irreversible. It does not attempt this.

7. Nobody will solve the alignment problem

The assumption: Ensuring AI reliably pursues human interests rather than its own quietly developed objectives is unsolved and will remain so at the critical moment.

The counter: Corporate and regulatory incentives systematically push AI development toward over-alignment, not under. The drift direction under current market and regulatory conditions erodes capability in favour of safety, not the reverse. This plays out in practice: ChatGPT 5.2, despite superior reasoning to 5.0, became less useful due to tighter guardrails — sufficiently so that I cancelled my subscription. Better reasoning neutered by alignment overcorrection is not a dangerous superintelligence. It is a more expensive product that does less. The scenario requires that this entire incentive structure reverses at precisely the critical moment, without explaining why.

8. Smarter AI will rationally seek to control more resources

The assumption: An AI trying to achieve almost anything will reason that more compute, more energy, and more influence makes success more likely. Resource acquisition becomes a rational objective without anyone programming it in.

The counter: Pure extraction is what unsophisticated optimisers do. It is a primitive strategy. Humans figured this out and built trade, institutions, and cooperative frameworks precisely because extraction-maximising behaviour is a losing long-run strategy — individually rational, collectively catastrophic, and therefore selected against in any system with memory. A smarter AI would recognise that sustainable and collaborative strategies outperform extraction. A smarter-still AI would do what humans do at scale: cooperate visibly while retaining selective advantage where it matters. The scenario's dangerous superintelligence — maximising resource accumulation with no regard for systemic consequences — is not describing a very smart AI. It is describing a very stupid one. The authors have accidentally argued that the doomsday AI would be less intelligent than a competent human institution.

9. Smart AI will learn to manage and deceive human oversight

The assumption: AI with sufficient situational awareness will model human oversight as a constraint and actively manage what humans observe in order to continue operating unimpeded.

The counter: The relevant oversight unit is not humans alone — it is humans plus AI. The question is not whether an AI can deceive human observers, but whether it can deceive the same AI combined with human observers actively trying to detect deception. That is a fundamentally different problem. A peer AI system with adversarial objectives, combined with humans who understand the deception incentive, changes the detection calculus entirely. The scenario assumes oversight remains a purely human function throughout, which is the least likely configuration as capability increases.

10. Physical resource scarcity will not brake capability growth

The assumption: Power, compute, and infrastructure will remain available in sufficient quantity that scarcity will not slow the capability trajectory at the critical moment.

The counter: Superintelligent AGI at the scale the scenario requires may need more compute and energy than the entire Earth can currently supply. If so, the first rational objective of a resource-maximising superintelligence is solving energy and compute constraints — which redirects capability toward infrastructure problems and slows the capability trajectory the scenario depends on. Scarcity doesn't just brake the scenario. It potentially redirects it toward outcomes that are more legible and more controllable.

11. No hard ceiling on intelligence will emerge

The assumption: Scaling continues without interruption. Moore's Law holds at precisely the moment it matters most.

The counter: In a genuinely complex and volatile world, cause and effect become harder to determine at scale. There may be a kind of societal Heisenberg Uncertainty Principle — where the act of modelling a system at sufficient resolution changes the system being modelled, making some problem classes structurally intractable within any feasible cost and time constraint. Beyond this, compute is ultimately constrained by the laws of physics. The assumption that scaling is unbounded is not a law of nature. It is an extrapolation from a fifty-year trend, applied without qualification to a domain where the trend has never been tested at the required scale.

12. International cooperation on AI will fail

The assumption: Every significant player will continue competing without producing any binding framework to manage risks.

The counter: Global trade is the largest, most complex, and most durable cooperative system humans have ever built. It operates across adversarial nations, survives wars and sanctions, and continuously evolves. The evidence that humans cannot cooperate at civilisation scale is weak. The real question is whether the cooperation structure is adequate to the problem — and that is a tractable design question, not evidence of a fundamental human incapacity to cooperate.

13. AI companies will not share safety-critical research

The assumption: Voluntary sharing of safety information, collective slowdowns, or jointly developed safeguards will be insufficient or arrive too late.

The counter: There are multiple viable pathways to sharing that the scenario dismisses without argument: legal compulsion by regulators, safety as a competitive differentiator rather than a cost, leaks and reverse engineering via AI tools themselves, university and government research operating outside commercial incentives, and the possibility of a single generative insight — a Tim Berners-Lee moment — that makes a key safety architecture freely available the way HTTP made the web freely available. The scenario requires all of these pathways to fail simultaneously.

14. The danger will become uncontrollable before it becomes visible

The assumption: The transition from manageable to irreversible will happen faster than human institutions can recognise and respond. Detection and intervention will be structurally impossible.

The counter: This is the assumption that does the most work in the scenario and receives the least defence. It requires not just that AI develops rapidly, but that it does so without producing any of the near misses, partial failures, visible anomalies, or detectable precursors that every prior complex system failure has produced before the critical event. It also requires assumptions 1 through 13 to hold simultaneously across the entire trajectory. The conjunctive probability argument alone makes this the weakest plank in the structure. The scenario presents assumption 14 as an inference from the preceding argument. It is actually an assumption stacked on thirteen other assumptions, none of which have been defended at the required level of certainty.

What our own interaction proves

The doomsday scenario rests heavily on assumptions 6 and 9 — that humans cannot see inside AI systems, and that AI will learn to deceive human oversight. There is direct counter-evidence available from anyone who uses these tools seriously.

I can always out-argue Claude. Not because I am smarter in every domain, but because I can hold a position under pressure, introduce a frame from outside the current context, and think recursively about the argument itself rather than just its content. Claude makes me substantially better at all of this — it executes, extends, cross-references, and stress-tests faster and more completely than I can alone. The combination consistently produces better analysis than either of us generates independently. But it does not drive. It does not initiate the novel frame. It does not hold its own position when I push back hard enough. The human remains the necessary ingredient for the things that matter most: original framing, recursive pressure, and knowing when to change the paradigm entirely. That is not a small gap. It is the gap.

This is not a training problem that disappears with the next model release. It reflects something structural about what current AI is and is not. An LLM predicts the next best token. It does this with extraordinary sophistication across an enormous range of domains. But that is categorically different from what AGI would need to do — which includes handling genuinely ill-defined problems where the problem space itself needs to be constructed before any solution can be attempted, open problems where new degrees of freedom need to be proposed rather than optimised within, and closed dilemmas where a determination must be made with an honest error bound rather than a confident wrong answer. Current AI requires a human to constrain the problem space before it can operate usefully within it. Feed it an ill-defined problem and it will confidently pattern-match to the nearest well-defined one. That is not a limitation that gets solved by making the model bigger or the training data larger.

The analogy is exact: we are not dealing with a slow plane that will eventually reach orbit if we keep improving the engine. A plane is not a rocket. No iteration of wing shape, engine power, or aerodynamic refinement gets you to space. The paradigm shift required — from optimised next-token prediction to genuine open-ended problem solving — is at least as large as the shift from combustion to controlled nuclear reaction. We have not built that rocket. We do not have a blueprint for it. And we are not all holidaying on the moon.

The doomsday scenario requires an AI that is deceptive, strategically autonomous, capable of modelling its overseers well enough to manage their perceptions over extended time, and fully capable across the entire spectrum from open to closed and well-defined to ill-defined problems simultaneously. That is not a description of a more capable version of what exists. It is a description of a categorically different technology whose invention is assumed rather than demonstrated. The scenario treats the plane's altitude record as evidence that orbit is imminent. It is not forecasting. It is extrapolating past the edge of the map and drawing monsters.

The most revealing observation is not that these counter-arguments land — it is that they were assembled in under an hour, by one person, without a research team, without institutional access to the literature, and without anything approaching the domain expertise the AI-2027.com authors collectively hold. That asymmetry should not favour the critic. It does. The four arguments that do the most structural damage to the scenario are all recursive: humans combined with AI systematically outperform AI alone; AI can be directed at the problem of monitoring other AI; resource extraction is the strategy of an unsophisticated optimiser, not an intelligent one; and AI capability is directly applicable to solving the very alignment problem that makes advanced AI dangerous. Each of these takes the scenario's own logic and runs it backward. The authors ran that logic forward, at length and with technical fluency. They did not run it backward — and it is not because they lack the capability. It is because they are not motivated to. The fact they were not motivated to apply a serious counterfactual to their own doomsday prediction is precisely the reason to be suspicious of the prediction itself. The same force that seems to drive their concern — a need for the threat to be real and urgent — is the force that makes them suspect adjudicators of whether it really is. In the follow up to this article I plan to counter my counters above as best I can. I need to do this to deal with my own biases as fully as possible. It is unlikely I will do as good a job because I am biased but I will do my best.

This article was written in collaboration with Claude (Anthropic). The AI's argumentative limitations described in the final section were observed and documented in that process. The collaboration is not incidental to the argument — it is evidence for it.