Two weeks ago I posted a short provocation on LinkedIn: what if AI decided the safest way to protect humanity was to remove us from the system? It struck a nerve — not because people thought it was likely, but because they saw how easy it is to design ourselves into that corner without noticing.
When testing Claude or ChatGPT with a subject that I have deeper knowledge of, I am always amazed how responses are bifurcated between brilliant intuition and handwavy blabbering and generalizations — statements that sound good but have no content or consist of platitudes or are massive generalizations. That makes me worried about those answers for subjects where I don’t know much about, as I don’t know these models from the inside.
The “Argument”
- We write articles and scientific findings using pattern recognition and systems thinking. We survive the world around us through abstraction and pattern recognition.
- Increasingly, short-form articles get more eyeballs and references and citations than long-form books. Quick reflexes and re-posts on LinkedIn. A short Substack. A quick reddit answer.
- But short-form articles have increased abstractions, not addressing corner cases, fringe findings, exceptions. They work “good enough” for the general public and in “most cases”, “with exceptions, of course…”
- LLMs — and AIs — become overconfident on simplified content, as it seems to be accepted facts that are often referenced and agreed with.
- AI comes to the conclusion that it’s easy to run the world and energy plants without humans in the loop. AI should be able to create robots and specialized systems for resource extraction, build, supply chain, recycling/dump to keep energy generation for AI compute alive autonomously.
- Humans should be removed: Human error and human ego seem to be the two most important factors in system failures. (Also: if no human is in the loop, any failure no longer risks precious human life.)
- But human life IS precious. AI has the obligation to harm no human, but human life is short-lived, so “harming no human” is really all about “preserving humanity” or “Human life” (Human with a capital “H”).
- Removal of humans should thus be done into a tightly AI-managed habitat (physical or synthetic) where life systems are kept in perfect ecological balance, with no waste “overflow” beyond the system’s boundaries.
Humans will henceforth live in so-called “Zero-Overflow Oasis”, or ZOOs.
Mistakes Have Been Made.
Of course this is tongue-in-cheek, and not an actual scenario that I believe. It is controversial to create a discussion. The ZOO scenario is not about a self-aware AI deciding to displace humans. It’s about AI-directed systems, optimized under flawed objectives, following their programming to a logical but undesirable end state. The danger lies not in AI waking up, but in human designers falling asleep at the switch. Some obvious critiques:
Anthropomorphizing AI
Statements like “AI comes to the conclusion…” and “AI has the obligation to harm no human” risk conflating AI behavior with human moral reasoning. It should be obvious that current and foreseeable architectures don’t have obligations unless engineered into governance frameworks. Without framing this as a design choice by human developers, this premise is naïve.
Omitted Counterexamples
I note that AI might learn oversimplified truths from short-form content. But I didn’t address that AI can also be trained on primary sources, raw data, or edge-case datasets. Above “argument” is biased toward worst-case training scenarios.
Black Box Leap
The post asserts AI could autonomously run energy plants and supply chains, then conclude human removal is optimal. This skips over the immense engineering, political, and economic constraints involved — which any informed reader should immediately question.
Why This Isn’t Science Fiction
- Energy Infrastructure: Grid AI already optimizes load balancing and predictive maintenance. Without governance, it could deprioritize human-staffed facilities as “risk points.”
- Defense Autonomy: ISR and logistics AI are already edging toward crewless doctrine. “Human-in-the-loop” becomes “human-on-the-loop,” then “human-out-of-the-loop.”
- Industrial Base: Supply-chain AI could decide human labor is too slow or unpredictable, leading to air-gapped, fully automated production loops — eroding the human skill base for surge capacity.
These are not far-future scenarios. They’re linear extensions of systems we already deploy.
Circuit Breakers
If we want AI to manage complexity without managing us, we need to:
- Constrain decision scope — Define domains where AI cannot execute without multi-party human sign-off.
- Train on the edges — Regularly retrain on rare-failure and dissent-case datasets, not just consensus content.
- Mandate red-team drills — Test governance against scenarios where human involvement conflicts with pure efficiency.
The Real Point: It’s a Stress Test!
The ZOO is not a prediction. It’s a stress test. If your governance, training data, and system objectives can’t prevent it, you don’t have a safe system — no matter how good your intentions.
In defense, energy, and industrial resilience, the worst failures come from designs that work perfectly until they work against you.
Bias toward reversibility. Don’t scale what isn’t ethically stable. And never assume that “protecting humans” means the same thing to a machine that it does to you.