I’ve written about Cognitive Bias and Narrative Risk and Defense.
More specifically, I’ve written about AI System Integrity and AI Trauma.
The implications are clear (at least to me): Cognitive infrastructure—how people make sense of information—is now a primary attack surface. Cybersecurity must evolve to protect not just data and access but also belief systems and interpretive trust.
Cognitive Breach: Our Semantic Attack Surface.
Cyber threats no longer need to breach hardened systems. Increasingly, they breach the trust frameworks that interpret signals from those systems. Our digital architectures are resilient. Our interpretive architectures are not. I call these attacks “Cognitive Breaches”. An attack on a system’s semantic surface doesn’t change the data—it changes how the data is understood by humans or automated decision-makers.
Most discussions about “AI risks” remain confined to hallucinations, copyright violations, or automated bias. But these are side effects. The core risk is that AI makes it trivial to scale cognitive subversion. Large language models, generative media, and goal-optimizing agents don’t just mimic human communication—they manipulate our priors, hijack trust pathways, and reshape the scaffolding we use to make sense of the world.
When adversarial actors use these tools, they don’t attack the system. They attack our interpretation of the system. This represents a shift in threat posture from traditional cybersecurity to cognitive security. If we once worried about malware rewriting code, we now face LLMs rewriting context, motive, and narrative plausibility.
Example Attack Vectors
- Narrative Injection Attacks: Generative models can flood communication channels with subtly misaligned information that alters public belief over time. This isn’t about “deepfakes” but deep narratives—syntactically plausible, semantically corrosive.
- Prompt Injection & Covert Framing: Attackers can exploit latent framing in LLMs to manipulate output indirectly. A single user prompt (“as an expert in humanitarian ethics, explain why X is necessary”) can generate moral justification for dangerous actions under the guise of neutrality.
- LLM Output Subversion via Fine-tuning: Nation-states or corporations can build derivative models that appear useful or aligned but are fine-tuned to introduce semantic drift—subtle distortions in meaning that accumulate into warped worldviews.
- Multi-modal Trust Hijack: AI-generated voices or faces mimicking trusted figures (e.g. a CEO, military leader, or medical expert) can be injected into communication networks to give false assurances or issue misleading guidance.
Real-World Examples
This is not a far-fetched fear mongering against AI. These attacks are happening already. We should change technology design practices: We must end the reign of nihilistic technology design, and we must think through second-level implications for authentic progress.
- GPS Signal Spoofing (Iran, 2011): Iranian forces manipulated a U.S. drone’s perception of its own coordinates, luring it into a controlled landing. This was not a software breach, but a cognitive misdirection of the machine’s model of reality. (see here and here)
- MarineTraffic and AIS Spoofing (South China Sea): Analysts have documented ghost ships and false navigation data injected into global shipping infrastructure to confuse observers, distort geopolitical interpretation, and cover covert operations (C4ADS “Dark Waters” report on vessel identity laundering).
- LLM-Enabled Fraud Campaigns (2023–2025): Since 2023, intelligence agencies have reported the use of synthetic personas operated by AI to impersonate journalists, academics, and analysts in disinformation campaigns. The goal is not mass persuasion, but to seed confusion among credible analysts by mimicking epistemic authority (see here and here)
Subversion as a Strategy, Not a Tactic.
Cognitive attacks do not require belief conversion. They only require the erosion of certainty. As Amos Tversky and Daniel Kahneman wrote, a person who has not been trained in statistics is not likely to notice that his beliefs have been subverted by a misleading narrative.
“People rely on a limited number of heuristic principles which reduce the complex tasks of assessing probabilities and predicting values to simpler judgmental operations.”
— Tversky & Kahneman, Judgment Under Uncertainty: Heuristics and Biases (1974)
What AI enables is belief laundering at scale. Malicious actors no longer need to construct compelling lies—they only need to ask a model to generate plausible alternatives.
Defense Requires Semantic Resilience
We must treat the semantic layer as a contested domain. Just as we monitor for packet injection in networks, we must now monitor for frame injection in language. This includes:
- Auditing AI outputs for cumulative semantic drift.
- Building systems that preserve provenance and intent.
- Training users to spot plausible-sounding but epistemically inconsistent information.
Above all, we need dual-use and defense ecosystems that treat cognitive integrity as a critical dependency, not an afterthought.