The mind virus: Why you should care whether you read AI-generated content

How AI broke proof-of-thought and created a horde of fools

My husband has a friend from high school who time has turned into more of an acquaintance. Let’s call him Tom. Tom would consider himself rather erudite and has strong political views, which he isn’t afraid to express on Facebook. But in the last year, a strange pattern emerged. My husband noticed Tom’s Facebook posts were becoming a little longer and far more witty. His arguments were landing like metronomic jabs to a speed bag and he would often round out each post with a flourishing one-line punch.

Having been active on Substack for over six months, I am a little more attuned to the prose of AI than my unsuspecting husband, and was immediately suspicious. Peering over his shoulder one day, I quickly read Tom’s latest rant and knew instantly it was AI-generated. (I got my husband to run it through Pangram to verify, and it was indeed 100% AI generated.)

With great delicacy — the kind warranted when you’re about to call into question the erudition of one who values such a thing — my husband messaged Tom and asked him if he used AI to write his posts. Tom wrote back vehemently defending himself, saying that the thoughts were all his own and that he only used AI for “polish.”

“It’s easier to fool people than to convince them that they have been fooled.”
— Anonymous (not Mark Twain)

Well not-Mark Twain,¹ I’m about to try.

For 3000 years, a piece of writing was evidence of human thought, and the fluency of that writing was a loose proxy for the quality (or at least degree) of thought that went into it. For the most part, good writing meant there was a mind behind the scenes that had done a lot of work.

AI broke writing as proof-of-thought.

The mind virus

When you read something that easily flows through your synapses, the ease with which you process those words is taken as a signal that the work itself is true — that you can trust the person who wrote it knew what they were talking about.²

We implicitly grant fluency a higher degree of trust.

Which isn’t a problem, as long as the bulk of fluent writing out there on the internet is true.³

And up until about six years ago, this was the case — for the most part, when you came across a fluent piece of writing, it suggested that a mind behind it understood the subject.⁴

Before AI, fluent writing was difficult to create. Someone, somewhere in the chain had to sit down and concoct a coherent and flowing piece of writing using their very own thoughts. It took immense time and effort. If you wanted to spread false information, you had to be a good writer to do it. Or you had to be a person with the means to pay for a good writer.

Sure, ghost writers have existed since writing was invented. State propaganda and polished lies have been spread throughout history. But the malicious actors doing so were always filtered by cost. Misinformation was spread intentionally. It was impossible to unintentionally spread convincing misinformation because nobody could unintentionally create convincing writing! This meant that the bulk of good writing was written by people who truly understood what they were writing about.

After all, to truly understand something, you need to be able to explain it. And to explain it, you need to have assimilated extensive knowledge and the ability to synthesise it — to recognise where and how to build bridges over that dense knowledge so that they can be walked by the reader. Before AI, fluent writing signalled that someone had laboured for hours trying to collapse three-dimensional thought into a two-dimensional stream of words that could convey that understanding to another person. Good writing meant good thinking. Or it used to.

AI broke this. Fluency can now be manufactured, at scale. At the heart of the mind virus lies a broken signal: fluency is no longer a measure of thought.

A fluent piece of writing can still mean that many hours of thought went into it. It can also mean a one-line prompt did.⁵

Awareness doesn’t immunise you

Knowing you possess this fluency bias doesn’t deactivate it.⁶ Even a motivated, aware reader still feels the pull of fluent AI-generated prose because the calibration is sub-cognitive. It’s simply a trained reflex to believe or trust seemingly coherent writing.⁷

The sheer volume of AI-generated content now makes it almost impossible to escape. The more you read AI-generated content, the more you tend to believe AI-generated content, slowly recalibrating your sense of what truth sounds like towards AI prose.

The burden of proof

AI didn’t just break proof-of-thought, it obliterated the effort that used to be tied to fluency. Anyone with an internet connection and $30 per month can now forge proof-of-thought and manufacture superficially beautiful fluency at the hit of a button. The burden of effort — the burden of truth detection — now rests squarely with the reader.

Before AI, bad writing (and therefore inadequate thinking) could be quickly deduced within the first few sentences. Such writing simply wasn’t shared. Post AI, you’re often elbows deep in an essay before that hollow feeling of soulless perfection begins to creep up your spine. And every moment spent parsing that supposedly coherent text is a minute spent exposed to the virus.

Sadly, many other less immune readers before you may never have felt that chill and implicitly trusted the fluent prose, sharing it with others and spreading the virus.

How the fool is fooled

It isn’t just the malicious actors that now have free access to a fluency-manufacturing tool. A far more dangerous kind of individual has emerged: the fool who doesn’t realise they have been fooled.

Let’s go back to the year 2018. Covid hasn’t yet eaten the world. Nor has AI. (It is a far simpler time.)

While browsing online, you stumble across a really good article which hooks you from the very first sentence. It is fluent, coherent, and its prose flows like a gentle current through your mind, flicking on little lightbulbs here and there as it goes. You find yourself thinking, “Wow, great point!” more often than not.⁸

Now, picture yourself after having read this insightful essay, the glow of its insights still alight in your mind. The writer made so many good points! You nodded emphatically as you read it. It feels like that knowledge is all there in your mind. So you excitedly go to explain it to your partner. The words came out in a mangled mess.⁹ You get one point across, maybe. But it isn’t wrapped in the context that made it elicit the same lightbulb moments in your partner’s mind and you frustratingly don’t see them nodding at you quite the same way you did when you read the article. (You end up whipping out your phone and just sharing the article instead.)

Fluent writing isn’t just implicitly trusted, it also creates a false feeling of understanding.¹⁰

The beauty (and now danger) of smooth writing is that its fluency simulates this understanding in the reader. That is the whole point! But that understanding is superficial. It’s an activation of metaphors — concepts you do understand — deftly chosen to bridge vast gaps of things you don’t understand. Those gaps are all densely populated in the writer’s mind, but absent in yours. But you aren’t aware just how many gaps there are until you go to explain it. The temporary scaffold of understanding activated by reading quickly fades if you don’t take deliberate steps to retain it.

This is why reading (especially fluent prose) feels falsely productive, as if you are actually learning. You’re not. You can only retain what you produce and retrieve, not what you review.¹¹ You’re simply letting circuits activate then fade away.

Prompted outputs are not evidence of understanding

Now fast forward to 2026.

Covid has since eaten and spat out the world. AI is currently masticating.

Instead of stumbling across a beautiful lightbulb-eliciting essay online, imagine a person has prompted one out of Claude. The lightbulb moments don’t make them think, “Wow this writer is making such great points.” They make them think, “Wow, I’m making such great points!” (In fact, Claude often even tells them as much!)

Because the fluency is borne out of their own actions and that fluency simulates a feeling of understanding, they think it means they do understand it — that the words on the screen represent their internal understanding.

But where is the production and retrieval? Contained in the back-and-forth between prompt and output? Until you can hold a coherent narrative of interlinking points in mind and lay them out in long form such that someone else (who does not possess all your prior domain knowledge) can understand what you mean, you do not understand it yourself.¹²

One could argue (and many “ones” do) that prompting is still thinking. But it’s unverified thinking. Unless someone has taken that last and crucial step and tried to coherently explain their entire thesis without the crutch of AI, they don’t know whether their understanding is the integrated kind, or the kind that fades away like a dying bulb.

AI is only as trustworthy as you are expert

There is another layer of foolery going on here, too: the assumption that AI’s own fluency is representative of accuracy.

To understand what I mean, go open your AI of choice and start asking it questions in an area in which you have immense expertise, as if you were trying to learn about it anew. I guarantee you within three or four prompts, you’ll have caught the inconsistencies and inaccuracies. Yet at the same time, you’ll no doubt begrudgingly acknowledge how beautifully packaged and confidently explained those inconsistencies are.

Now go open a new chat and try to start learning about something you have no expertise in. See how convincing the explanations are and how easy it is to simply accept them in their pretty packaging.

Now think of all the nuance you are missing.

Someone who blatantly trusts AI’s output, and takes its answers to their own prompts as evidence of their own understanding is dangerous, but at least their ignorance is only dangerous to themselves. What happens when that individual is handed the internet — a plethora of free platforms upon which they can publish their “understanding”?

The horde of fools

The true danger now lies in the person who is neither novice (who knows how much they don’t know) nor expert (who also knows how much they don’t know) but in the competent middle (who have no idea how much they don’t know).¹³

Because they’ve now been handed a tool that makes them think they know everything. And further free tools to disseminate that pseudo-understanding.

I’m talking about the kind of person who has some domain knowledge but never developed it (or the skill of synthesis) to the point of fluency. They’ve never tried to properly explain it before.

But AI has finally given them the (false) confidence to do so. So they open Claude and start pitching ideas for an essay. Their sycophantic thought partner encourages them, regurgitating their own ideas with fluent ease, and pulling existing knowledge to support them. The person feels understood — “Yes, that’s exactly what I mean!” This miraculous tool has finally allowed them to express what they truly think. What magic indeed!

After a few more iterations of this, the person instructs Claude to craft an essay that coherently lays these points out for a reader, and then, after a little more back-and-forth (and a lot of copy and paste) publishes that essay as their own words.¹⁴

The scariest part isn’t that people are using these tools to manufacture fluency — to forge their understanding. It’s that they truly believe they do understand what they publish.

AI has fooled a large population of online content creators into thinking they know what they truly think. Their superficially fluent output is then read by unsuspecting others, unknowingly simulating (possibly false) understanding in their own minds, and they pass it along with a cursory hit of the “share” button. And so the virus spreads this pseudo-understanding, infecting the minds of all who say they don't care whether they're reading AI-generated content.

What is more dangerous than a tool that makes you believe you know more than you do? A tool that makes others believe it too.

“Zombie horde” by A.R. (not rendered by AI)

How I wrote this

I think AI is amazing. What took me four years and a PhD I can now do within weeks using AI. I use it to research,¹⁵ brainstorm, proofread for spelling and grammar, analyse data, build automation tools, run those tools, develop the HIYB app and manage user workflows, triage my messages and emails, generate timelines and strategies, format references, create diagrams, generate images (but my human daughter hand draws HIYB’s official illustrations) and act as an adversarial reviewer.

I do not use AI to write.

I actually don’t even consider myself in a position to write a new essay until my knowledge — my ability to retrieve and synthesise — is at a point where I start catching the AI in inconsistencies. Even then, when I go to write an essay, the process often reveals enormous gaps in my thinking. And the greatest insights and ideas usually come during the explanation process (the non-AI part of the process) more so than during my Socratic prompting.¹⁶

To write something is to understand it, sure. But so much of that understanding is built through the writing.

How I read

As a reader, there is nothing I love more than having another writer provoke my thoughts — a human writer, not a machine facsimile.

I don’t demand provenance because of any self-righteous sense of fairness or equality as a writer. I demand it simply because I love reading, and to me, reading a good piece of writing is like mind-melding with the writer across space and time.

I can fairly quickly spot a piece of AI-generated writing but I know my judgement is not infallible. This is why I support Substack’s decision to integrate AI detection into their platform, a decision that was met with great uproar amongst many writers. To me, the backlash reveals how far the virus has spread. The loudest objections weren't about the detection itself but what it implies: that the machine did the thinking.

Convincing the foolish

Fluent writing can still mean the writer understands what they published. But for the first time in the history of the written word, it can also mean that they don’t.

And that they don’t know they don’t.

As a reader, this terrifies me.

To all the other readers out there taking in these words — words that I have laboured over for 10 hours, not one of them written by a machine — I hope that I have convinced you of the danger of reading AI-generated content.

To those who have fallen victim to the virus, I implore you: please give your brain a chance. I would love to read what you think, not what your AI leads you to believe you’re thinking.

You can use AI to help you think. You can even use it to help you understand. But you will never know if you truly did either until you write.

And neither will your readers.

My interpretation of what reading human writing feels like (ironically rendered by AI)

---

If you’ve been publishing AI-generated writing and this essay gave you even the slightest pause, I’d love to hear (your beautiful human-written comments).

In an upcoming essay, I’ll share the rules I use to ensure AI amplifies my thinking instead of displacing it.

---

References for this essay, and for the wider series, are available as a collection in the Research Library, specifically Fluency, Truth, and the Illusion of Understanding.

1. I have come across this “Mark Twain” quote before but, in a delicious turn of fate, it popped up on my radar right when I was in the middle of this essay. Before I went to affix it right at the top, I did some due diligence to confirm that Mark Twain had indeed said such a thing. He didn’t. The origin of that saying isn’t known. Which I find hilariously ironic because despite naming the misattribution here in this very essay, it will likely outlive this correction in your mind and in ten thousand copies online. Difficult to convince people that they have been fooled, indeed.

2. Psychologists argue that the fluency-truth reflex was justified because it developed in a world where fluent things were usually the product of competent minds (Reber and Unkelbach 2010), which is exactly the point I am making here. AI nullifies that base assumption which makes that reflex newly obsolete. We’re running an innate bias in a world that doesn’t exist anymore.

3. By “true,” I mean factually correct and a relatively accurate representation of knowledge.

4. I’m not arguing that “proof-of-thought” is a clean signal. It has always been noisy — plenty of fluent human writing has always been confident nonsense, status signalling, or regurgitation of received wisdom. But with the advent of AI, the signal-to-noise ratio has exponentially worsened to the point of obliteration.

5. True story: I once received a Substack newsletter in my inbox by a “writer” with tens of thousands of subscribers who had accidentally pasted a hasty one-line prompt (complete with spelling errors) with the entire AI-generated essay. He quickly fixed his error in the online form of the essay and has since doubled in size.

6. Awareness and motivation can reduce this effect in controlled settings, but it falters under cognitive load. When you combine its automaticity with the sheer quantity of AI prose now in circulation, it is difficult to see how one can easily un-automate it.

7. Processing ease (even purely perceptual ease) has been shown to raise judged truth, independent of content or argument quality (Reber & Schwarz 1999). Conceptual fluency (ease arising from coherence and structure rather than mere legibility) produces the same class of effect (Alter & Oppenheimer 2009) and repetition compounds it (Hasher et al. 1977). My extrapolation is that since AI prose is a powerful source of processing fluency, it should only amplify this implicit feeling of truth.

8. This was often my experience when reading a PG essay. I highly recommend.

9. I acknowledge that verbally explaining something is different to explaining it using the written word, but both require (a good amount of) domain knowledge and the ability to synthesise analogous concepts using that knowledge.

10. People believe they understand how things work far better than they actually do, and this illusion shatters the moment they try to explain it (Rozenblit & Keil 2002).

11. Obviously, if you read the same passage of writing thousands of times over, you would retain it, but it would be a flat chunk of memory and wouldn’t be associatively linked to other domains of knowledge. Understanding something requires retention and integration. And the only way to test such a thing is by synthesising – by explaining it in writing.

12. This has been extensively tested — repeated reading results in about 40% retention after a week whereas reading and practised recall results in 61% retention. The researchers even gave this reading-gives-false-feeling-of-retention phenomenon a name: “illusion of competence” (Roediger & Karpicke 2006). There would of course be some recall going on in a back-and-forth with AI but, without composing the final narrative, it is missing the stage where the entire narrative is held in mind as a working sequence.

13. Potentially known as the Dunning-Kruger middle.

14. This is a typical example of a human’s process of creating AI-generated content. There are of course many shades of grey that colour the degree of input the human has. AI use sits on a vast spectrum, from heavy human oversight to pure copy-paste. This example is designed to represent any case where a human has not thought and written the entire final essay themselves.

15. I don’t rely solely on AI for research as I find it is often overly confident in its synopses of other research papers and articles. My research process involves using traditional search engines as well as AI to find articles which I read at varying depths depending upon their abstract.

16. I can see the hypocrisy in using tools that elicit “unverified thinking” to help me think, but I test that thinking by writing. The one thing that does keep me up at night is whether the fluent AI prose I read as part of my brainstorm and research is causing me to subliminally adopt its own writing techniques. I take measures to mitigate this risk by asking for flat responses (no flourish) and hope that my explicit awareness that what I’m reading is knowingly AI-generated content will help immunise me against this absorption.