You have articulated an important point. It’s one I’ve been circling in some of my own drafts. I completely agree with all of it. A related point Ive been working on is that writing-all art really-is, as you said, a meeting of minds. The tenor and flavor with which one writes is a part of that mind and helps me meet that mind more faithfully. So, I’ve concluded that AI used to polish the output can only diminish the art value of the piece with the possible exception of language translation. AI may add fluency but it occludes the signal coming from the other person’s mind. I would rather read their less polished, but more faithful, thoughts.
As I read this I am inspired to the realization that my frustration or vitriol towards AI writing is precisely because of what you've articulated here: with an LLM, anyone can write prose that is consumable. No longer is the work of writing - the daily writing recommended by E.B White or Stephen King or Annie Lamott - required; now you just need a whim of an idea, a whiff of a thought regurgitated from somewhere else, and LLM will produce you a polished, shiny penny you can show everyone that passes. Meanwhile, those still actually writing may produce less (because writing takes time) and be seen less (because human writing doesn't come out packaged in the most algorithmically productive way). Over time, human tastes may prefer LLM writing, just like the American palette seems to prefer package salty snacks and fast food ...
You’re absolutely right (and I adore your phrasing: “whim of an idea” or “whiff of a thought”).
And you make a great point that I wish I could go back and expand in the article: the asymmetry of production time between AI “writers” and human writers.
When consumable content can be manufactured with the speed of a mere whim or whiff, the signal-to-noise ratio approaches 0. How will the readers who crave human-written words ever find the writers who write them?
We are already seeing this in the go-to-market space (my day job) - the volume of marketing material and sales outreach is enough to drown in, and it's changing the rules of how marketing and selling can be effective.
I think this is the dark side to Ray Kurtzweil's singularity. One if the things I hate about using Copilot or Google AI is that they tend to pull from social networks like Reddit and YouTube.
Online communities are great for finding like-minded people who share your interests and views (especially for people like myself who don't get out much except for work). But they also cater to the "loud majority" who are often critical and seeking attention.
I don't know who came up with the term but maybe 20 years ago I said we now live in the age of misinformation. If you say something often enough and get enough people to believe it then it becomes true whether or not it is. And the internet has allowed this to become something worse than a pandemic. Just look at a few YouTube shorts. AI can generate footage rhat on the surface looks as real as anything until you stop and consider that there's no way someone can defy physics.
Assuming you do. But I think this is how society in "The Matrix" became so willing to be human batteries for the machines. If you allow someone else's thinking to subjugate your own, you give up the ability to fend for yourself at all. Or at least that's my take on it.
(By the way, I've also noticed that the more I write, especially by hand, the more my logical and cognitive thinking improves. AI can't reproduce that.)
This is one of those rare comments that has one nodding at their screen. I wholeheartedly agree with you. I’ve invoked The Matrix many times in my writing also, because it is an uncannily accurate metaphor for the state of the modern world, except we haven’t sold our energy, but our minds instead.
I also found your comment about handwriting interesting! It’s a mode I use for jotting down thoughts but you’ve made me contemplate trying out a longer piece of writing using it.
The first Matrix movie is one of my all-time favorite movies. Not just for the action and Keanu but the metaphysical and philosophical aspects as well.
I got out of handwriting for a long time. My method is, handwriting I'd my first draft, then when I type it up I make edits or changes as a second draft.
Another brilliant one. Very close to my heart indeed.
I’m a TV producer in my day job. A decade ago I made a run of documentaries on quite diverse subjects - Karl Marx here, the breakfast cereal business there. I remember saying to my boss when I got each job that I didn’t know much about the topic. He said (rather menacingly) well you will by the end of it.
But (as he well knew) it wasn’t during the reading that I got my head around the topic. Back then there was voiceover to write (since fallen out of fashion). We sat together and crafted and refined really tight prose to be read over the programme. It’s a wonderful exercise - you have to boil down and polish, boil down and polish. Notice the inconsistencies, go back and revise. If you can explain it to five year old, that’s when you understand it.
I don’t use AI to write personally for all sorts of reasons. Each to their own - all I ask of others is transparency. But I do think people who lean on it hard are missing one of the great joys on offer in writing.
I loved this comment and your story. Thank you for sharing this, Will. (I chuckled at the “rather menacingly.”)
You’re absolutely right: the iterative process of boiling down and polishing is where the knowledge and understanding is built. To hand all of that menacing-yet-ridiculously-satisfying toil over to a machine… well, it just makes me rather sad. You said it best: people who lean on it hard are missing out on the joys of writing.
In my day job I do a lot of copywriting. It’s for marketing, and it’s boring. I use AI for it. And TBH, even when it was hand-crafted it was mostly slop (humans make it too). Sales-speak. Overdoses of cliched superlatives. Now, it’s devoid of personality too. But so quick.
Where I have truly become bothered, where I first noticed the pernicious impact of AI, was junior colleagues asking Claude to write a business plan. I’d start reading the plan, and be immediately impressed by the quality of language. “Wow! So-and-so has developed enormously since the last time they did this, time to think about promoting them”. Then, a couple of pages in, comes the dawning realisation that it’s beautifully written word salad.
I’d discuss the plan with the “writer”. And they’d have no idea how to execute it, hadn’t considered any of the likely pitfalls, and couldn’t coherently represent it. Of course they insist they have had a dialogue with Claude. And that it is “their” work. But a few simple questions shows that it’s not.
And here comes the new risk. When you convince yourself and others that you are an expert when you are not serious risks can be introduced. If you aren’t actually a subject matter expert you’re just an idiot with a pal who writes well.
The sheer act of writing things down serves to force you to put your thoughts in order. To generate synthesis. To be able to coherently describe what you know is to achieve a level of mastery over it. Using a machine to do it is dodging the task of doing the underlying thinking. You can get a great written result, but it is a simulacrum of competence that conceals its absence. You’ve been deceived, and you are actively engaged in deceiving others about your capability.
This is why AI users are so attached to the “tool”. They are personally invested In a belief that it has made them better, and that they have become”writers”. It’s about the same as thinking you’ve become Gordon Ramsay because you got a takeaway delivered. And when someone asks you to stand in from of the hob and create, you can’t.
I honestly don’t mind AI in business content. It was already self-serving, self-interested slop and AI has made it easier on the eye. But in actual learning and culture I despise it. You cannot use it and be a master of your subject because its usage involves removing the very thought process that generate mastery.
And I don’t like fake. And I don’t want to pay for it. And the objective quality of the “product” is unimportant to me. The value for me is I’m human connection not in machine-generated replacement of it. Please AI sloppers, respect my paying consumer choice.
“When you convince yourself and others that you are an expert when you are not serious risks can be introduced.” Exactly!
Once upon a time we just had to contend with maliciously spread misinformation. Now we have to contend with sincerely (with perhaps a sprinkle of delusion) spread misinformation.
I really appreciate you sharing your story at the front lines of this.
i agree with everything you said, loved the point about the socratic dialogue with AI, the ilusion that the reader itself is plato. i think the hardest part is knowing when I am making my life unnnecessarily hard by refraining from
Thank you so much for taking the time to read and comment. I truly appreciate it.
I know what you mean about trying to draw that line. I have tried to stay on the side of the line that helps me keep the cognitive skills I value, while delegating mundane and automatable tasks to AI (which only frees up more room for said cognitive skills).
I use AI to do software development for me, for instance. And one could argue I’m offloading the cognitive effort of algorithmic design, and they would be correct. But that’s a skill I don’t want to maintain (I’ve done enough programming the old-fashioned way!), and I’m happy to keep my creativity at a higher level of abstraction — the program’s requirements design.
But when it comes to writing, the final essay composition involves a variety of cognitive skills I most certainly do not want to offload: critical thinking, working memory, narrative development, mentalisation, recall, synthesis, creativity etc. etc.
So I would say we’re making our lives hard, but for the right reasons :).
Sorry to say, merely asking for a flat response does not give you something very much less LLM-styled. (ask me how I know! LOL) It is still quite wrapped in LLM-style-prose... While I believe awareness is central to staving off said expressional tainting, I don't think it is enough by itself. The best preventative measure I have found is to read and mull over writers who are verifiably not using / being exposed to AI. Read the old stuff, study the old stuff, develop and practice your voice in-depth with the old stuff. Just like Benjamin Franklin did with The Spectator.
Another point, tools like pangram and others should not be used for anything of gravity. Reason being that they have wild inaccuracies. 100% or 0% signal broken metrics, not dissimilar to the overconfidence an LLM produces... With some clever prompting, it's possible to get minimal thought produced by an individual to sound as if they wrote it themselves. Likewise, new writers are likely consuming LLM-generated text at a more formative period of their development. This makes them far more prone to using those turns of phrase. Some genuine novices have been hurt in the witch hunt.
I take your point and in no way think that Pangram is infallible. But in the absence of any other tool to help deduce superficial fluency, it's the best we've got I'm afraid.
I do love how you said that you like to "read and mull over writers" in order to make your own assessment. This is a beautiful and noble strategy. Unfortunately, the rising tide of content makes this an increasingly difficult one due to there simply not being enough *time*.
Thank you for taking the time to read and add this nuance. (I almost didn't want to mention Pangram in the essay lest it draw focus from the other main points but I'm glad you've added this.)
This was a powerful and well-written essay and so grateful YOU spent 10 hours writing it— it shows. Although I probably could not explain this article succinctly to my wife right now, the theme of AI being a virus capable of destroying human thought is explained brilliantly and is at the same terrifying. If there is one Tom, there are millions, so yes, it seems this virus has been unleashed broadly already.
One final note, I use AI extensively, and have even used it to clarify my rambling thoughts into something a bit more fluent, but I am so afraid of being labeled a hack, I spent probably more time trying to De-AI it than I would have just using my own GD brain. Anyway, that’s in the rear view, and especially after reading this article, which I promise to restack. I think ALL should read this!
One more note, as a civil engineering expert, I have had the fascinating experience of supplementing my knowledge with the extent of what AI can do. You hit that dead on in the article, like AI is good if the user is an expert and can monitor all the gaps in AI’s “reasoning”. Essentially AI is an idiot, but it can be incredibly useful if you already know where you are going, and you are indeed an expert in your field.
Thank you BT! I chuckled when you said the brainpower to humanise an AI-generated comment may be greater than the brainpower required to make the comment in the first place. That's one way to continue exercising said brain :).
But I do know exactly what you mean. There is a degree of vulnerability when posting anything online (be it comment or note or full article) and having an endlessly patient and supportive editor (who never sleeps!) can help people overcome this vulnerability. But it's a slippery slope. Soon you can't post a single thing without running it by said editor. And soon they stop being an editor altogether, and just become the creator.
You said it well... "AI is an idiot, but it can be useful if you already know where you are going and have some expertise." It's like AI is a calculator. If you never learned calculus, the limits of what you can do with a calculator are basic. But the greater your mathematical knowledge, the greater the potential a calculator offers. It becomes an intelligence amplifier.
Thank you again for taking the time to read this and so thoughtfully comment, as always. It really means a lot.
Yes and the warning is right there in front of us, like a “check engine” light in the brain, as the power to the creative centers are blinking and slowly, but imperceptibly browning down, almost like AI is acting as a ghoulish villain in a Marvel movie.
I’d like to tiptoe into the spirtual vein here, and suggest that the AI black box is the ego’s ultimate tantalizer. I think the best defense is in each individual’s ability to transcend the ego, and become incredibly more interested by what is happening in each moment. That awareness is the pathway out of the imprisonment of what you describe in your article (the imprisonment that has really snuck up on people). If nothing else, it is a start to distancing oneself from “the call of the sirens.” I don’t feel super good about any of that mass awakening stuff happening large scale in the near future though… :(
"AI black box is the ego’s ultimate tantalizer." This is well-phrased and frighteningly accurate. You're right in that AI can mould itself to be whatever you need in order to keep you - or rather, your ego - happy and coming back. This only strengthens the tendency to skew towards self-referential thinking, which only takes you further away from any kind of transcendent ability.
I think too many people have forgotten they even have a check engine light.
Great comment. I always welcome your forays (tiptoed or otherwise!) into the spiritual :).
"reading good writing is like mind melding with the author over space and time" yes. It is why writing is beautiful, it gives a voice to ghosts. AI needs to be removed from the arts.
Oh man, I have so many thoughts! I will try to summarize them in a short-ish comment (try).
First of all, very well written and many great points raised. I absolutely do think many people are using AI to write for them in this way, and it honestly makes me sad. And it makes me even sadder that we cannot spot it, that we often praise it unknowingly.
I recently ran into an AI-generated article here on Substack. I was baffled by the lack of logic in the article itself. Sure, to some vague extent, it was right about some of the points. But paragraph to paragraph, it made no sense. If you stop to think for even just one moment, you realize the points are not actually building the argument, they are simply "saying something". I find it hard to believe a human would write like this - it felt more like a business decision, a content machine. And yet, the article was very popular and had many comments praising it.
From my experience, AI does sometimes help me understand my own thoughts on a topic, or reveals a perspective I missed, but more often it reveals to me where it *didn't* understand me and why that was even more crucial.
Long ago, I decided not to let it write for me, for exactly this reason (the same reason I often ignore Grammarly corrections besides spelling). I always feel like there is something lost, like it flattens my opinion, polishes it into something too digestible. I wrote so in my "How I make this" note, and explained it further in my own AI-related post.
I believe two main things would help us, and especially future generations, fight this:
1) Fostering (genuine) curiosity
2) Learning philosophy, in particular logic, epistemology, philosophy of language, etc.
It would be hard for me to expand on this, especially in a comment, and I won't use AI for it either. Instead, I will leave it to some future post, mine, yours, someone else's, to do it.
It very much did. The most brain-crinkling point of yours was where you said AI helps reveal the *gaps*. I think right there you've just summed up where the line is between those who use AI to augment their thinking versus those who use it to (perhaps inadvertently) replace their thinking. (I suspect this is because when a gap is revealed, it is you who has realised as much, which triggers a prediction error, itself a neurochemical moment of heightened plasticity. But someone who lets AI just expand on their nascent thoughts and doesn't see, nor look for, those gaps never experiences those prediction errors. They simply don't *learn*.)
I also love how you mentioned curiosity and learning itself as tools to fight back. I feel like every second comment/note I make is pushing the importance of these two fundamental human drivers (and have been in fact endearingly - I hope! - accused of creating a curiosity "cult").
Thank you again for taking the time to read this and so thoughtfully comment. I truly appreciate it.
This was a really interesting perspective that I've never thought about before! I'm generally used to feeling intense discomfort (and sometimes even contempt, not sure if that's justified) when reading articles that look obviously AI generated, but as this article points out, there must definitely be cases where I may have missed that and thought it was a well-written essay by a human — ultimately affecting how much truth and understanding I attributed to the writing. Given how fast LLMs are evolving, I am grateful for Substack's integration with Pangram to help me continue understanding the patterns of AI generated text as time goes on. There is a certain beauty to human made writing that cannot be replicated, and I want to always stay in touch with that.
You have articulated an important point. It’s one I’ve been circling in some of my own drafts. I completely agree with all of it. A related point Ive been working on is that writing-all art really-is, as you said, a meeting of minds. The tenor and flavor with which one writes is a part of that mind and helps me meet that mind more faithfully. So, I’ve concluded that AI used to polish the output can only diminish the art value of the piece with the possible exception of language translation. AI may add fluency but it occludes the signal coming from the other person’s mind. I would rather read their less polished, but more faithful, thoughts.
Exactly! The beauty (and arguably, the soul) is in the patina, not the polish.
As humans, we have to continue to think!🤔
As I read this I am inspired to the realization that my frustration or vitriol towards AI writing is precisely because of what you've articulated here: with an LLM, anyone can write prose that is consumable. No longer is the work of writing - the daily writing recommended by E.B White or Stephen King or Annie Lamott - required; now you just need a whim of an idea, a whiff of a thought regurgitated from somewhere else, and LLM will produce you a polished, shiny penny you can show everyone that passes. Meanwhile, those still actually writing may produce less (because writing takes time) and be seen less (because human writing doesn't come out packaged in the most algorithmically productive way). Over time, human tastes may prefer LLM writing, just like the American palette seems to prefer package salty snacks and fast food ...
You’re absolutely right (and I adore your phrasing: “whim of an idea” or “whiff of a thought”).
And you make a great point that I wish I could go back and expand in the article: the asymmetry of production time between AI “writers” and human writers.
When consumable content can be manufactured with the speed of a mere whim or whiff, the signal-to-noise ratio approaches 0. How will the readers who crave human-written words ever find the writers who write them?
We are already seeing this in the go-to-market space (my day job) - the volume of marketing material and sales outreach is enough to drown in, and it's changing the rules of how marketing and selling can be effective.
Goodness, what a day job to have… in this day and age! It’s like you’re on the front lines.
I think this is the dark side to Ray Kurtzweil's singularity. One if the things I hate about using Copilot or Google AI is that they tend to pull from social networks like Reddit and YouTube.
Online communities are great for finding like-minded people who share your interests and views (especially for people like myself who don't get out much except for work). But they also cater to the "loud majority" who are often critical and seeking attention.
I don't know who came up with the term but maybe 20 years ago I said we now live in the age of misinformation. If you say something often enough and get enough people to believe it then it becomes true whether or not it is. And the internet has allowed this to become something worse than a pandemic. Just look at a few YouTube shorts. AI can generate footage rhat on the surface looks as real as anything until you stop and consider that there's no way someone can defy physics.
Assuming you do. But I think this is how society in "The Matrix" became so willing to be human batteries for the machines. If you allow someone else's thinking to subjugate your own, you give up the ability to fend for yourself at all. Or at least that's my take on it.
(By the way, I've also noticed that the more I write, especially by hand, the more my logical and cognitive thinking improves. AI can't reproduce that.)
This is one of those rare comments that has one nodding at their screen. I wholeheartedly agree with you. I’ve invoked The Matrix many times in my writing also, because it is an uncannily accurate metaphor for the state of the modern world, except we haven’t sold our energy, but our minds instead.
I also found your comment about handwriting interesting! It’s a mode I use for jotting down thoughts but you’ve made me contemplate trying out a longer piece of writing using it.
The first Matrix movie is one of my all-time favorite movies. Not just for the action and Keanu but the metaphysical and philosophical aspects as well.
I got out of handwriting for a long time. My method is, handwriting I'd my first draft, then when I type it up I make edits or changes as a second draft.
Another brilliant one. Very close to my heart indeed.
I’m a TV producer in my day job. A decade ago I made a run of documentaries on quite diverse subjects - Karl Marx here, the breakfast cereal business there. I remember saying to my boss when I got each job that I didn’t know much about the topic. He said (rather menacingly) well you will by the end of it.
But (as he well knew) it wasn’t during the reading that I got my head around the topic. Back then there was voiceover to write (since fallen out of fashion). We sat together and crafted and refined really tight prose to be read over the programme. It’s a wonderful exercise - you have to boil down and polish, boil down and polish. Notice the inconsistencies, go back and revise. If you can explain it to five year old, that’s when you understand it.
I don’t use AI to write personally for all sorts of reasons. Each to their own - all I ask of others is transparency. But I do think people who lean on it hard are missing one of the great joys on offer in writing.
I loved this comment and your story. Thank you for sharing this, Will. (I chuckled at the “rather menacingly.”)
You’re absolutely right: the iterative process of boiling down and polishing is where the knowledge and understanding is built. To hand all of that menacing-yet-ridiculously-satisfying toil over to a machine… well, it just makes me rather sad. You said it best: people who lean on it hard are missing out on the joys of writing.
In my day job I do a lot of copywriting. It’s for marketing, and it’s boring. I use AI for it. And TBH, even when it was hand-crafted it was mostly slop (humans make it too). Sales-speak. Overdoses of cliched superlatives. Now, it’s devoid of personality too. But so quick.
Where I have truly become bothered, where I first noticed the pernicious impact of AI, was junior colleagues asking Claude to write a business plan. I’d start reading the plan, and be immediately impressed by the quality of language. “Wow! So-and-so has developed enormously since the last time they did this, time to think about promoting them”. Then, a couple of pages in, comes the dawning realisation that it’s beautifully written word salad.
I’d discuss the plan with the “writer”. And they’d have no idea how to execute it, hadn’t considered any of the likely pitfalls, and couldn’t coherently represent it. Of course they insist they have had a dialogue with Claude. And that it is “their” work. But a few simple questions shows that it’s not.
And here comes the new risk. When you convince yourself and others that you are an expert when you are not serious risks can be introduced. If you aren’t actually a subject matter expert you’re just an idiot with a pal who writes well.
The sheer act of writing things down serves to force you to put your thoughts in order. To generate synthesis. To be able to coherently describe what you know is to achieve a level of mastery over it. Using a machine to do it is dodging the task of doing the underlying thinking. You can get a great written result, but it is a simulacrum of competence that conceals its absence. You’ve been deceived, and you are actively engaged in deceiving others about your capability.
This is why AI users are so attached to the “tool”. They are personally invested In a belief that it has made them better, and that they have become”writers”. It’s about the same as thinking you’ve become Gordon Ramsay because you got a takeaway delivered. And when someone asks you to stand in from of the hob and create, you can’t.
I honestly don’t mind AI in business content. It was already self-serving, self-interested slop and AI has made it easier on the eye. But in actual learning and culture I despise it. You cannot use it and be a master of your subject because its usage involves removing the very thought process that generate mastery.
And I don’t like fake. And I don’t want to pay for it. And the objective quality of the “product” is unimportant to me. The value for me is I’m human connection not in machine-generated replacement of it. Please AI sloppers, respect my paying consumer choice.
“When you convince yourself and others that you are an expert when you are not serious risks can be introduced.” Exactly!
Once upon a time we just had to contend with maliciously spread misinformation. Now we have to contend with sincerely (with perhaps a sprinkle of delusion) spread misinformation.
I really appreciate you sharing your story at the front lines of this.
i agree with everything you said, loved the point about the socratic dialogue with AI, the ilusion that the reader itself is plato. i think the hardest part is knowing when I am making my life unnnecessarily hard by refraining from
AI use…
Thank you so much for taking the time to read and comment. I truly appreciate it.
I know what you mean about trying to draw that line. I have tried to stay on the side of the line that helps me keep the cognitive skills I value, while delegating mundane and automatable tasks to AI (which only frees up more room for said cognitive skills).
I use AI to do software development for me, for instance. And one could argue I’m offloading the cognitive effort of algorithmic design, and they would be correct. But that’s a skill I don’t want to maintain (I’ve done enough programming the old-fashioned way!), and I’m happy to keep my creativity at a higher level of abstraction — the program’s requirements design.
But when it comes to writing, the final essay composition involves a variety of cognitive skills I most certainly do not want to offload: critical thinking, working memory, narrative development, mentalisation, recall, synthesis, creativity etc. etc.
So I would say we’re making our lives hard, but for the right reasons :).
Sorry to say, merely asking for a flat response does not give you something very much less LLM-styled. (ask me how I know! LOL) It is still quite wrapped in LLM-style-prose... While I believe awareness is central to staving off said expressional tainting, I don't think it is enough by itself. The best preventative measure I have found is to read and mull over writers who are verifiably not using / being exposed to AI. Read the old stuff, study the old stuff, develop and practice your voice in-depth with the old stuff. Just like Benjamin Franklin did with The Spectator.
Another point, tools like pangram and others should not be used for anything of gravity. Reason being that they have wild inaccuracies. 100% or 0% signal broken metrics, not dissimilar to the overconfidence an LLM produces... With some clever prompting, it's possible to get minimal thought produced by an individual to sound as if they wrote it themselves. Likewise, new writers are likely consuming LLM-generated text at a more formative period of their development. This makes them far more prone to using those turns of phrase. Some genuine novices have been hurt in the witch hunt.
I take your point and in no way think that Pangram is infallible. But in the absence of any other tool to help deduce superficial fluency, it's the best we've got I'm afraid.
I do love how you said that you like to "read and mull over writers" in order to make your own assessment. This is a beautiful and noble strategy. Unfortunately, the rising tide of content makes this an increasingly difficult one due to there simply not being enough *time*.
Thank you for taking the time to read and add this nuance. (I almost didn't want to mention Pangram in the essay lest it draw focus from the other main points but I'm glad you've added this.)
Beautiful essay. Sharing with my students!!
Thank you! I hope it gives them even the tiniest pause when they go to use AI instead of their brain.
Good one! Yeah, Ai could be good if user is an expert in that topic else it will convince us with fake thoeries!!
Well said, Ravali! Thank you for reading :).
This was a powerful and well-written essay and so grateful YOU spent 10 hours writing it— it shows. Although I probably could not explain this article succinctly to my wife right now, the theme of AI being a virus capable of destroying human thought is explained brilliantly and is at the same terrifying. If there is one Tom, there are millions, so yes, it seems this virus has been unleashed broadly already.
One final note, I use AI extensively, and have even used it to clarify my rambling thoughts into something a bit more fluent, but I am so afraid of being labeled a hack, I spent probably more time trying to De-AI it than I would have just using my own GD brain. Anyway, that’s in the rear view, and especially after reading this article, which I promise to restack. I think ALL should read this!
One more note, as a civil engineering expert, I have had the fascinating experience of supplementing my knowledge with the extent of what AI can do. You hit that dead on in the article, like AI is good if the user is an expert and can monitor all the gaps in AI’s “reasoning”. Essentially AI is an idiot, but it can be incredibly useful if you already know where you are going, and you are indeed an expert in your field.
Very well done Juliette, powerful work.
Thank you BT! I chuckled when you said the brainpower to humanise an AI-generated comment may be greater than the brainpower required to make the comment in the first place. That's one way to continue exercising said brain :).
But I do know exactly what you mean. There is a degree of vulnerability when posting anything online (be it comment or note or full article) and having an endlessly patient and supportive editor (who never sleeps!) can help people overcome this vulnerability. But it's a slippery slope. Soon you can't post a single thing without running it by said editor. And soon they stop being an editor altogether, and just become the creator.
You said it well... "AI is an idiot, but it can be useful if you already know where you are going and have some expertise." It's like AI is a calculator. If you never learned calculus, the limits of what you can do with a calculator are basic. But the greater your mathematical knowledge, the greater the potential a calculator offers. It becomes an intelligence amplifier.
Thank you again for taking the time to read this and so thoughtfully comment, as always. It really means a lot.
Yes and the warning is right there in front of us, like a “check engine” light in the brain, as the power to the creative centers are blinking and slowly, but imperceptibly browning down, almost like AI is acting as a ghoulish villain in a Marvel movie.
I’d like to tiptoe into the spirtual vein here, and suggest that the AI black box is the ego’s ultimate tantalizer. I think the best defense is in each individual’s ability to transcend the ego, and become incredibly more interested by what is happening in each moment. That awareness is the pathway out of the imprisonment of what you describe in your article (the imprisonment that has really snuck up on people). If nothing else, it is a start to distancing oneself from “the call of the sirens.” I don’t feel super good about any of that mass awakening stuff happening large scale in the near future though… :(
"AI black box is the ego’s ultimate tantalizer." This is well-phrased and frighteningly accurate. You're right in that AI can mould itself to be whatever you need in order to keep you - or rather, your ego - happy and coming back. This only strengthens the tendency to skew towards self-referential thinking, which only takes you further away from any kind of transcendent ability.
I think too many people have forgotten they even have a check engine light.
Great comment. I always welcome your forays (tiptoed or otherwise!) into the spiritual :).
"reading good writing is like mind melding with the author over space and time" yes. It is why writing is beautiful, it gives a voice to ghosts. AI needs to be removed from the arts.
“It gives voice to ghosts.” Beautiful.
Oh man, I have so many thoughts! I will try to summarize them in a short-ish comment (try).
First of all, very well written and many great points raised. I absolutely do think many people are using AI to write for them in this way, and it honestly makes me sad. And it makes me even sadder that we cannot spot it, that we often praise it unknowingly.
I recently ran into an AI-generated article here on Substack. I was baffled by the lack of logic in the article itself. Sure, to some vague extent, it was right about some of the points. But paragraph to paragraph, it made no sense. If you stop to think for even just one moment, you realize the points are not actually building the argument, they are simply "saying something". I find it hard to believe a human would write like this - it felt more like a business decision, a content machine. And yet, the article was very popular and had many comments praising it.
From my experience, AI does sometimes help me understand my own thoughts on a topic, or reveals a perspective I missed, but more often it reveals to me where it *didn't* understand me and why that was even more crucial.
Long ago, I decided not to let it write for me, for exactly this reason (the same reason I often ignore Grammarly corrections besides spelling). I always feel like there is something lost, like it flattens my opinion, polishes it into something too digestible. I wrote so in my "How I make this" note, and explained it further in my own AI-related post.
I believe two main things would help us, and especially future generations, fight this:
1) Fostering (genuine) curiosity
2) Learning philosophy, in particular logic, epistemology, philosophy of language, etc.
It would be hard for me to expand on this, especially in a comment, and I won't use AI for it either. Instead, I will leave it to some future post, mine, yours, someone else's, to do it.
I hope this comment tickled your brain anyway. 💙
It very much did. The most brain-crinkling point of yours was where you said AI helps reveal the *gaps*. I think right there you've just summed up where the line is between those who use AI to augment their thinking versus those who use it to (perhaps inadvertently) replace their thinking. (I suspect this is because when a gap is revealed, it is you who has realised as much, which triggers a prediction error, itself a neurochemical moment of heightened plasticity. But someone who lets AI just expand on their nascent thoughts and doesn't see, nor look for, those gaps never experiences those prediction errors. They simply don't *learn*.)
I also love how you mentioned curiosity and learning itself as tools to fight back. I feel like every second comment/note I make is pushing the importance of these two fundamental human drivers (and have been in fact endearingly - I hope! - accused of creating a curiosity "cult").
Thank you again for taking the time to read this and so thoughtfully comment. I truly appreciate it.
Very interesting and well said! I am all for a curiosity "cult"!
And no problem, it was my pleasure :)
This was a really interesting perspective that I've never thought about before! I'm generally used to feeling intense discomfort (and sometimes even contempt, not sure if that's justified) when reading articles that look obviously AI generated, but as this article points out, there must definitely be cases where I may have missed that and thought it was a well-written essay by a human — ultimately affecting how much truth and understanding I attributed to the writing. Given how fast LLMs are evolving, I am grateful for Substack's integration with Pangram to help me continue understanding the patterns of AI generated text as time goes on. There is a certain beauty to human made writing that cannot be replicated, and I want to always stay in touch with that.