The Machine John Smith is Using to Think for Him is Wrong about Why No One Wants to Read Its Essays

Reflections on my least favorite account on Substack

This picture is all that remains of John Smith on Substack, a threat perhaps

Note: After I wrote most of this post, it appears that the account in question was nuked by the Substack team. It’s great that this happened, but he will be back.

The machine behind the man

John Smith is not a real man, and yet he has begun influencing many. His words are not real either, and yet you can read 5,000 of them every day.

John Smith is not a real man, and yet he influenced many. His words were not real either, and yet you could read 5,000 of them every day.

I felt something was up the first time I encountered him, in a post titled “Abortion is Fine” under the blog called “The Second-Best World.”I thought it was so painfully boring. For an earlier blog post, I took the time to go through hundreds of small accounts. I was shocked by how many new accounts are using AI to speak for them, to take their inputs and return something lifelessly polished. I began to notice some patterns in LLM-generated language. Everyone knows “it’s not X, it’s Y”, but there are many, many more signs. X happening while Y quietly happens. The gap between X and Y converging with the intersection of X and Y. X is the key that unlocks Y, it flattens Z, stops functioning as AA, and that’s the point. It’s tough work being X, but at least it’s quite repetitive.

No single tell is sufficient for determining something was written by AI, but with enough, and at great enough density, AI use at present becomes undeniable.

John Smith’s writing certainly had some of these tells, but it was less overtly obvious than most other AI writing. Still, after only a couple of minutes, my eyes started to glaze over in the way they always do whenever I happen upon the same stupid, omniscient narrator that has become so prevalent now in online discourse.

A week or so later, another post of his was in my feed. I sighed and looked at it. “A Trillion Mediocre Lives Are Better Than Yours”. This one was arguing that the repugnant conclusion is good, actually. Because I like dunking on utilitarianism for often leading to unintuitive, monstrous conclusions, I was excited to read someone gleefully accepting one of the most intuitively monstrous. I began reading and minutes later my eyes glazed over again. Someone making stupid points about a subject I care about usually captivates me, but once again I just couldn’t do it.

I looked at his account and noticed how between the two posts I saw, he had posted about a dozen others, one per day, each a 20+ minute read, and covering all sorts of disparate topics. Truly, I didn’t need to wonder how that was possible.

I posted “How does Substack’s new AI philosopher manage to copy/paste 5,000 words a day? So impressive! I can’t wait to have ChatGPT read what he writes tomorrow.”

The man behind the machine behind the man

I wasn’t the only person to notice. Other, more confrontational writers pressed him on it, focusing on the quantity of polished writing more than the latent characteristics.

Pascal Smith was the first person I saw to accuse him of AI use, though he did it indirectly, saying, essentially, that his writing is too prolific and so AI seemed the most plausible explanation.

Amos Wollen directly pressed him on the quantity. How long must it take to write 5,000 words daily of completely polished prose spanning all sorts of different topics? Smith’s response was essentially to immediately concede that his work was not just AI-assisted, but entirely, 100% AI generated. Entirely untouched by human hands once the prompt was submitted. This was a shocking admission for someone who had previously evaded the question or showed up and repeated “boop beep boop beep yeah i’m totally just a robot haha” (so charming).

But with the revelation came Smith’s defense: “This was just a social experiment… uh, if you could just be honest you would admit that this writing is better than most of the writing on Substack…uh, This is the future. Yeah, this is the future and you better get used to it, okay??”

Unsurprisingly, people did not want to get used to it.

It’s funny that John Smith, the second he spoke for himself, immediately made himself out to be an immoral fool while also losing all the cultural influence he had worked so leisurely to attain. I understand why he had a computer speak for him – in his case, it may be the right call.

So after losing the thing he was trying to attract with his machine writing, i.e. human attention, he rebranded, changed the name of his publication and persona, and kept going. The process was far too automatic to flip a switch and stop it, I suppose. He kept publishing essays, taking credit under either his real name or a second pseudonym for the writing of LLMs. After a week of terrible engagement, he published an article titled “You Liked it Until You Knew”. In it, he used those LLMs to defend his use of those LLMs and all his deception. This is obviously pathetic, but a wise move for him since defending himself went so poorly.

In this article, I presume¹, his bitterness was softened by the machines as they chastised ears who were no longer listening for caring about more than just the content of the prose. The obvious question to be asked, is if the only thing that matters about words is how correct they are, why does he want so desperately for human beings to appreciate them? Why not just write for machines—the machines he believes are smarter than people anyway? They would analyze the words for what they are, without judgment.

Why John Smith is Wrong—And Why it Matters: The Social Experiment that Fooled No One:: What C.S. Lewis has to Say About Letting a Machine Talk for You—And What it Says About Us—And Why it Matters

²Many people have written about the way AI will change writing. I’ve contributed to this, focusing on how we ought to respond to new AI-powered exploits. I found a new exploit back in January, saying at the time that it is not enough to respond reactively to this new technology as it will advance much faster than we could possibly hope to react to.

So here we are, confronted with a new possible exploit and once again needing to respond reactively to it. In just four months, LLMs have gotten good enough that a clever person can convince a significant number of people, some clever themselves, that they are a real person writing with sophistication. But before we figure out how to respond, we need to figure out exactly what the problem is here.

Well, is there a problem? Is it okay if someone uses AI to write entire essays? Is the problem just the quantity? Had Smith lowered his cadence to twice weekly, would anyone have accused him? I think yes, though it would have taken much longer and accusations would have been more cautious. Both Sides Brigade focuses on the stolen valor of the effort, saying “all ‘John Smith’ is doing is just using AI as a shortcut towards a cheap simulacrum of what actual human beings normally produce…it’s very clear to see that ‘John Smith’ here wants the experience of putting out pieces that others enjoy and engage with…”

To me the clearest problem and the one on which everyone seems to agree is that John Smith wasn’t forthright about his scheme. Indeed most of the harshest criticism of his actions have been directed at his lying and manipulation, to be sure a critical part of what he calls his “experiment”³.

Smith himself thinks the problem was he left the essays too polished, too good. He thinks AI is too good at writing well and it makes people insecure.

Now, I disagree with Smith, but only because I think his “writing” is hideously ugly, like the uncanny valley begging me for attention. Matt Whiteley made my favorite point on this, saying “...everyone has been pointing out how obviously AI his tedious, short sentence rammed prose was. As far as I am concerned, [AI writing] may improve in the future but right now AI writing remains akin to the donkey at the end of C.S. Lewis’ Last Battle who puts on a lion skin and pretends to be Aslan. You can fool a few people, but if anyone catches you in broad daylight and looks twice it’s still going to be obvious.”

We’re Still in the Best Possible World

We need to be clear about what it is that makes writing valuable; what we are and are not willing to tolerate as AI continues to become a more and more persuasive human facsimile. The problem will only get worse, as Smith has demonstrated through what he is now embarrassingly calling a “social experiment” (Prank’d ya!,Look at the cameras, wow you look so stupid!). At some point, even discerning readers like Whitely will be fooled, at least occasionally.

So at what point does it stop mattering whether there is a real human behind the words?

I think the answer is “never”.

Joseph Rahi made the point that this saga has shown that it is not just the words that we care about, we care about the human elements that formed them. I think this makes a lot of sense, “the key that quietly unlocks the whole puzzle”, if you will.

So if your friend beats you in chess, you may want to practice to get better than him. If you find out that he won by using a computer, you’ll be angry and think less of him. The goal of chess might be to win the game, but the purpose of playing chess is not. By using a computer, your friend might achieve their goal to win but they have forsaken the game’s purpose and in so doing robbed you of both. Put differently, if I watch Magnus Carlsson play chess, it damn well matters that his human brain is what is moving the pieces. It’s why cyborg-baseball will never replace the glory of bananaball⁴, no matter how many home runs a robot can hit.

The goal of writing is to write well, to write persuasively. But the purpose is to impact people, to persuade them. And humans will always have a massive advantage on this front. This is why the lying element has drawn the most ire. Not that AI itself is bad, but rather the deception. The meaning of words changes when they were written by a human rather than a machine. To claim the former while using the latter is to lie to everyone you are trying to persuade. The whole game falls apart. Tell me if I’m reading an AI and let me decide for myself if I want to keep reading it. Otherwise the meaning changes and I as many other people will feel deceived.

What John Smith does not see in his desperate attempt to justify his actions, is that words spoken from human lips, written with human hands carry greater value to human ears and human eyes—the very ears and eyes he, ironically, obviously cares most about. They carry a different sort of value that words generated by a machine can never replicate. When the human mind produces words, it fills them with something ineffable; the words convey more than just their academic definitions. They tell us about something else, about the author, their life, their perspective. We wonder what they’re really thinking, what they really mean, why they chose the words they did. When you remove the person and all that life they put into their words, those words, no longer a vessel for something greater than themselves, are shown to be empty.

LLMs will continue to get better. Clever people will continue to be able to use them to create fragile names for themselves. It will get easier for them to do it, too. Maybe they will be more patient than Smith, less greedy. Maybe they will take time to put in even a slight human touch. But they will still be frauds. They may look more and more like lions, but underneath they will still be dishonest, little donkeys, hiding behind pseudonyms and hoping everyone eventually believes that all that matters on a page are the words, not who wrote them.

John Smith is working tirelessly cynically to create the second-best world, but he is telling everyone that we are there already and there is nothing we can do about it. My prayer is that we do not get used to it. That when someone lying about their AI use and masquerading as Aslan has their mask pulled down, we let their empty words fall to the ground and shatter, too.

JFS

---

You May Also Like:

1. I couldn’t make myself read it and now his whole account has been locked down.

2. I thought of this heading all on my own

3. Funny how these sorts of experiments always happen to benefit the experimenter at everyone else’s expense

4. They’re selling out college football stadiums now btw