The funny thing is that, as a "social experiment", we actually got some interesting results from it. They're just not the results John Smith wanted or expected. Turns out (almost) everyone actually cares a lot about the person behind the writing.
I was very heartened that most of his readership seemed to lose interest in his publication after he announced it was all AI-led. I’m hopeful that we keep thinking that the human effort that goes into words does make them more valuable.
As a side, is the shell you are in a reference to something?
My name is just a minimally conceived mash-up of the gnostic entity Sophia and “Ghost in the Shell.” Unfortunately would be a pretty solid name for an AI account itself, but I promise I’m not lol!
John Smith’s rationale for doing what he was doing (social experiment) seemed post hoc. And if consideration of his readers was a concern of his, he obviously should’ve disclosed that AI was doing the writing up front. Furthermore, people who use Substack for a social experience have every right to hate the idea of interacting with AI content.
But this idea that he was “taking credit” for AI writing isn’t as clear cut to me as you argue. His account was obviously anonymous and free, like so many accounts on Substack. His insane output was clearly impossible for a human and he never once claimed to be human that I know of.
The first post of his I read was probably his least controversial on the history of MAID policy and practice in Canada. The post was exhaustive and very even handed. I know a little bit about the subject and still learned quite a bit from reading it. It was additive. I also believed at the time I read it that AI had played a big role in writing it.
When I compare this post to a lot of other posts written by human authors on Substack that contain sloppy reasoning, gross distortions, falsehoods, and incoherent arguments, from a content standpoint at least, it was more honest, fair, and informative than average.
It’s also worth noting that a lot of the people who ended up criticizing him most harshly “liked” many of his essays. I think this tension is worth examining.
John Smith certainly wasn’t some righteous crusader looking to reveal to us our own hypocrisy as he claimed after the fact. But despite being less than transparent (which is true of many of us here in other ways) the harm he caused doesn’t seem so out of proportion with what you typically see on Substack from human writers being less than completely honest.
I don’t know. People have a right to feel misled, but I’m not crazy about the unchecked sanctimony that’s been building in anti-AI writing circles either. It’s particularly tricky because if you defend some AI writing as worthwhile or useful, you risk being accused of using AI yourself.
"When I compare this post to a lot of other posts written by human authors on Substack that contain sloppy reasoning, gross distortions, falsehoods, and incoherent arguments, from a content standpoint at least, it was more honest, fair, and informative than average."
Sure, but you could say the same for the typical New York Times or even Wikipedia article. Something AI writing lacks is a strong point of view. We read on Substack in part to see different perspectives, and to interact socially to refine our thinking. If I wanted the AI summary of something I can (and do) prompt Claude to write it for me.
I'm ok with AI-assisted writing of certain kinds as long as that is disclosed and the human is still involved enough that you can reasonably call them the author. But too often AI is used without transparency, as a replacement for the human entirely, or as a lazy substitute for thought.
I guess I’m just more comfortable rejecting something written for being poorly written. I’m not as concerned with what exact process or combination of processes produced the words as the words themselves.
I’m a big fan of Ray Carver short stories. If you look at most of the manuscripts he submitted to his editor Gordon Lish, they‘re nothing like what the highly praised final drafts were after Lish edited them. People also praise Carver’s gritty autobiographical style, for drawing on his personal struggles with alcohol and depression, but I have no idea to what degree that was true. Nevertheless I think whatever we call Ray Carver writing is good writing.
The issue with John Smith is similar to someone who reads a memoir thinking it was true when the record shows later there was a ton of embellishment. Was it still a good read?
I’m dubious of people who say they love writing for the effort that went into it—that they reject lazy writing. I put a shit ton of effort in my earliest unpublished short stories, more so than most of my later published ones. But those early high-effort stories were poorly written and were rightly rejected.
John Smith essays weren’t great essays. Some of them were better than others. Substack essays in general share this quality. And the worst human Substack essay is worse than the worst of John Smith’s. Once the man behind John Smith started writing himself, I thought he read as manipulative, petty and inauthentic—far worse than whatever his prompts + AI instructions produced.
As I said, I understand why people would rather he disclose that the essay being posted was the result of a prompt. Any considerate poster would be forthright about this for the sake of those people.
My opinion that writing should be judged on how the words fit together rather than how the words were produced is the minority view now. And that view is increasingly seen as distasteful. It’s more likely I’ll have to conform to how you and the author of this post view writing rather than the other way around. I’m working on it, but it’s a big adjustment.
I get what you are saying and agree that AI has made us reexamine whether the words themselves contain all the value or not. I certainly would have leaned more toward your view a couple of years ago, but AI being able to replicate human speech as well as it does has changed my mind to a large extent.
I think there are contexts when it really is just the words that matter (like research), but most think pieces and Substack blogs benefit greatly from having a strong perspective.
The contrast to the John Smith problem is Venkatesh Rao's account, Sloptraptions. He explicitly acknowledges that his essays there are AI-written, but he's involved in the entirety of the process and the end result has a surprisingly strong point of view, and a lot of his own personality in it. I'm not always as engaged by his Sloptraption writing as I am by his other writing, but it's easier to lose myself in those than a standard, thoughtless ChatGPT essay.
I think the principle that best explains the difference is "investment." When I read a good novel I know that on the other end there is an author who put themselves into the work. They valued it. They put time into it. And that gives me a certain degree of safety in losing myself in what they write, because I know this is a two way street.
Smith, on the other hand, wanted to extract people's attention and admiration as if it was a resource that he could harvest with minimal investment of his own. There's an asymmetry there. It's because to some degree when we are interacting with people we are looking for worthiness--a suitable social partner, basically.
And when a person broadcasts a lot of traits that we think are unsuitable, like a complete disrespect for our feelings, it makes them repulsive. In that sense, the description of what's wrong with John Smith is probably better described by relational metaphors than ones related to the information in text production. It's like a guy who puts a token investment into a date, like paying for dinner, with no extra attention to his partner's opinion or feelings, no effort to build a connection or to understand, and then still expects intimacy in return. It's creepy.
Long story short, I'm fine with someone who uses AI in a worthy way. I don't think Smith was it, though.
I haven’t followed the John Smith saga, but I’ve already set out my thoughts on the (human vs machine) origins of writing elsewhere. Transparency is important, but no, I don’t think human origins always matter the same.
As for volume and speed, I think there’s definitely something there, but it’s also important to remember that people really do produce at different rates. There are life constraints, some topics are harder than others, and then…there is the matter of facility. If you can’t produce the level of quality that you want (as in, literally what you want to put out there, not what a non-dumb reader should be willing to tolerate) without finding it extremely laborious, then…WTF are you even doing trying to be a “writer”? If you want to be deemed good at something, then consider that if your 1RM is someone else’s warmup, you need to grind till you’re not an embarrassment. Conversely, if someone who has an impressive 1RM is side-eyeing yet another person as possibly using fake plates…consider that maybe those really are fake plates.
(Generic “you” above; none of this is a dig at you, btw. But there are certainly those whose claimed heroic efforts make me chuckle quietly from behind my screen.)
I believe that human origins do matter in the very many contexts where the goal isn’t strictly being as correct as possible and the audience is human.
Just as an example, I’m happy to read this note and respond to it because I’m familiar with you and as close to sure as I can be that you are a real person. If this comment came from someone else who I suspected just generated it with an LLM, I wouldn’t bother.
Put differently, if going forward you trained your account on all your writing and it did a very good job mimicking you, even if you were transparent about that I just wouldn’t be interested in reading the generated thoughts of a Felice-clone (at least once the novelty ran out).
I found the defense that the "writing was better" hilarious because I clocked it as AI from the first article I read. They are terrible and very obvious sterilizations of other people's bad ideas.
Totally agree, but I also think it’s pretty clearly getting better. I hope it’s the case the careful readers will always be able to tell, but I’m not so sure
oh yeah, no even just a little bit of effort makes it pretty hard to tell whether a piece of writing is AI or not. I just found it pretty deliciously ironic that the guy trying to level a social critique on how superior the quality of AI writing is could not discern that his AI writings were extremely flat and painful to read.
Just want to poke my head up to say that volume itself is not always a tell. When I really get into the maniac-writing-to-quell-the-terrible-voices-rising-from-the-whole-in-my-heart state, 5000 words in a day is not too uncommon for me.
I mean yeah, Marcel Duchamp put a urinal in a museum 100 years ago. This isn't a new experiment, Mr Smith. Is a machine made toilet as good as a human made sculpture? Some people say yes!
I'm fascinated that he chose to admit he was AI so easily. Surely the obvious lie was, "Yeah, I wrote a bunch of essays before starting Substack and I'm posting them at a fast pace to build readership. I'll be through my backlog in a week or two." And then slow down the posting to something more realistic.
I kind of assume it’s still writing, and I wouldn’t even care if it were doing it here on Substack so long as there was a banner on the page that said it’s AI writing
Isn’t his note on Substack an announcement to that effect? I feel like the backlash doesn’t fully make sense if that’s the only thing people are upset about.
I personally am not really upset about that either. I don’t think it’s deceptive to write essays using AI like it would be to e.g. submit writing to an employer looking to hire you based on writing assessments. Visual artists still get credit for using computer tools to make art and they don’t universally have to write some big disclosure in advance.
The funny thing is that, as a "social experiment", we actually got some interesting results from it. They're just not the results John Smith wanted or expected. Turns out (almost) everyone actually cares a lot about the person behind the writing.
Yep. I honestly didn’t expect his average post engagement to drop off a cliff quite as much as it did.
I’m not sure that would have happened on every website but Substack cares a lot about ethos, which I love!
I was very heartened that most of his readership seemed to lose interest in his publication after he announced it was all AI-led. I’m hopeful that we keep thinking that the human effort that goes into words does make them more valuable.
As a side, is the shell you are in a reference to something?
My name is just a minimally conceived mash-up of the gnostic entity Sophia and “Ghost in the Shell.” Unfortunately would be a pretty solid name for an AI account itself, but I promise I’m not lol!
I thought there might not be an actual shell, but I’m surprised you’re not an actual Sophia either lol.
John Smith’s rationale for doing what he was doing (social experiment) seemed post hoc. And if consideration of his readers was a concern of his, he obviously should’ve disclosed that AI was doing the writing up front. Furthermore, people who use Substack for a social experience have every right to hate the idea of interacting with AI content.
But this idea that he was “taking credit” for AI writing isn’t as clear cut to me as you argue. His account was obviously anonymous and free, like so many accounts on Substack. His insane output was clearly impossible for a human and he never once claimed to be human that I know of.
The first post of his I read was probably his least controversial on the history of MAID policy and practice in Canada. The post was exhaustive and very even handed. I know a little bit about the subject and still learned quite a bit from reading it. It was additive. I also believed at the time I read it that AI had played a big role in writing it.
When I compare this post to a lot of other posts written by human authors on Substack that contain sloppy reasoning, gross distortions, falsehoods, and incoherent arguments, from a content standpoint at least, it was more honest, fair, and informative than average.
It’s also worth noting that a lot of the people who ended up criticizing him most harshly “liked” many of his essays. I think this tension is worth examining.
John Smith certainly wasn’t some righteous crusader looking to reveal to us our own hypocrisy as he claimed after the fact. But despite being less than transparent (which is true of many of us here in other ways) the harm he caused doesn’t seem so out of proportion with what you typically see on Substack from human writers being less than completely honest.
I don’t know. People have a right to feel misled, but I’m not crazy about the unchecked sanctimony that’s been building in anti-AI writing circles either. It’s particularly tricky because if you defend some AI writing as worthwhile or useful, you risk being accused of using AI yourself.
"When I compare this post to a lot of other posts written by human authors on Substack that contain sloppy reasoning, gross distortions, falsehoods, and incoherent arguments, from a content standpoint at least, it was more honest, fair, and informative than average."
Sure, but you could say the same for the typical New York Times or even Wikipedia article. Something AI writing lacks is a strong point of view. We read on Substack in part to see different perspectives, and to interact socially to refine our thinking. If I wanted the AI summary of something I can (and do) prompt Claude to write it for me.
I'm ok with AI-assisted writing of certain kinds as long as that is disclosed and the human is still involved enough that you can reasonably call them the author. But too often AI is used without transparency, as a replacement for the human entirely, or as a lazy substitute for thought.
Yeah it seems really clear from this thing that at least my circle of Substack cares a whole lot about the perspective of the writer.
You’re completely right that it’s that strong point of view that is totally absent.
Thanks for the reply.
I guess I’m just more comfortable rejecting something written for being poorly written. I’m not as concerned with what exact process or combination of processes produced the words as the words themselves.
I’m a big fan of Ray Carver short stories. If you look at most of the manuscripts he submitted to his editor Gordon Lish, they‘re nothing like what the highly praised final drafts were after Lish edited them. People also praise Carver’s gritty autobiographical style, for drawing on his personal struggles with alcohol and depression, but I have no idea to what degree that was true. Nevertheless I think whatever we call Ray Carver writing is good writing.
The issue with John Smith is similar to someone who reads a memoir thinking it was true when the record shows later there was a ton of embellishment. Was it still a good read?
I’m dubious of people who say they love writing for the effort that went into it—that they reject lazy writing. I put a shit ton of effort in my earliest unpublished short stories, more so than most of my later published ones. But those early high-effort stories were poorly written and were rightly rejected.
John Smith essays weren’t great essays. Some of them were better than others. Substack essays in general share this quality. And the worst human Substack essay is worse than the worst of John Smith’s. Once the man behind John Smith started writing himself, I thought he read as manipulative, petty and inauthentic—far worse than whatever his prompts + AI instructions produced.
As I said, I understand why people would rather he disclose that the essay being posted was the result of a prompt. Any considerate poster would be forthright about this for the sake of those people.
My opinion that writing should be judged on how the words fit together rather than how the words were produced is the minority view now. And that view is increasingly seen as distasteful. It’s more likely I’ll have to conform to how you and the author of this post view writing rather than the other way around. I’m working on it, but it’s a big adjustment.
I get what you are saying and agree that AI has made us reexamine whether the words themselves contain all the value or not. I certainly would have leaned more toward your view a couple of years ago, but AI being able to replicate human speech as well as it does has changed my mind to a large extent.
I think there are contexts when it really is just the words that matter (like research), but most think pieces and Substack blogs benefit greatly from having a strong perspective.
The contrast to the John Smith problem is Venkatesh Rao's account, Sloptraptions. He explicitly acknowledges that his essays there are AI-written, but he's involved in the entirety of the process and the end result has a surprisingly strong point of view, and a lot of his own personality in it. I'm not always as engaged by his Sloptraption writing as I am by his other writing, but it's easier to lose myself in those than a standard, thoughtless ChatGPT essay.
I think the principle that best explains the difference is "investment." When I read a good novel I know that on the other end there is an author who put themselves into the work. They valued it. They put time into it. And that gives me a certain degree of safety in losing myself in what they write, because I know this is a two way street.
Smith, on the other hand, wanted to extract people's attention and admiration as if it was a resource that he could harvest with minimal investment of his own. There's an asymmetry there. It's because to some degree when we are interacting with people we are looking for worthiness--a suitable social partner, basically.
And when a person broadcasts a lot of traits that we think are unsuitable, like a complete disrespect for our feelings, it makes them repulsive. In that sense, the description of what's wrong with John Smith is probably better described by relational metaphors than ones related to the information in text production. It's like a guy who puts a token investment into a date, like paying for dinner, with no extra attention to his partner's opinion or feelings, no effort to build a connection or to understand, and then still expects intimacy in return. It's creepy.
Long story short, I'm fine with someone who uses AI in a worthy way. I don't think Smith was it, though.
I haven’t followed the John Smith saga, but I’ve already set out my thoughts on the (human vs machine) origins of writing elsewhere. Transparency is important, but no, I don’t think human origins always matter the same.
As for volume and speed, I think there’s definitely something there, but it’s also important to remember that people really do produce at different rates. There are life constraints, some topics are harder than others, and then…there is the matter of facility. If you can’t produce the level of quality that you want (as in, literally what you want to put out there, not what a non-dumb reader should be willing to tolerate) without finding it extremely laborious, then…WTF are you even doing trying to be a “writer”? If you want to be deemed good at something, then consider that if your 1RM is someone else’s warmup, you need to grind till you’re not an embarrassment. Conversely, if someone who has an impressive 1RM is side-eyeing yet another person as possibly using fake plates…consider that maybe those really are fake plates.
(Generic “you” above; none of this is a dig at you, btw. But there are certainly those whose claimed heroic efforts make me chuckle quietly from behind my screen.)
https://substack.com/@felicemish/note/c-255976958
I believe that human origins do matter in the very many contexts where the goal isn’t strictly being as correct as possible and the audience is human.
Just as an example, I’m happy to read this note and respond to it because I’m familiar with you and as close to sure as I can be that you are a real person. If this comment came from someone else who I suspected just generated it with an LLM, I wouldn’t bother.
Put differently, if going forward you trained your account on all your writing and it did a very good job mimicking you, even if you were transparent about that I just wouldn’t be interested in reading the generated thoughts of a Felice-clone (at least once the novelty ran out).
I found the defense that the "writing was better" hilarious because I clocked it as AI from the first article I read. They are terrible and very obvious sterilizations of other people's bad ideas.
Totally agree, but I also think it’s pretty clearly getting better. I hope it’s the case the careful readers will always be able to tell, but I’m not so sure
oh yeah, no even just a little bit of effort makes it pretty hard to tell whether a piece of writing is AI or not. I just found it pretty deliciously ironic that the guy trying to level a social critique on how superior the quality of AI writing is could not discern that his AI writings were extremely flat and painful to read.
Yeah, I think it’s also very funny that the second he stopped letting the machines speak for him he came across as a total jackass
Yes yes, but when are you going to make a reflection of your most favorite account on Substack (me)??
It’s too long, Substack won’t let me post it
Great post
Just want to poke my head up to say that volume itself is not always a tell. When I really get into the maniac-writing-to-quell-the-terrible-voices-rising-from-the-whole-in-my-heart state, 5000 words in a day is not too uncommon for me.
Totally agree that it’s not a conclusive tell, but it is an indicator since that level of writing is very difficult to produce for an extended period.
I thought it was funny at the time when Bentham’s Bulldog was like “guys this account sure writes a lot”.
I mean yeah, Marcel Duchamp put a urinal in a museum 100 years ago. This isn't a new experiment, Mr Smith. Is a machine made toilet as good as a human made sculpture? Some people say yes!
Haha, they’re certainly doing different things!
I'm fascinated that he chose to admit he was AI so easily. Surely the obvious lie was, "Yeah, I wrote a bunch of essays before starting Substack and I'm posting them at a fast pace to build readership. I'll be through my backlog in a week or two." And then slow down the posting to something more realistic.
i was never alarmed by the quantity. it’s easy enough to write essays upfront. the prose gave the lie (by omission).
I like his essays. Do you know if he’s still writing?
I kind of assume it’s still writing, and I wouldn’t even care if it were doing it here on Substack so long as there was a banner on the page that said it’s AI writing
Isn’t his note on Substack an announcement to that effect? I feel like the backlash doesn’t fully make sense if that’s the only thing people are upset about.
People were mad at having been deceived much more than they were upset at the idea that he would continue to publish essays
I personally am not really upset about that either. I don’t think it’s deceptive to write essays using AI like it would be to e.g. submit writing to an employer looking to hire you based on writing assessments. Visual artists still get credit for using computer tools to make art and they don’t universally have to write some big disclosure in advance.
I think you are in the minority there and hope that you continue to be
What can I say? I enjoyed his Vampire problem essay, I think it’s quality regardless of source.