Plato detested the written word. When an idea is spoken, it retains a close connection with its author. How could it not? In his dialogue Phaedrus, Plato points out that once written words are separated from their speaker, they can be read by anyone, they cannot answer questions, and they may provide the appearance rather than the reality of understanding. An idea captured in written language is sent into the world on its own, and must make its own way.
And now each of us can boot up a large language model and turn on an endless stream of words — our own Shakespeare faucet, Woolf pipe, or Angelou tap. Normally great writing is like a key that opens a door. Now, artificial writing seeps into every pore of society, taking Shakespeare, Woolf and Angelou, recutting their language into endless new keys, too many of which seem to open the same doors.
Over the last few months I’ve seen a lot of commentary on this. If you haven’t been following this debate, here are some of the major contours.1
Should We Let the Robots Speak?
Firstly, people differ on whether they are happy with using artificial words. You’re welcome to just hate them. Sam Kriss, for instance, vehemently despises them: “AI writing all tends towards a very specific mood. Poignant, wistful, simpering, dickless.”
Many sober voices push the other way. Richard Hanania argues that artificial words give stronger voice to those who were less able to express themselves, rendering the LLM as an crude emancipator of the unheard.
Let’s say you’re perfectly willing to allow silicon to assist you with writing. This creates another set of quandaries around how you should use it
.Jason Crawford prosaically writes that AI is great for researching a claim, but it isn’t as punchy with the same information. His argument is just to use it for research, much as you would use Google Scholar or Search in the past. This is a double-edged sword. AI will find good research articles and is a strong researcher (I use it frequently for this), but overuse could dent the efficacy of your own research skills.
This does leave open questions. If an LLM writes a section of my argument more coherently and concretely than I have, shouldn’t I just use those words? Is it important that all of the words come from me?
Dynomight writes that authors could just clarify what they’re doing with an LLM if they end up using one, and he takes no umbrage with people using artificial words and not informing anyone.
But I think it does matter to me if I’m reading something and the author uses artificial words. When I read anything, I usually apply a subsconcious fine-graining of reliability. A scepticism meter, if you like. A blog on the internet from a lesser-known author (like this one) will have been through less rigorous fact-checking than a report from the IPCC, and should thus score higher on the scepticism meter. The meter is also heavily influenced by other factors, like how much I know about the topic.
With writers I know well, I know where to set the meter. This is similarly true if I directly prompt ChatGPT. I know broadly how the words that it produces were constructed, and what potential strengths and limitations those words are likely to have.
Using LLMs to produce text and then placing this text under a person’s name makes this much less tractable. Part of the reason we value authorship is because it makes accountability clearer.
There’s one easy way around this. If we could easily discern artificial words from organic ones, this wouldn’t be a problem. Quizzes exist where you can test yourself on your ability to do this. Machines are often better at doing it, as they can pick up on imperceptible stylistic tendencies in text.
Some, like Sam Kriss above, claim that they can always tell. For the sorts of examples he gives, yes, I can tell too. But those examples are often taken from models that are old now, and the structures of artificial words change frequently over time. As time goes on, we will likely all end up in the camp of Malin Hay, who finds it pretty difficult to tell AI and human writing apart.
She also points out that many in the literary world prided themselves on not using artificial words, which meant, when the time came to tell the difference, they lacked the nous to do so. This may have been why we saw a literary prize being awarding a prize to an author who likely used large quantities of artificial words.
Regardless of who you are, telling the two apart isn’t that easy, as this Nature study shows:
“We found that AI-generated poems were rated more favorably in qualities such as rhythm and beauty, and that this contributed to their mistaken identification as human-authored. Our findings suggest that participants employed shared yet flawed heuristics to differentiate AI from human poetry: the simplicity of AI-generated poems may be easier for non-experts to understand, leading them to prefer AI-generated poetry and misinterpret the complexity of human poems as incoherence generated by AI.
By definition, the distribution of artificial writing closely resembles the distribution of human writing. There may be gaps at the edges, depending on which pieces of writing were tokenised and trained on. The trouble is that “writing that people value” is likely also the exact selection criteria that Anthropic used to pick the millions of books which they bought in order to remove their bindings, cut their pages apart, scan them into machine-readable text and then discard and pulp them.
So anything that people consider as good to great writing is likely to be in the training data. Of course, there are shortcuts out of the fine-tuned distribution that we can access - a quick one is to write the word fuck - but this only works because frontier models are fine-tuned. AI models are trained on the internet, after all. They are perfectly capable of generating some of the most horrific slurs imaginable if you take the training wheels off.
There are a number of software companies springing up to make the claim that they can cleave organic and artificial writing in twain, like Alexander and his Gordian knot. Freddie de Boer wrote this piece on Pangram, a piece of software which classified a significant proportion of text he wrote before 2019 as artificial.
This comes with the territory. You can increase the chances of your detection software catching artificial words by classifying more text as machine written. This in turn increases your false accusation rate when it comes to human text.
Take this study, a 2025 academic detection challenge. Several systems here caught more than 99% of artificial writing with a 5% false-positive rate. This was on a fixed collection of domains and model families, so it was a relatively controlled detection problem.
If AI writing makes up 1% of 10,000 texts:
In our big net we catch almost every artificial word, but because the sample of human-written texts is so large, we also accuse 495 humans wrongly. This means that only one flag in six, or approximately 17% of alarms, would be correct. It seems likely that most detection software will lean in the direction of catching more artificial text and so also towards irritating more humans. The alternative is to take a hit on your top-line accuracy rate.
There are more difficulties. An Epoch study notes that AI checkers are actually relatively good at catching artificial words, but if the AI writer is told to mimic specific human styles, the checkers can get worse at detecting it.
de Boer also points out that since the checker’s performance depends on the text that is put in and varies in performance with length, people who want to determine that a piece of writing is artificial can and will do so.
It is also a solution which carries its own problems. Mike Masnick discusses the danger when AI checkers are used in schools, noting that many students are currently being forced to prove that they are not using AI by having their writing subjugated to a model’s ruling.
Whether they were originally using artificial words or not, going through this process turns their writing into a cantankerous half-breed hobgoblin:
The false accusation resulted in the student subscribing to multiple AI services and studying how the detection systems work. Not because they wanted to cheat, but because they felt they had no other option for self-defense.
In an online test of human words against artificial ones, you have the advantage of knowing that you might be about to encounter text generated by an LLM. In real life, authors might freely blend machine and human text. Take the following passage, from a highly suspect article about the Bordeaux fires:2
There is ample evidence that free-ranging or semi-feral animals once did quietly, and for free, what fire brigades and water-bombers now do at enormous cost and under enormous strain: they kept the undergrowth down. The mechanism differs from place to place — a flock in Attica, a résinier‘s blade in the Landes, a vineyard’s edge in Castile — but the underlying story is the same one, told three times across a thousand years. Wherever animals and people stop clearing the land for a living, the landscape grows a memory of its own, in the form of litter, brush, and resin-soaked debris, and eventually asks to be burned.
Accordingly, what is smouldering above Bordeaux this summer is not only pine and gorse. It is the accumulated cost of absences we stopped noticing decades ago — the échassier gone from the heath, the résinier gone from the pines, the viticultor gone from a slope no longer worth the harvest. None of them left because of the coming fires. But their leaving is, in each case, why the fires came.
In all this, the more elegant solution is to let semi-feral animals once again roam the countrysides of the Mediterranean, blazing trails and grazing down the litter, thereby creating a more liveable and safer world in a situation where the abandonment of intensive agricultural practices has opened the way to yet more devastating wildfires. On a promising note, this is a debate currently running through the corridors of Brussels.
I’m fairly confident the first two paragraphs are artificial, and the last is organic. The tropes are everywhere, from the em-dashes, to this idea of the landscape growing a memory of its own, a phrasing which consistently erupts from the models when you ask them to write creatively. I put this into Pangram, which agrees:
It can be easier to tell AI writing apart in the long-form. LLMs still seem to lack overall conceptual abstractions, although the level at which they are capable of seeing a project or piece of writing seems to be moving out consistently.
So for the present, there are some traceable contours of the present distribution of artificial writing. Without overly specific instructions, current LLMs overuse specific vocabulary which is broadly positive but excessively vague. They love ruling something out before ruling something else in. They love a “slightly strange but workable” phrase (the quotation marks are important) and they love producing walls of text - walls of it - that are exhausting to read through.
Look at this sentence:
It is the accumulated cost of absences we stopped noticing decades ago — the échassier gone from the heath, the résinier gone from the pines, the viticultor gone from a slope no longer worth the harvest.
This is on the cusp of being a well-wrought sentence, but there are too many components. For me, the metaphor is malformed: the absences, much like the pine and gorse, are smouldering? What does that mean?
If I were rewriting it, I might slash it down, much like the unwieldy vine:
We are now feeling these absences: the échassier from the heath, the résinier from the pine, the viticultor from the slope.
I’m sure there are many among my readers could do a better edit than this, but I’m just trying to elicit those strange contours which delineate artificial and organic words. For now, LLMs seem to lack the ability to move seamlessly between different levels of abstraction in writing.
This mirrors the experience of software engineers, who find that LLMs are powerful coding agents provided the task doesn’t become too complex. Once the task requires a more holistic understanding of the work, there are hacks - you can deploy swarms of agents, or wrap different harnesses around them, or use a VectorDB memory system - but these are still hacks to get around the fact that the LLM doesn’t form an overall abstraction of the project.
Agent swarms often have to be prevented from fighting each other when editing files, which suggests that the LLM still might still lack an overall sense of what is going on. This doesn’t particularly matter if it has a human’s guidance, but when the human also lacks that ability in a field, this is one way to delineate artificial words.
When I describe the way that an LLM writes, I am probably describing my own prompting behaviour but through a glass darkly. An LLM behaves in a certain way—for me. With a different model, a different set of inputs, and different context, the LLM would behave differently.
The Machine Produces Words Like Rabbits Breed Rabbits
Once a certain standard of quality is met, the true problem of artificial words is their density. Making it overly easy to produce thousands of words has a good chance of drowning out any real voices from being heard. It might also help to divert the attention of readers into increasingly narrow funnels.
A few years ago I talked to someone who worked for a large consultancy firm, who said they were experimenting with having a version of a partner in AI form. Now, this is a start-up.
As I mentioned above, literary prizes have now been awarded to words that were likely artificial. If the literary elite cannot detect them easily, then how can we ever be sure of provenance?
Uri Bram writes that:
I’m also struck that during every moment of change, the relevant privilege group – that is, the people who had the fortune to already be good at the thing the technology helps with – will, by default, be aghast about the change; moreover, that even after hearing this argument (in general) they will believe that in this case their rights are earned.
So I think there’s something a bit treacherous about e.g. highly literate people who have spent their life benefiting from being good at spitting out words, assessing the merits of a new technology that is extremely good at spitting out words.
Agents and published authors, who theoretically choose what is canonically good and what gets media attention, are perhaps unhappy about artificial words because it makes their work more difficult, but it also creates fissures across the entire literary landscape.
The task of locating, accessing, and assessing which words are valuable enough to capture our attention is already a difficult one. Squeezing billions more words out of the machine and dumping them online is likely to exacerbate this problem.
As mentioned above, Richard Hanania suggests that this may level the playing field:
The fact of the matter is that wealthier and more established authors already have ghostwriters and research assistants. Some of us without great wealth have natural writing talent that similarly puts us at an advantage. AI can be seen as a leveler.
But levelling the playing field by increasing the number of players doesn’t always improve the sport. Having millions of competing alternatives, with little way to differentiate between them, renders the literary landscape completely impotent.
Real or No Real?
Let’s say I wanted to put in an interlude into all of my posts which conjures you somewhere else, for a moment of reflection and consideration. I might have a passage where I wanted to write a sentence which said something like: “It was a cool, sultry day in the alleyways of Baghdad. Even the shadows spoke softly, knowing that they too, would have to pay the toll of the wind.”
Or I might write: “For a moment, let us go elsewhere: to a distant city where evening gathers on unfamiliar rooftops, and the distance from our own lives makes their shape easier to see.”
Which of these is real? Does it matter?
In the Turkey City Lexicon, Rudy Rucker describes this type of writing as an eyeball kick:3
That perfect, telling detail that creates an instant visual image. The ideal of certain postmodern schools of [science fiction] is to achieve a “crammed prose” full of “eyeball kicks.”
Even if you could write an algorithm to distinguish these two lines of text, LLMs have been getting steadily better at manipulating text. Their overall architecture is so powerful that there must be a way through the slop. The writer Gwern is currently questing after better LLM writing, and I’d encourage you to read the sorts of poems you can craft with the right context.
Or try this, a story I created as part of an experiment for a start-up I’m working for:
“Welcome,” said the centre, in a woman’s voice neither young nor old. “This is the first evening of the Red Heat Protocol. Please keep documents and medical tags ready. We are currently projecting demand above safe overnight capacity.”
A sound went through the line, not quite speech: a tightening, a little communal intake, like the old bell had struck inside each chest.
Marlene Beckford heard “projecting” and thought of Mr. Avery’s maths room, 2B, where the boys once threw compass points into the ceiling tiles and somebody had written M.B. LOVES D.T. on her desk, though she had not. The entrance had been the girls’ entrance then. She could see, despite the new doors and the mist fans bolted overhead, the place where she had stood in 1978 with her blouse stuck to her back, waiting to be called in after pushing Sharon Keene near the cloakroom pegs. Now she leaned on her trolley, with its one squeaking wheel, and tried to make her breathing look ordinary.
The old lineaments of artificial words are latent there in that second paragraph, but the rest of the story disguises it. Try this:
Mara tightened her grip on the trolley handle. Inside were two nightdresses, blood-pressure tablets, a paperback with softened corners, and the framed photograph of her sister she had taken down at the last minute, unable to leave Audrey facing the heat alone.
At the scanner, Tariq held out his tablet. “You’re pre-registered. Flat on the top floor, living alone, cardiac history, yes?”
“Does it say I dislike being summarised?”
“It says you’re priority band amber.”
“I was hoping for emerald.”
“It’s not a prize system, Miss.”
“No,” she said. “I remember prizes. They were rigged too.”
His smile faded. He tapped the tablet with his thumb. “Go through. Medical station first.”
Maybe these lack the complexity or the timbre or the psyche to be great writing, and maybe if I went deep enough, I could find the original source of these words, in the same way that an LLM took the phrase “a democracy of ghosts” from Vladimir Nabokov’s novel Pnin. But I enjoyed this writing, and it wrote about a space I needed a story about—in this case, about a heat-wracked London in the near future.
It feels like a catchphrase at this point, but we’re only at the beginning. There are already interesting tricks you can do to squeeze more interesting writing out of these machines. Count Bayesie suggests basing word sampling on future-entropy, which allows the text to oscillate rhythmically between both predictable and strange words. Each model offers a new set of affordances for such experiments.
Step Right Up For the Amazing Artificial Extravaganza
Certain silicophile communities seem to assume that we are close now, closer than we have ever been, to prompting an LLM with a phrase like ‘push the literature forward on this topic’, hitting enter, and watching as in five minutes, a cacophony of text is tokenised and warped through cables through the internet to a data centre in deep America, where GPU chips perform vast amounts of matrix multiplication, before reversing the process and sending a complete response to your screen which suffuses the world with a new warm glow of understanding.
In the last few months, it has felt like we are slipping in that direction. In this conversation with ChatGPT, the mathematician Dmitry Rybin basically says a version of “find a proof” four times, until eventually it does so.
Or take this comment from one of the moderators of Less Wrong (LW) about the development of breasts:
We are not yet at the point where I can ask an LLM to write me a first-rate LW post on an arbitrary topic, e.g. boobies, and get a first rate LW post. For now, it’s an impressive accomplishment: this is a great LW post about boobies.
We will ever reach this point? I am concerned that we will and humans will be flooded with words and data that we could spend a million years with and never make sense of, let alone integrate the ideas into science or society.
Plagiarism
LLMs now perform a form of verbal regurgitation with great dexterity, which many people consider plagiarism, even if District Judges believe that this counts as “fair use”.
Does it matter if LLMs are plagiarising? Our concept of plagiarism is relatively modern, as Terry Eagleton has recently argued. But it is clear that at least as far back as the ancient Greeks, authorship has mattered. It mattered to the Greeks, and they had few reliable ways of reliably transmitting the original authors text.4 The source of the words has and always will matter.
Thus, large corporations hoovering up other people’s work and offering a stream of words which come seemingly from nowhere is a bad thing, as ChatGPT 5.6 Sol notes:
“Something economically valuable was taken: access to a vast human-made cultural corpus... ... The unresolved question is whether today’s rules distribute the resulting power and wealth fairly. I do not think it is credible to say that they clearly do.”
Words are a way of engaging with others, and this process will damage the quality of the writing that we can access. Already we are seeing people forced to move their creative work behind paywalls, away from AI models which extract, summarise, and output elsewhere, without contributing anything to the producer. It was bad when Facebook did it, but at least there was a chance that people clicked on the source material. This is worse.
Creating a machine that vacuums up other people’s words to pump them back into the world, but mostly in confined spaces which discourage interaction with others is damaging. Stories, recipes, ideas, arguments, and craft must all be shaped and bounded around the needs of people. They also require dialogue.
I have seen people say that they used to discuss ideas with others, but now they only discuss them with Claude. By design, Claude isn’t going to push back, or challenge you, or to make you think about a topic you don’t want to, or to make you feel uncomfortable. Humans will. Besides, ideas have no function in a vacuum.
Don’t get me wrong. I like LLMs, and I use them regularly. I just believe that there is a great danger in allowing large companies to extract information from the internet without offering any meaningful return contribution.
The work of writers, now humdrum training data, was not free to produce. I don’t believe that any of the large AI companies could afford to properly compensate the people whose work they used. However, they should be engaging with the world of words. If this training data is so important, they should be supporting literature, much as they should be supporting the world of open-source code which their platforms are also built on.
Proper licenses may emerge as a solution for more established writers, but for others, it would be good to see large AI companies recognise that their models have been extractive, and invest in funding cultural enterprises. As far as I’m aware, human data is still needed to prevent model collapse, and so extractive behaviour in the literary ecosystem is likely to damage the future capabilities of models in general.
And What About the Future?
For me, the future of writing seems to depend on a couple of important dimensions (read these as possibilities, rather than predictions!).
Firstly, it will depend on whether the quality of artificial writing continues to improve. Currently it is possible to discriminate between the human words and artificial words, even if it is not consistently possible. LLMs seem to go from strength to strength in areas with easier verification such as code, but it may be harder to evaluate and tune writing, although I doubt it.
If they continue to improve, we might enter an ecosystem where authorship ceases to matter, and text is deemed important by who is willing to put their name to it. Even if LLM writing remains at its current quality level, it will continue to seep into the world. Silicon Valley is empty and the words are here.
Secondly, if provenance continues to matter, we may see increasingly drastic ways of signalling the veracity of given text. Thirdly, if artificial words continue to proliferate, the sorts of discussions that currently happen mostly through writing on the internet may end up being displaced. This displacement may take two forms.
The first is displacement of the medium, as writers increasingly move to high-provenance environments, escaping the profusion of LLM text by becoming YouTubers or Podcasters or Greek Bards. The second is displacement of community, as writers may end up writing for smaller groups of followers who trust them and their writing, behind pointier walls such as Discord or paywalled Substacks.
I’m not sure what the future holds. It feels as if we are in that moment in The Bridge on the Drina, when the Austrians arrive and take over Višegrad from the Ottomans:
Naturally here, as always and everywhere in similar circumstances, the new life meant in actual fact a mingling of the old and the new. Old ideas and old values clashed with the new ones, merged with them or existed side by side, as if waiting to see which would outlive which. People reckoned in florins and kreutzers but also in grosh and para, measured by arshin and oka and drams but also by metres and kilos and grams, confirmed terms of payment and orders by the new calendar but even more often by the old custom of payment on St. George’s or St. Dimitri’s day.
By a natural law the people resisted every innovation but did not go to extremes, for to most of them life was always more important and more urgent than the forms by which they lived. Only in exceptional individuals was there played out a deeper, truer drama of the struggle between the old and the new. For them the forms of life were indivisibly and unconditionally linked with life itself.
I will use the distinction artificial words and organic words in this article. This is partly because it is strange cleavage. If an AI writes text and then I write that same text, is it human? Perhaps I should say artificially produced words and organically produced words, but that also feels wrong, as all artificially produced words are produced in that way because they were once organically produced.
For our purposes here, artificial words are anything that has been through the models, while organic words are anything that comes directly from a human mind.
I’m not directly linking to this article because it is riddled with inaccuracies, such as the first line, which claims that there were plans to evacuate the city of Bordeaux (no such plans exist). If you’re really curious, it is here.
For a wonderful treatment of this topic, I’d recommend The Cambridge Handbook of Literary Authorship, particularly Ruth Scodel’s chapter on the Greeks. Authorship may be purportedly detached from the performer - both the Iliad and the Odyssey, in Robert Fagles’ translations, famously open with an invocation to a higher power to sing:
Rage--Goddess, sing the rage of Peleus’ son Achilles,
murderous, doomed that cost the Achaeans countless losses
The Iliad
Sing to me of the man, Muse,
the man of twists and turns
The Odyssey
But it is the poet who gives form to the words, as Scodel writes:
Furthermore, Greek heroic poets mystify the process of turning inherited material into new performances by calling on the divinities who supervise their performance, the Muses. The singers’ reliance on the Muses, however, did not limit at all their responsibility for their performance. First, in Homeric epic, at least, the Muses’ function is not primarily to make the song aesthetically powerful, but to ensure its accuracy. Second, since in archaic Greek thought any outstanding achievement required divine favor, and that favor was itself typically a response to the excellence of the human agent, the singer who received help from the Muses was even more praiseworthy.
I have no experience or confidence with writing poetry. But in a recent Substack (https://tomrearick.substack.com/p/we-are-not-the-smartest), I needed a limerick that did two things: describe Plato transcribing Socrates' speech and illustrate how rhyme provides an auto-correction function. I think Google's Bard nailed it.
There once was a sage named Socrates,
Whose teachings were quite a shock.
He questioned the wise,
And opened their eyes,
And Plato wrote down all his ****.
I sense a contradiction here. On the one hand we mourn displacement of human writers and on the other hand ("artificial words give stronger voice to those who were less able to express themselves") we praise giving the less talented a voice. But we're used to it. Social media starting with Facebook allowed everyone however marginal to broadcast their misshapen words into the public domain.
Predictions are difficult, especially about the future, but I see an increasingly higher proportion of those unable to think for themselves, ruled by the technocrats who are able to think.