:format(webp)/cdn.vox-cdn.com/uploads/chorus_image/image/65143874/AI_fiction_writing_GettyImages_1124658526.0.jpg)
I expect to suffer some degree of writer’s block pretty much every day for the rest of my life. I’m a journalist and a novelist; it comes with the territory. But I have a feeling I’m going to suffer less from now on, thanks to my new best friend, GPT-2.
Let me back up a bit: Six months ago, the research lab OpenAI created an AI system that generates text — from fake news to poetry — that in some cases actually sounds like it’s written by a human being. The OpenAI team has been rolling it out in stages, each time giving us a more powerful version of the language model they dubbed GPT-2, and carefully watching to see how we use it.
They’ve just put out the most powerful version yet. It boasts 50 percent of the power of the full version, which has yet to be released. As you can tell by trying it out for yourself, this model is already plenty powerful.
I was excited to find out what this AI system could do for me as a fiction writer. Over the past couple of years, AI has been creating some pretty striking music and paintings and even Renaissance-style selfies. While some artists worry that AI will put them out of a job — just as it’s expected to do for, say, truck drivers and factory workers — I’ve been more inclined to see it as a collaborator than a competitor. I don’t think AI will be good enough to write a superb novel on its own, but I do think it can be very helpful in a novelist’s creative process.
The mathematician Marcus du Sautoy, whose highly abstract work involves no small measure of creativity, sees things the same way. In a recent conversation with me, he expressed it like this:
Rather than both of us being out of a job, what I hope is that maybe we’ll be able to push ourselves in interesting ways as the AI becomes a partner or tool to extend our own creativity. I get so stuck in ways of thinking and sometimes I need something to kick me out of that. AI can help us behave less like machines and more like creative humans.
Du Sautoy’s articulation put me in mind of how some literary theorists define the role of art: It’s meant to enhance our perception of the familiar human condition — the thing we’re so used to that we’ve become blind to it — by making the familiar strange.
When artists present ordinary things in non-ordinary language, they’re engaging in the technique that Russian Formalist Victor Shklovsky called “defamiliarization.” (For the theory nerds out there: Defamiliarization is related to Derrida’s concept of “différance” and Brecht’s “alienation effect,” as well as Freud’s concept of the uncanny.)
I’ve long suspected that AI can be an incredible tool for writers precisely because it’s great at defamiliarizing our world. The human-ish language it generates can startle us into seeing things anew, so we can in turn jolt readers awake through that sense of -ish. And, for me at least, the new version of GPT-2 proves just that.
What the new AI system taught me about my own novel
I was curious about how GPT-2 would respond to the material in a work of literary fiction. So I did the first thing that came to mind: I fed it a few sentences from my 2015 novel, The Mystics of Mile End, which tells the story of a dysfunctional Jewish family dealing with mysticism, madness, and mathematics in Montreal.
In bold, you can see the sentences I wrote. What follows is how GPT-2 decided to continue the story:
She was muttering under her breath: “If not p or q entails not p and not q, and p is true if and only if r, then we prove that for every r…” Mr. Glassman kept right on studying like he didn’t hear anything, so I did the same. “And then if p or q is true, then what p means and what q means… And then we know that the word p…!!”
He felt the heat rising up to his face, but he didn’t care. There was no point in trying to say more about the world, the realness of it. It was too hot to think, too humid, too humid, too heavy. He was so wrapped up in it, he didn’t feel his fingers touch his eyelids, he didn’t need a mirror to see himself. His whole life was a dream, an illusion. He didn’t know how much longer...
Reading this, I felt strangely moved. The AI had perfectly captured the emotionally and existentially strained tenor of the family’s home.
Then I chose some other sentences from the novel, written from the point of view of an 11-year-old boy, and popped them into GPT-2. It generated a continuation that made me laugh out loud:
On my way home from school, I walked past Mr. Katz’s house. Mr. Katz lived down the block and the reason why his name was Mr. Katz was not because he had lots of cats, it was because he was a Hasidic Jew.
There were lots of Hasidic Jews in Mile End. … Mr. Katz was religious like them but as for his cats, there was no cat with a Hasidic past. I noticed the first one as I passed Mr. Katz’s front door, a tiny black cat that looked as if it had been ripped right off the inside. It had been left on an empty sidewalk and had not been fed since. I quickly made a mental note to not try to feed it at his house again; this isn’t how the Hasidic Jews do things.
Ha! This was almost something my eccentric young character would say. It’s like GPT-2 tapped into his sense of humor and turned it 20 or 30 degrees clockwise, defamiliarizing it. It was somewhat stranger than what I’d actually written in the book. And it was definitely funnier.
Since I sometimes struggle when writing funny scenes — humor is not my forte — it occurred to me that working with GPT-2 could probably help me out by knocking my imagination off-kilter a bit, nudging it in more surprising and playful directions.
Next, I wanted to know how the AI would handle a character who’s losing her sanity, to the point that she’s eating a manuscript penned by a dead man.
His words, and all around them in the margins, my words, my dreams, my commentary. My mouth grew bolder—teeth biting into paper—a ragged hole ripped right through the middle of the manuscript. I ate and ate and ate. Letters stumbled into my mouth and I swallowed them. Ink poured down my throat and I drank it. Words I didn’t know flowed through my skin and I drank them and drank them and drank them all over again. I ate, sated, until I vomited. I woke and it was all gone, replaced by fresh and raw words. My mouth was raw and my face was raw and my bones were raw.
You can see how GPT-2 has learned from my sentence “I ate and ate and ate” that it should engage in repetition. But it generated text that uses that device too much to be effective. On the other hand, it came up with an idea that I honestly wish I’d come up with myself: After the character gobbles up the dead man’s language, she vomits it all up. Her body rejects it.
The night I was writing this passage, I didn’t think to have her respond with that violent physical reaction. But in retrospect, it would’ve been perfect.
I also find it noteworthy that GPT-2 did a pretty decent job of conjuring up madness. It captured that state of mind with just a couple seconds of computation — rather more efficient than the writing method I used, which was to get drunk on cheap red wine.
Now that I have GPT-2, here’s how I plan to use it
My next project is a children’s novel about two little girls who discover a black hole in the middle of an ancient hotel and, naturally, jump into it.
Unfortunately, I’ve been experiencing some writer’s block. I know the world-building is going to be crucial to pulling off this fantasy story, and I don’t yet see the world as clearly as I need to. So I plunked a few sentences into the AI, this time choosing a passage with dialogue:
They climbed into the wormhole. The air inside the tunnel felt cool and fresh and blue, like the inside of clouds. Her skin tingled.
She turned to Roy G. Biv. “Would you, um, like to join us?”
“Join you?” The physicist looked utterly baffled. His brow furrowed. “Yes.“
“Are you sure, Mr. Biv?”
“Yes. I am. And here’s the deal. Is the wormhole closed, or open? And is the wormhole stable? And does it feel like it takes shape when you look at it, or like it is a fluid thing, like it has to be squeezed? And does it hold its shape in place?“
I gasped with pleasure when I saw this. Prompted with the idea of wormholes, GPT-2 had returned to me a bunch of questions seeking to pin down exactly how these wormholes worked. That’s monstrously helpful to a writer in the world-building stage. It forces me to envision everything with extreme granularity, lending believability and texture to the narrative.
To be clear, I wouldn’t want to simply copy-paste sentences written by GPT-2 into a novel. The generated text is often full of non sequiturs and bits of weirdness that don’t advance the plot, serve the story, or belong in any way. And although an AI can excel at mimicking a human artist on the local level (say, a few sentences at a time), it fails at generating a larger structure that feels satisfying (like a complete work of art).
Conceivably, we might one day invent an AI capable of writing an entire novel that has a compelling narrative arc. But that sort of technology doesn’t exist yet, and I doubt whether even the full version of OpenAI’s model comes very close.
The OpenAI team expects to release the full version in a few months. They didn’t release it to the public all at once for fear that malicious actors might use it to spread misinformation or churn out endless reams of racist and sexist garbage on websites like 4chan. Some experts praised the team for taking AI risks seriously; others said delaying the full release was a publicity stunt.
Either way, I can’t wait to meet the even more robust version of my collaborator.
Using AI as a replacement for all authorial work? I don’t see that happening anytime soon. But using it as a cognitive enhancement, a creative prosthesis? Sign me up.
For more on AI risks and GPT-2, listen to this Future Perfect podcast episode.
Sign up for the Future Perfect newsletter. Twice a week, you’ll get a roundup of ideas and solutions for tackling our biggest challenges: improving public health, decreasing human and animal suffering, easing catastrophic risks, and — to put it simply — getting better at doing good.
