Category: AI

My Own Personal Move 37

I’m doing Nanogenmo this year, which involves using software to produce novels. I started out by seeing whether Claude could produce a book with very little guidance. It’s worked better than expected. The result is better than many of the human-written books I’ve read this year.

Move 37 refers to a particular moment in the Go match between Lee Sedol and Deepmind’s AlphaGo computer program, which was won by AlphaGo. After the match Sedol said “I thought AlphaGo was based on probability calculation and that it was merely a machine. But when I saw this move, I changed my mind. Surely, AlphaGo is creative.”

In 2024 I tried producing a book with OpenAI and the prose was bland, uninteresting. This time, I worked with Claude’s Opus model. It’s long been known that LLMs work best with fiction based on found documents, so I thought I would try that. I also wanted something that would entertain me. So I set out an initial basic prompt.

I want you to come up with a breakdown for a novel about the novel House of Leaves creating an incursion into our universe. The novel should be 50,000 words long, and work a little like World War Z, producing a set of characters and formats to describe a growing catastrophe caused by Danielewski’s novel. You can break it down into chapters as you think fit. Use lost media, information hazards and creepypasta as inspirations.

This is objectively not a very good prompt. I know this because I’ve done my Claude certification. But I was only playing at the start of NaNoGenMo, trying to produce something as a starting point. Claude came up with a plan, which I didn’t read beyond a few paragraphs. I tweaked it from being a little too close to World War Z, then asked the LLM to produce text. I was spectacularly lazy, and requested that Claude generate its own prompts. I sent the text-in-progress to my Kindle and set about reading.

Claude’s book, The Quarter Inch, certainly did a better job with this idea than I would have done. The model has also avoided upsetting itself over copyright because, in the world of its book, quoting from the original text is considered dangerous..

The novel is definitely derivative of other works, and intentionally so. “[-] of Leaves” (as the book refers to it) is one of my favourite novels, one I know well and find genuinely terrifying. The Quarter Inch builds on that terror by describing that book escaping into the real world.

The Quarter Inch describes the first few incidents, book groups who measure their flats as a joke. Slowly, the rumours spread and more people encounter the House from the original book – until 29th February 2032, when all the openings close, leaving people trapped on the other side. In the aftermath, the world’s governments do their best to manage an impossible crisis. Meanwhile, Eve Calloway, a records manager at Hereford Council moves into a round building and begins creating an archive about ‘the Interior Event’.

There are some clumsy moments but the prose is generally smooth and unflashy. It’s very readable. I spotted a couple of potential edits, but very few of them. I’m currently reading a professionally published music biography where actual humans left notes to the editor in the text – so the standard of Claude’s book is beyond that of some human ones I’ve paid money for.

It’s possible the scenes that impressed me last night are based on other stories. Although there’s a great section about an AI company who have illegally included “[-] of Leaves” in their training data, and face the consequences. That doesn’t sound like a pre-existing story. There’s a council leaflet about ‘Measuring Week’, where everyone is asked to check their interior spaces which has a Scarfolk vibe.

The bit that blew my mind this morning featured a copy of a printer’s invoice from Lagos, item number QI/2039/017. The letter accompanying the invoice began as follows:

Dear Mrs Calloway,
My father, Tunde Okafor, died in 2033. You asked whether he printed the book. Yes. Everybody in Lagos who had a press printed it in 2027 and 2028, and my father had three. I am sending you one invoice because you asked for one. There were more than four hundred.
You must understand how it was. Someone had said, in a newspaper first and then everywhere, that if you wrote in this book your flat would get bigger. In a city where a family of six lives in one room, people heard that and nothing else. They did not hear the rest. They came with money in their hands. My father was not a wicked man, but he was not a fool either, and he printed what people paid for.

That passage takes an angle on the concept I’ve not considered. I love the phrasing in that last sentence too. Adaeze Okafor’s letter gives me a feeling of her personality. It’s a passage that expands The Quarter Inch‘s world beautifully.

Overall, this book has been very entertaining. Yes, there are errors. Yes, there are issues with LLMs as a technology. None of that takes away from the fact that I asked one to write me a 50,000 word novel and it did a good job. It’s impressed me several times with a good phrase. There are bits I’d change (“She gave me tea in a front room with a piano and a clock that ticked loudly enough to be a third person” – that seems a little overwritten compared to the rest). But, broadly, this works.

I don’t think Claude is going to replace human writers. I asked it to produce a specific style of novel I knew it might be good at. The text is based on a massive sample of human writing. But the fact that it’s done a good job here is significant. No publisher is likely to produce a story about [-] of Leaves – but with a £20 Claude subscription I’ve produced a specific book that I wanted to read.

There are obviously significant ethical issues around whether I should be producing a work based on Danielewski’s book without his permission. The Quarter Inch is certainly not a book that works without House of Leaves, but it is about the reaction to it. And it doesn’t replace that book, but instead points to it as an important work that people enjoy obsessively.

I’m generating the last couple of chapters in the book in the background while I write this, and I’ll add the finished text to this post once I’ve read it myself. I’ve not discussed my views on the significant issues around AI here. I am aware of them, and I have thoughts on that. But, for this post I wanted to focus on this fact: Claude has produced me a bespoke novel that I’m enjoying reading.

What to use agentic code generation for?

Last weekend I was playing with Claude AI. I need to get more experience of generative AI for work, and that requires a larger project to play with. But what do I work on?

Suddenly, code is easy. I should be excited by the opportunities that agentic code generation gives me. The problem is, there weren’t any projects I was blocked on due to software complexity. If I’m not careful, I’ll simply generate another source of clutter in my life.

I did have something that stalled due to the coding. It’s due to emerge as a real-world project in collaboration with a friend, using very little technology. We’re not giving that one a URL or even a web presence.

Another project involved taking data from the FitBit API. This floundered on the issue of needing an official privacy policy and terms of service. That API was intended for corporations rather than hobbyists.

There was a controversial essay by Mike Judge, Where’s the Shovelware, which claimed that agentic AI had yet to produce an uptick in finished projects on places like Steam and the app stores – code is not the only barrier to getting things out in public.

Maybe the power of agentic AI is in enhancing private workflows. Craig Mod has talked about building his own private accounting software. Maybe I should write bespoke editors or plugins to fix the clumsy errors I make when writing. I’m definitely thinking of using agents to structure my hugely complicated South Downs Way project. That is, at least, public-facing – but not something I can talk about in my corporate role.

Which means a new project, but one that won’t add clutter to my life. Maybe it should be a game, a photo scavenger hunt. Or a collaborative art project. But those would involve a lot of non-technical work to manage the participants. Other ideas are half formed – something to do with mapping?

So many projects have the drawback of making my life more complicated. Improving private workflows looks like the best case. Fortunately, it looks like I have a friend who needs this for a public project. Something I can work on for now.

Some notes on AI & poetry

I’m due to give three talks in the next month and I’d been particularly anxious about one of these, on GenAI and poetry a month tomorrow. Last night I had a long intense dream which featured me worrying about the three talks, as well as a mysterious fourth. But on waking up I felt inspired about the whole thing, and have been gathering my thoughts.

  • I’ve started reading Funkhouser’s Prehistoric Digital Poetry, which looks at the form pre-world-wide-web, starting from 1959. It seems like an interesting complement to 50 Years of Text Adventures due to the digital archaeology involved and the overlapping time periods. Funkhouser places digital poetry in a wider framework – Oulippo, obviously, but he refers to examples of permutation poetry dating back to ancient times.
  • Obviously, much of poetry’s power comes from the link to human experience. Someone made the point that an AI-generated account of the Spanish Civil War could never work as well as Orwell’s first hand account. Would a computer-generated version of Howl ever work?
  • NaNoGenMo has been running since 2013, well before the emergence of LLMs. I found reading All the Minutes a genuinely moving experience, with its gestalt voice emerging from twitter posts. I think there are interesting ways we might use GenAI to generated such summarising texts.
  • I recently got hold of Jeff Noon’s Cobralingus, which imagines a word engine transforming text into new forms. It’s something that could easily be done using ChatGPT.
  • William Deresiewicz proposed that AI Will Never Rival Human Creativity through LLMs since these are designed to pick likely decisions. I know friends that preferred earlier text-to-image models since these were less accurate and produced more interesting outputs. But I can’t imagine it being too difficult to train/design an LLM to be more ‘creative’.
  • GenAI is an interesting tool for reappropriation of existing texts. I had an interesting session where I generated haiku from some of my favourite poems, with some interesting results.

A red wheelbarrow,
Glazed with rain beside white hens,
Much depends on this.

Book Review: Benny the Blue Whale

Benny the Blue Whale book is the latest collaboration between ChatGPT and an established writer. The core of it is, effectively, a transcript of the sessions where Andy Stanton persuaded ChatGPT to tell a long story about a blue whale with a tiny penis.

The book’s layout is stunning, with the transcript on the left-hand pages, and the right hand pages devoted to notes. There are also footnotes, as well as footnotes within footnotes. The book feels like a screen with multiple windows. I’d love to read more books with this sort of layout.

I found the story itself less interesting – it was not really my sense of humour and I often found it tiresome. But I enjoyed Stanton’s observations about ChatGPT and the writing process. A lot of responses to ChatGPT are either credulous or dismissive – it’s more interesting to see a writer engage with the question of the possibiliy of ChatGPT producing great work.

This is a book very much of its time – it is basically someone describing a series of prompts they made to ChatGPT. It’s is a book about first encounters with LLMs. I suspect its long-term importance will be in capturing a particular moment.

I most enjoyed Stanton’s discussions of improv and narrative theory. In one section, he demolishes the idea that authored art will be replaced by people interacting with GenAI. We don’t want to have to work for our stories. “Ultimately I want my fiction to be frozen. I want someone to have picked the very best throughline they could”

The shock of ChatGPT

Gen AI is one of the most incredible things I’ve ever encountered. I’ve read explanations of how it works, but when I play with it, the responses feels like magic; as if I am talking to a creature that knows a huge amount of information, and can summarise it very quickly.

The improvements over the past few months are astounding. When I played with GPT-2 a couple of years ago, it seemed like a toy. In the past few days, I’ve been asking Chat GPT 3.5 for help with programming tasks in Angular. I’m not familiar with the framework, but ChatGPT can give me specific tutorials for the tasks I’m dealing with. I’m finding it easier to learn Angular than ever before.

A lot of the discussion I see on social media points out problems with genAI – yes, there are issues around how the data is collected. Yes, these things are not perfect. But a lot of the complaints feel like bargaining, people trying to persuade themselves that the results of this technology are mundane.

I do think there is a lot of hype around LLMs, but I am also convinced that they are going to have a huge effect in my work and career. I’m not convinced they will eliminate programmers, but they are going to make them much more productive – think stack overflow but more so. One article compared working with ChatGPT like being assisted in the task you’re working on by a very enthusiastic colleague whose code wasn’t perfect, but it would speed you up.

Stack Overflow has not reduced the number of software developers in the world – rather it has improved productivity and enabled people to build more complicated systems than ever before. In the early days, people were suspicious of IDEs, but now they are making it easier to manage large codebases. I look back at the web applications being built around 2000 and I’m amazed at the scale of what a team of developers can produce now.

ChatGPT fills me with existential dread – there are huge philosophical implications to the idea that a language model can do sophisticated tasks so easily. But as much as I’d like to pretend it is not significant, the best thing to do is learn how to use it, and work out what the role of a software developer is in this new world.

Playing with AI

When AI images started appearing on social media, I was impressed but didn’t find it particularly interesting. What changed my mind is the work of Rob Sheridan. His Spectagoria project involves spooky images produced with AI, which claim to be from the 1970s, originating in “a renowned underground fashion photography magazine surrounded by rumor and mystery”.

I love these images and their feeling of faked authenticity. Something like this could have been produced via photoshop, but only at the cost of much more work. I know these images are AI, and Sheridan is open about it, and there’s a hauntological feeling that comes from knowing that they are not real.

I found out about Sheridan through Ryan Broderick’s Garbage Day, where Ryan put forward his personal rule for AI art: “Is it trying to do something interesting and not hiding the fact it’s AI generated? Cool. Give it a whirl! See what happens. Is it a bad, automated replacement for human-made content? No thanks.”

Another AI project I keep thinking back to is 2022’s Summer Island comic by ‘Steve Coulson and Midjourney’. This uses the AI illustrations as the background for a story about folk horror and kaiju. The images here served the story and I enjoyed the writing. This one could have been produced using an illustrator, but the Coulson would likely not have been able to pay an illustrator to tell his story.

I’ve been inspired to begin playing with AI art, just to see what sort of thing I can produce. There’s an interesting aesthetic here, as can be seen with this image at the top of the page – which Stable Diffusion based upon a photo of me.

There’s something here worth playing with. Interestingly, my friend Dan prefers Stable Diffusion version 3 rather than 5, as the earlier version has a more interesting aesthetic.

Some interesting links on AI – July 2023

  • Superintelligence: The Idea that Eats People was a good counterpoint to the idea that AI is inherently dangerous.
  • Erin Kissane jokingly suggested that ChatGPT’s writing style (“Suuuuuper heavy on the adjectives, dialogue especially wooden, lots of overtly charming touches.“) comes from being trained on fanfic. There is an important point here, that we are judging AI’s specific abilities based on ChatGPT’s massive, general purpose dataset.
  • Naomi Klein wrote about how all AI responses are hallucinations, not just the ones that are nonsense. While I diagree with some of the points, she’s absolutely right about how the term ‘hallucination’ frames the debate.
  • Language models have been trained on a massive range of text, and it seems that this includes some very specialist slash fiction.
  • Stephen Marche has spoken about how he approached writing with ChatGPT, asking for “a murder scene in the style of Chinese nature poetry” and applying some transformations to get something a little like Chandler – rather than asking for Chandler’s style directly.
  • “You are already an AI-assisted author,” Joanna Penn tells her students on the first day of her workshop. Do you use Amazon to shop? Do you use Google for research? “The question now is how can you be more AI-assisted, AI-enhanced, AI-extended.” (Link)
  • It’s not a story about AI, but Werewolf erotica is the latest global gig work trend shows some odd effects of technology on writing.
  • The Great Fiction of AI is a similar article, talking about the very fast cycles running in e-books, and how AI is helping with the production of very specific types of novel.
  • A Storefront for Robots talks about how online language is already distorted by AI, as people need to write both for search engine bots and for humans.

Links from my AI and Creative Writing workshop

Towards the end of June, I gave a small workshop about writing with AI. We looked at some techniques used to generate creative work with ChatGPT. During the session, I referred to a number of resources, which are collected below:

AI Book review: Death of An Author by Aidan Marchine

Death of An Author is an ebook novella produced by Stephen Marche using AI tools, which was commissioned by Pushkin Industries. I read a couple of good interviews about the book and, while I’d found the excerpts underwhelming as prose, I was curious to read the whole thing, particularly Marche’s afterword.

The book is about 95% AI text, using three tools. The original text was generated using ChatGPT, with Sudowrite used to tidy the text, making it “more active” or “more conversational”, and finally Cohere used to generate figurative language.

Death of An Author is a sly book, one that understands its place within a larger debate, and making allusions and interventions based upon this. There are references to the act of reading, the meaning of copyright and the book’s literary context – this is very much a book produced by someone with an English PhD, and the text itself is aware of what is at stake. The pseudonymous collaborative writer (Aidan Machine) even refers to an imaginary article by Stephen Marche, the human collaborator in the text. The writing itself is brisk if superficial, but the mystery had an interesting resolution – at least it felt so to me, as someone who rarely enjoys mysteries.

An example of the book’s commentary is in the description a dream the main character has:

That night, Gus had a terrible nightmare. He was taking an oral exam in front of his mother and ex-wife. Each time he tried to answer, a different writer’s voice came out: William Faulkner, Ernest Hemingway, Jane Austen.

Hemingway is the name of another AI writing tool. The dream questions the nature of imitation for human writers, as well as referring to one of ChatGPT’s finest tricks, that of imitating well-known writers.

The book also occasionally digs into specific descriptions, as in this outline of a meal.

That night, Gus made himself a meal of fried mushrooms in a cream sauce on toast. He started by heating a pan on the stove, adding butter to it. He then sliced the mushrooms and added them to the pan, cooking until they were browned. He added cream to the pan, stirring until the sauce thickened. He placed slices of toast on a plate and spooned the mushroom and cream sauce on top.

That passage is obviously AI, right? But I can’t be sure – maybe this is part of the 5% Marche wrote by hand. In a Guardian interview, M John Harrison spoke of an ambition he has: “I want to be the first human being to imitate ChatGPT perfectly. I bet you it’s already got mimickable traits”. Either way, it’s interesting that Marche chose to leave this prosaic description in the text.

Given the way in which the book’s plot takes it to such interesting places, it’s likely that Marche gave the AI some fairly clear leads about the overall story. It would be interesting to know the actual prompts used, although Marche talks about how he generated the book’s style:

What you need is to have it write something about a murder scene in the style of Chinese nature poetry, then make it active, then make it conversational, then Select All and put it in the style of Ernest Hemingway. That gets you something interesting. Raymond Chandler, after all, was not trying to write like Raymond Chandler.

The prose is often workman-like, but some flourishes and philosophical asides stick out. One sentence I particularly liked was “The policeman at the door of Gus’s office was a tall, thin, cadaverous man wearing a dove-gray suit that did not fit him well”. It is simple but has a lovely rhythm, and fits the hardboiled style. The choice of ‘dove-grey’ is interesting when we’d normally think of doves as white. And it makes me wonder how much work was done to get this just right. Marche has spoken about how the best results come from very precise prompts that are specific about substance and style. How much work was this, compared with just writing the sentence one wants?

Marche says that he did a significant amount of sifting of the produced text. This is some way from the dream of giving an AI a short prompt and having it produce perfect, entertaining prose. However, it’s notable that when Burroughs worked on his cut-ups, a great deal of time was spent exploring the results for interesting lines. Again, I’m curious about how much work was required to get the best lines (“He wondered why there was no good English word for slimy in a good way.”).

Marche talks about how good the AI is at ’heteroglossia’, with an uncanny ability to turn its hand to specific modes of speech such as “a paragraph from a Lacanian literary critic”. Marche says that he “[defies] any writer to improve on AI at that particular skill”. While the AI struggles to produce crisp narrative prose, it is an excellent mimic. In a podcast interview , Marche made the exciting suggestion that an AI might be very good at turning out a book like Dracula, which consists of different forms of documentation.

(A discussion in Wired Magazine placed Death of an Author alongside books including Moby Dick, which used extensive found text, and Graham Rawle’s Woman’s World, which sampled 50s women’s magazines)

Discussing the debate around AI, Marche writes:

So little of how we talk about AI actually comes from the experience of using it… Like the camera, the full consequences of this technology will be worked out over a great deal of time by a great number of talents responding to a great number of developments. But at the time of writing, almost all of the conversation surrounding generative AI is imaginary, not rooted in the use of the tool but in extrapolated visions.

It is important to actually play with these tools in good faith to see what they can do. The first book I bought made using ChatGPT was shardcore’s remix of John Higgs’ writing, The Future has Already Begun. Shardcore has spoken about how having the book as a physical object changes things. Reading Aidan Machine’s book makes a compelling argument better than the thousands of thinkpieces and opinion chatter. I only wish it had been made available as an actual paperback.

Marche concludes his afterword by saying that “What makes a good painting and what makes a good photograph are different. That transition required a complete reevaluation of the nature of visual creativity” The best AI art will not come from reproducing writing that humans can do better, but from finding new forms for this medium.

Is AI/ChatGPT as exciting as people say?

On this page, I’m collecting together some links about AI that I want to refer back to. Those are below; but first I wanted to discuss what I find interesting about AI right now.

I’ve been suspicious of the hype around AI for a long time, but I keep reading articles saying that AI models will revolutionise the economy, replacing millions of jobs. I’m even seeing serious articles saying that these new AIs are approaching sentience and are an existential threat to humanity.

One theory I’ve read is that the AI hype is so loud because the crypto grifters have moved onto this: there is always a profit to be made from hyping the next big thing. It was also an amazing marketing move from OpenAI to claim that their early models were too dangerous for people to have open access.

At the same time, very smart people I know, people I trust, are telling me there is something important here. And I’ve had several dreams about prompting AIs – so I should not be dismissing this too easily.

I’ve been playing a little with ChatGPT recently. Showing a friend how it could generate an email was an enlightening moment. He doesn’t like writing formal messages and having a detailed text produced from a prompt seemed revolutionary. My own experiments show that ChatGPT is good at certain types of output, but its grasp on facts is hazy. Asking it about things I know well, like hiking routes, it returns plausible information, but lacks telling details and gets significant facts wrong. This is beguiling, miraculous technology, but it (currently) has very clear limits.

Links on AI

The following are some good pieces that I have read on AI:

My current view

I find it hard to see how these huge statistical models are related to ‘true intelligence’, even as they raise questions by doing things that we once thought relied on intelligence. One notable thing is that (as with machine translation) these models are entirely reliant on human-produced work. This has led to the ethical questions around the model incorporating copyrighted works – and I note (via the Washington Post) that this blog is one of the sources for Google’s C4 data set:

I also wonder how much further these models can go. It won’t be long before the deluge of AI content begins to be absorbed into the models, which may undermine their effectiveness.

I also suspect there is a limit to the effectiveness of LLMs for problem solving. Matt Webb’s Braggoscope is the most compelling experiment I’ve seen, where ChatGPT was used to classify the thousands of In Our Time podcasts into the dewey decimal system. It’s a task where small inaccuracies will cause little harm, and Webb estimates that the automation of this was 108x faster than doing it manually.

But for tasks like programming, much of the art is not in producing the code, but figuring out what code needs to be written. It’s possible that a new paradigm of programming emerges from AI, but for any form of programming as we currently understand it, the trick is not writing the code but defining what code we want written, and making sure that we have achieved our aims.

The difficulty with AI is in producing very specific text. Producing remarkable sonnets about odd subjects is breath-taking, but getting an AI to write Allen Ginsberg’s Howl or Pierre Menard’s Selections from Quixote would be a different matter.