On Creativity

July 28, 2023

I. Nielsen on Creativity

I was reading some old Michael Nielsen essays when I stumbled across an exchange he had with Julia Galef. It contained some of the most insightful few sentences I’ve come across on the subject of creativity:

To be creative, you need to recognize those barely formed thoughts, thoughts which are usually wrong and poorly formed in many ways, but which have some kernel of originality and importance and truth. And if they seem important enough to be worth pursuing, you construct a creative cocoon around them, a set of stories you tell yourself to protect the idea not just from others, but from your own self doubts. The purpose of those stories isn’t to be an air tight defence. It’s to give you the confidence to nurture the idea, possibly for years, to find out if there’s something really there.

If you google around for insights into creativity, you’ll run into lots of anecdotes about how creative people make connections that others missed. (Exhibit one, exhibit two.) They’ll cite Einstein, or Seneca, or Steve Jobs, and then tell you to go learn a bunch of stuff so that you too can connect, as if creativity is some manual process that you perform after your morning crossword.

There’s no doubt that part of creativity is making connections, but there are nearly an infinite number of connections one could make in principle—are creative people simply those who happen to stumble on the most interesting ones? Further, if the essence of creativity is making connections, it would suggest that creativity should rise in lock step with knowledge. But many of us know ten- and twenty-somethings who are more creative than forty- and fifty-somethings, even if they have similar backgrounds. In fact, a common belief among mathematicians is that they often do their most innovative work when young¹. This idea also has nothing to say about the kinds of habits and environments which foster creativity.

Nielsen’s analysis is refreshing because it presents us with some insight into the actual creative process. What it’s like in those moments of ingenuity. What I find most interesting is the emphasis on error and doubt. I think he’s right. A creative thought is not usually accompanied by confidence. It can be exciting, sure—but often vague and uncertain as well. This makes sense, upon reflection. A truly novel idea should not be obvious from prior experience or knowledge.

It’s always a good sign when the same idea is presented by different people. In his 2020 book Creativity (h/t Vaden Masrani), John Cleese expresses much the same idea as Nielsen. He writes

When we’re trying to be creative, there’s a real lack of clarity during most of the process. Our rational, analytical mind, of course, loves clarity—in fact, it worships it. But at the start of the creative process things cannot be clear. They are bound to be confusing. If it’s a new thought, how can you possibly understand it straight away? You’ve never been there before. It feels unfamiliar. (pg 40-41).

II. Hairy Chelonians

Cleese frames his discussion around the concept of the “hare brain versus the tortoise mind,” a distinction introduced by the cognitive scientist Guy Claxton. The hare brain is fast, logical, analytical. It wants to seek truth, to criticize, to prove, or discard. It is directed and deliberate. The tortoise mind is the opposite. It is playful, leisurely, and unconcerned with decisively deciding. It revels in the uncertainty, more interested in discovering patterns than deciding on truth or falsity.

Tortoise mind is on display during hypnagogia, that alluring mental state sometimes experienced on the edge of sleep. Many inventors have cited hypnagogia as playing a key role in their idea generation. To quote wikipedia,

The hypnagogic state can provide insight into a problem, the best-known example being August Kekulé’s realization that the structure of benzene was a closed ring while half-asleep in front of a fire and seeing molecules forming into snakes, one of which grabbed its tail in its mouth. Many other artists, writers, scientists and inventors – including Beethoven, Richard Wagner, Walter Scott, Salvador Dalí, Thomas Edison, Nikola Tesla and Isaac Newton – have credited hypnagogia and related states with enhancing their creativity.

In fact, Edison would deliberately try to enter a state of hypnagogia. Cleese writes,

He found that he got his best ideas in that funny no man’s land between being awake and being asleep. So he used to sit in a comfy armchair with a few ball bearings in his hand and a metal bowl underneath. When he dropped off to sleep his hand relaxed, the ball bearings fell on to the plate and the noise they made woke him up. He’d then pick up the ball bearings again and sit back and get into that same drowsy, dreamy frame of mind that he’d just been in. (pg 39)

Creativity requires turning off the hare mind. New ideas need time to grow, to be sheltered from premature criticism, because they will often have only a seed of truth to them. They’ll be surrounded by error and attached to misconceptions. You need time to play with the idea, to nurture it into something that’s ready to interact with the world.²

This is not to say that the initial idea does not occur to you in a flash. An insight often jumps into your mind unexpectedly—when you’re pouring your coffee, listening to a podcast, or after stopping work for the night. But it is to say that the initial idea is often not ready for prime time. It needs to be incubated by the tortoise mind. If you launch it into the world under-developed it won’t reach its full potential.

Put this way, the creative process has some interesting relationships to irrationality and close mindedness. Of course, once an idea is “ready,” it should be treated as any other idea and pressure tested in order to see if it has merit. Before that stage, however, perhaps you must be dogmatically attached to your own ideas, hesitant to expose them to too much criticism (both from yourself and from others).

Interestingly, the philosopher who thought the most about the criticism of ideas, Karl Popper, also recognized the importance of dogmatism, especially in the early stages of a new theory. Popper built his entire epistemology around the importance of criticism of ideas. He writes,

[C]riticism is, in a very important sense, the main motive force of any intellectual development. Without contradictions, without criticism, there would be no rational motive for changing our theories: there would be no intellectual progress. (Conjectures and Refutations, pg 424)

But he also argued that it is useful for some people to stick to a theory until absolutely untenable, so that others are forced to criticize the best version of the idea:

The dogmatic attitude of sticking to a theory as long as possible is of considerable signiﬁcance. Without it we could never ﬁnd out what is in a theory—we should give the theory up before we had a real opportunity of ﬁnding out its strength; and in consequence no theory would ever be able to play its role of bringing order into the world, of preparing us for future events, of drawing our attention to events we should otherwise never observe. (Conjectures and Refutations, pg 420)

Popper was discussing idea generation and criticism at a societal level. We might view Nielsen and Cleese as extending his claims to the individual level. For you to see the best version of your ideas, you need to let them sit and mature for some time without attacking them with your logical mind. You need to let part of you be dogmatically attached to new ideas so that you can find out their strength. Then, once they are mature, you attack them with irreverence.

III. Questions

The Nielsen-Cleese take on creativity raises several questions.

For one, when is a new creative thought ready to be shared with the world? I’ve been using words like a “mature idea” and an idea ready for “prime time.” But what does this mean? Cleese says that a good guideline is when you can vocalize the thought. That is, if it can be expressed by language then it’s ready to be shared with others.

Though it pains me to disagree with the man who brought us Fawlty towers, I think Cleese is wrong here. I don’t think there’s a general rule. Instead, the length of the incubation stage will depend on the idea. Different ideas require different amounts of time. A new joke will be different than an idea for a new product; an idea for a mathematical proof different from an historical thesis. Some ideas are clearly ready to share after an hour while some, as Nielsen wrote above, must be nurtured for years.

Second, are there external tools that can help engage the tortoise mind? Claxton seems to suggest no. He warns us against prematurely articulating an idea, implying that you should not speak or write about it for some period of time. He dedicates a chapter of his book to the art of mental gestation, comparing the birth of an idea with the birth of a child. (He makes this sound much less odd than you think.) The implication is that relaying the idea externally, in any medium whatsoever, can shut down the tortoise mind.

Perhaps this is true. But my guess is that there are modes of writing and speaking which allow you to stay in the light and playful state associated with the tortoise mind. Free-flow writing—writing without pause and without edits—for instance, strikes me as something that aligns well with the tortoise mind. Similarly, I’ve been in conversations where the goal is not to arrive at a decision, but to simply bat ideas around without judgment. And it certainly feels like I’m often struck by creative ideas in the midst of such conversations. But is the process of even talking about the idea ruining the incubation period, and not letting the idea grow as it should? Maybe.

Third, how is all of this related to flow state? When you’re deeply engaged in writing or programming or maths, it often feels creative. But here you’re generating ideas and acting on them almost immediately. There’s very little, if any, incubation period.

Perhaps flow state is best characterized as implementing ideas that have already been bouncing around in your head for a while, but that you’ve never made concrete before. More likely, “creativity” is a suitcase term that refers to a whole bunch of concepts that we don’t understand very well. Nielsen and Cleese are discussing one aspect–rare and substantial novel insights–but there may be distinct creative processes that work in different ways.

IV. Applications to AI

An ongoing debate in AI circles is whether modern generative models are creative. Obviously, this depends on precisely what we mean by creativity. But the arguments we’ve explored here would suggest that AI is not creative, at least not in the same way that humans are.

For one, current machine learning models have no subconscious. It doesn’t make sense to discuss the tortoise mind of GPT (in fact, it barely makes sense to talk about a mind at all.) Idea “incubation” is therefore impossible for current systems: they’re not active when they’re not being queried by a user.

But a more serious problem is the fact that these systems are purposely trained to produce output which matches the patterns in their training data. To simplify things a bit (though not much), if there are many sentences of the form “The atomic number of uranium is 93” and only a handful of the form “The atomic number of uranium is 92,” then given the prompt “what’s the atomic number of uranium?” these systems are likely to output 93.

However, insofar as Nielsen and Cleese are correct, novel creative insights are at odds with past experience. That is, they should not perfectly fit the patterns of previous thoughts and observations because they are inherently new. Machine learning methods are giant correlation machines, but creative thoughts should be inversely correlated with the past. How humans are doing this is a mystery, but it’s a striking contrast with modern AI.

Okay, in fairness it’s a bit more complicated than this. GH Hardy famously claimed that no mathematical breakthroughs were made by anyone past fifty.

"No mathematician should ever allow him to forget that mathematics, more than any other art or science, is a young man's game. … Galois died at twenty-one, Abel at twenty-seven, Ramanujan at thirty-three, Riemann at forty. There have been men who have done great work later; … [but] I do not know of a single instance of a major mathematical advance initiated by a man past fifty. … A mathematician may still be competent enough at sixty, but it is useless to expect him to have original ideas." - G.H. Hardy, A Mathematician's Apology.

But this has been disputed. See this math overflow post for some interesting links. ↩︎
It’s no wonder that Julia Galef disagreed with Nielsen. Galef represents a movement concerned with spreadsheets-style thinking and assigning credences to thoughts so that we can order them by confidence. Rogue, half-baked, scalar-free thoughts are anathema. ↩︎

Back to all writing