A machine that can perfectly understand and respond to human language has been the paradigm of success in artificial intelligence since Turing. The last decade has featured an ongoing revolution in language-focused machine learning, enabled primarily by new architectures and skyrocketing scale. A single model with billions of parameters, like OpenAI’s GPT-3, can complete a huge variety of tasks, from answering questions to generating text, based on few or even zero examples. Yet this revolution will not continue unhindered. I argue that large language models (LLMs) are structurally incapable of achieving human-level language abilities. By applying critical insights from cognitive science to deep learning, I develop both empirical and theoretical arguments to temper the overzealous excitement that LLMs will soon ‘solve’ natural language processing (NLP). There is more to language than LLMs can grasp. Finally, I sketch some more tractable pathways to language understanding by machines.
Crucially, my thesis is distinct from other long-standing and contentious debates in cognitive science. I do not defend the related but much more heavyweight assertions that artificial intelligence in general is impossible or that connectionism is false. Rather, I muster evidence against the specific approach of LLMs, showing that this method is extremely unlikely to match the human ability to understand and use language. These problems with LLMs are structural, and not merely incidental quirks of certain models or inadequacies due to insufficient scale or incomplete data. The issues stem from the constitutive features of LLMs, like applying statistical learning to text data.
Why should we care if LLMs can truly “think” or “understand”? Often, unarticulated fears and anxieties get entangled with this question – worries about our obsolescence as humans, of AI taking over our jobs and systems, of AI becoming sentient and evil, of being unable to keep up with our own creation. These are all fears worth exploring in their own right. Sometimes they are dismissed with a simple “oh, but it’s not really thinking.” This is not enough. We need to dig into these problems and their implications. But in this essay, I aim to bypass these fears somewhat by focusing on the structural and technical features of LLMs, and the philosophy and cognitive science of language understanding.
1. The Structure of Large Language Models (LLMs)
A language model (LM) is a system for predicting strings in a sequence. Formally, an LM is simply a probability distribution over a sequence of words. The model samples from this distribution to find the conditional probability of different words appearing in the sequence. Early research centered on N-gram LMs, which just use the relative frequency of words in the training data to predict what words come next. In contrast, LLMs are the state-of-the-art: models with (at least) millions of parameters that train an artificial neural network (ANN) to predict strings. Simply put, you can give an LLM a fragment of text, and it can tell you what is most likely to come next based on its statistical model of human language. Once the system gets good enough at predicting what comes next, it can generate new patterns of text that match the structure of its corpus.
The architecture of Transformer-based LLMs, the most common structure for LLMs (including GPT-3).
These models start by converting words to vectors (embeddings), where semantically similar words are closer together in the representational space. Then, they train an ANN to predict an output label (the next word in text) given some input vector (the context). Almost all modern LLMs involve transformers, an architecture that uses self-attention mechanisms to weight the significance of each word in the input and takes advantage of parallelization to process all the input data at once (Vaswani et al, 2017). Due to a perfect storm involving huge open-access databases of internet text, an outpouring of investments from tech giants, and faster compute with new cloud-based TPU and GPU processors, these LLMs have become the golden child of deep learning.
2. Evaluating language understanding
I asked GPT-NeoX, an open-source version of GPT, if LLMs have human-level language abilities.[1] Here are a some of the more coherent sentences from its response:
It is a complex task to define the human ability of language. As a community, we do not have a consensus…it is not a simple task to make an argument that the human ability of language is not contained in large NLP models.
– GPT-NeoX
I hold that this model is just unthinkingly reproducing patterns from its training data and that no LLM understands language. While limited tests can make LLMs seem successful, more thorough and extended imitation games will eventually betray the system’s inadequacy. Even Yeshua Bengio, one of the godfathers of deep learning, stated that the field “hasn’t delivered yet on systems that can discover high-level representations—the kind of concepts we use in language” (Saba, 2022). Just producing human-like text is not enough, as having human-level language abilities is a symptom of understanding extra-linguistic representations like concepts and situations.
How do we know if LMs have human-level language abilities? Current research relies on both intrinsic measures to calculate the LM’s theoretical accuracy, and extrinsic measures to evaluate the model’s performance on concrete tasks. These metrics illustrate both the successes and the blind spots of LLMs. Perplexity is one key entropy-based intrinsic measure for determining how well an LM predicts an unseen test set. Lower perplexity indicates higher predictive power and accuracy. A perplexity of 10-12 is considered human-level, and GPT-3 achieves a word-level perplexity of 20.5 (Shen et al, 2017). While this is an impressive result, it is still a long way from human performance – and it required a model with 175B parameters, 45TB of training data, and $12 million in compute costs (Wiggers, 2020). The most comprehensive benchmark for evaluating LMs on actual language tasks is SuperGLUE, a composite of many tests from reading comprehension to recognizing words in context to causal reasoning (Wang et al, 2020). GPT-3 achieved a total SuperGLUE score of 71.8%, where the human baseline is 89.8%. GPT-3 also reached an accuracy of 80.1% on the Winograd schema challenge, a difficult assessment that requires the model to reason about its world-knowledge to resolve an ambiguity in a statement (Levesque & Morgenstern, 2012). Are these remarkable empirical results enough to show that LLMs have already matched or exceeded human language abilities?
A closer analysis reveals serious flaws in these findings. LMs learn by finding co-occurrence patterns in the streams of symbols from the input data. LLMs trained on a huge corpus of internet text have almost certainly encountered sentences very similar to the test prompts. This is the duplication problem: testing a model on information it has already been exposed to, like giving a student an exam they have a cheat-sheet for or have practiced many times. Unlike humans, who can understand and reason about the underlying representations connected to words, GPT-3 just looks back on its many terabytes of training data to assess how often these tokens occur together. Therefore, LMs can simulate language comprehension, although more probing always reveals the lacuna of basic understanding behind this illusion. A LLM is like one of Searle’s Chinese rooms, except no philosophical arguments are needed to establish its blindness to meaning – it is enough to just understand the model and interact with it.
A diagram of Searle’s Chinese Room. Source: Wikicomms.
In fact, the creators of GPT-3 admit that the model has an “increased potential for contamination and memorization” (Brown et al, 2020, p. 7) because the tests are present in the training data. The authors tried to reduce this contamination by eliminating exact matches of the test cases from the input data. However, GPT-3 can still exhibit (illusory) high performance by exploiting similar patterns from the training data to answer questions, even if the exact answer is not present. Even more damning is the finding that the frequency of terms in an LLM’s input data is linearly correlated with its performance on related tests, suggesting these models mostly rely on memorization-like mechanisms (Razeghi et al, 2022). Other studies found that LLM performance can be fully “accounted for by exploitation of spurious statistical cues in the dataset” (Niven & Kao, 2019), and their accuracy results from employing heuristics that only work for frequent example types (McCoy et al, 2019). Thus, LLMs cannot make generalizable, novel, and robust linguistic inferences beyond statistical associations.
Abstract art based on an AI in Searle’s Chinese Room, generated with Midjourney.
Viewing LLMs as sophisticated search engines that scan through their training data can help explain why they perform well on some tasks and fail on others. Many examples show that GPT-3 lacks conceptual understanding and has no idea what the words it uses mean. For instance, it fails to keep track of objects and characters in stories, thinks a swimsuit is appropriate attire for a courtroom when “clean” is in the sentence, states that grape juice is poison when “sick” appears nearby, and wanders into irrelevant nonsense in any response longer than a few sentences (Marcus and Davis, 2020). The original paper also shows that the model cannot infer basic logical relationships between two sentences and cannot do any math but simple arithmetic that can be memorized from tables in the training data (Brown et al, 2020). Further, the same tests used to assess human linguistic capabilities reveal the failures of LLMs. On an extensive set of psycholinguistic tests, BERT (a major LLM) struggles with pragmatic inferences, shows context insensitivity, cannot predict clearly implied events, fails to prefer true over false completions for sentences on category membership, and generally only succeeds when it can exploit loopholes in the training data (Ettinger, 2020). Whenever uniquely human language abilities are tested, LLMs malfunction.
Ultimately, this analysis shows that even the most advanced LLMs do not understand language. Further, their successes are only possible with the aid of a human intelligence to cue the model with well-designed prompts, collect enough relevant training data, and scan through the generations for appropriate responses. This makes LLMs a human-in-the-loop system, which cannot exhibit linguistic capabilities without a person to guide the model. LLMs may employ similar computational structures as human language, for as Futrell (2019) finds, the “behavior of neural language models reflects the kind of generalizations that a symbolic grammar-based description of language would capture” (p. 1). However, the fact that LLMs fail on unfamiliar or untrained prompts suggests that they use a simpler and more rigid grammar than human language, where “even slight changes may cause the [program] to fail” (Granger, 2020, p. 27). Larger models simply allow the LLM to hide its inability to understand for longer intervals. Of course, LLMs also lack a basic element of language – communicative intent. They do not express meaningful intentions or try to interact with their prompters, instead just babbling about what they are prompted to babble about, often in seemingly random and contradictory directions. Therefore, LLMs can be seen as a kind of sophisticated search engine that crawls over its input data for matches. They can memorize and recall but do not reason or understand.
2. Why LLMs will not understand language, and how other models could
Beyond this evidence on the limitations of current LLMs, more theoretical arguments show that this kind of system cannot reach human-level language understanding. Critics of connectionism have long argued that language relies upon an underlying “language of thought,” involving representations with systematicity and combinatorial structure (Fodor & Pylyshyn 1988; Fodor 1998). Although these are important considerations, my arguments do not depend on these claims and are targeted specifically at LLMs. The fundamental problem is that deep learning ignores a core finding of cognitive science: sophisticated use of language relies upon world models and abstract representations. Systems like LLMs, which train on text-only data and use statistical learning to predict words, cannot understand language for two key reasons: first, even with vast scale, their training and data do not have the required information; and second, LLMs lack the world-modeling and symbolic reasoning systems that underpin the most important aspects of human language.
The data that LLMs rely upon has a fundamental problem: it is entirely linguistic. All LMs receive are streams of symbols detached from their referents, and all they can do is find predictive patterns in those streams. But critically, understanding language requires having a grasp of the situation in the external world, representing other agents with their emotions and motivations, and connecting all of these factors to syntactic structures and semantic terms. Since LLMs rely solely on text data that is not grounded in any external or extra-linguistic representation, the models are stuck within the system of language, and thus cannot understand it. This is the symbol grounding problem: with access to just formal symbol system, one cannot figure out what these symbols are connected to outside the system (Harnad, 1990). Syntax alone is not enough to infer semantics. Training on just the form of language can allow LLMs to leverage artifacts in the data, but “cannot in principle lead to the learning of meaning” (Bender & Koller, 2020). Without any extralinguistic grounding, LLMs will inevitably misuse words, fail to pick up communicative intents, and misunderstand language.
Art based on the symbol grounding problem for AI, generated by Midjourney.
Research on language acquisition shows that how children learn is strikingly different from the LLM training process. Infants learn language by drawing on a wide range of cues, while LMs only train on the tiny slice of the world in their input texts. When children are forced to use a more LLM-like learning process, limited to a single input modality and deprived of social interaction, they fail to learn language. For instance, Kuhl (2007) shows that infants quickly learned Mandarin words with live exposure to native speakers but learned almost nothing from TV or audio alone.
Further, statistical learning alone is not enough to ‘crack the speech code,’ as children need varied and frequent interactions with other agents in social situations to grasp the meanings of symbols (Kuhl, 2011). Indeed, it seems that a critical mechanism for language learning is joint attention, when a child and a teacher are focusing on the same thing and both aware of this (Baldwin & Moses, 1994). Recent research shows that how much babies follow other peoples’ gaze when speaking predicts their vocabulary comprehension 7-8 months later (Brooks & Meltzoff, 2005). Language is a system for communicating intents to real people in the real world, and the lexical similarity and syntactic structure of raw text are not enough to learn this system. LLMs are missing some key aspects of human language: these models are not part of a linguistic community, they have no perception or model of the world beyond language, they do not act as agents or express intentions, and they do not form beliefs about propositions. (At least as far as we know – and given their structure, attributing mental properties like beliefs and intentions to LLMs is not warranted unless we have very strong evidence to do so).
In defense of LLMs, some argue for the scaling hypothesis, the idea that high-level abilities like language can arise just by increasing the number of basic computational elements. As Granger (2020) argues, intelligence may be mostly a product of allometric scaling of brain size, and “human-unique and ubiquitous abilities, very much including language, arise as a (huge and crucial) qualitative difference originating from a (colossal) quantitative change” (32). Indeed, the performance of neural LLMs has scaled in a power-law relationship with model size, data size, and the amount of compute used for training (Kaplan et al, 2020). However, I do not need to dispute this hypothesis. It may be the case that sheer scale – the right quantity of the right kind of basic computational elements – is sufficient for human-level language understanding, but the kind of scale and the type of circuitry involved in LLMs will not singlehandedly achieve this milestone. Further, LLMs already have enormous scale—the recent Megatron-Turing model has 530 billion parameters and took the equivalent of 1558 trans-American flights in energy costs for training (Simon, 2021). Human babies access enormous amounts of high-definition data in many modalities, and LLMs cannot match either the quality or quantity of this information. How many more resources do we need to throw into scaling before we realize the LLM approach will not achieve full language understanding?
3. Conclusions and alternate approaches
Machine language understanding is still possible in principle, even if this popular current approach is a dead end. For example, deep learning could take inspiration from the 4E framework for cognition: to reach human-level understanding, machines must be embodied, embedded, enactive, and extended (Borghi et al, 2013). One way to implement this approach would be to integrate an LLM as just one module in a larger system, like a robotic agent in a reinforcement-learning environment. The agent could use the LLM’s powerful text processing capabilities as needed but could also use other modules to process different sensory modalities, interact with other agents, and take actions. By embedding the agent in a more real environment, it may over time learn how to elicit all of these sub-systems for the optimal behavior – potentially including language understanding. Building up a model of the world could allow the agent could connect the word embeddings from the LLM to external referents like objects and actions. Thus, embodied cognition could be fruitful approach to solving the grounding problem for machine language understanding.
A diagram showing how neuro-symbolic AI and deep neural networks (as in LLMs) could be combined into an integrated system. Source: Knowable Magazine.
Another approach argues that LLMs only need to be augmented with modules for symbolic reasoning and world modeling. Advocates for this approach argue that LLMs function like the System 1 of human cognition, performing fast, heuristic, but often flawed inferences, and just need to be supplemented with a more deliberative System 2 module. This System 2 could represent and update world knowledge with hierarchical Bayesian modeling, a promising approach to cognition (Tenenbaum et al, 2011). As human children seem to use built-in templates like intuitive psychology and physics to learn, these models will likely need to be preprogrammed with some basic theories informed by scientific research (Lake et al, 2017). For instance, one MIT team combined GPT-3 with a symbolic world state model to dramatically improve the coherence of the LLM’s text generation (Nye et al, 2021). These neuro-symbolic systems can harness the power of deep learning while rectifying its shortcomings.
Conclusively, LLMs alone cannot solve language. However, I may be wrong about this. It is risky to be an AI skeptic, as many naysayers have already been proven wrong. This paper does not make an unfalsifiable philosophical argument, but a prediction about the future of AI. Larger LMs will be highly impactful, but banal. These models may allow us to automate many routine linguistic tasks, but will not understand language or be “smart” in a way current LLMs are not. If it turns out LLMs can reach human-level language abilities, this will teach us a great deal, indicating that we can learn everything for language understanding by simply training on text data. Progress on LLMs can inform our theory of language, and psychology and linguistics should inform the development of LLMs. This interdisciplinary process is our best hope of instilling human language understanding in machines.
Appendix: The Aftermath of ChatGPT
I wrote this essay before ChatGPT was released and GPT-3 was improved with the new text-davinci-003 model. I stand by the arguments here, but of course I’ve updated my thinking after the undeniably incredible performance of GPT-3. It has achieved extraordinary capabilities with a lot of scale and tons of compute. However, as Murray Shanahan writes in the excellent paper Talking About Large Language Models:
“It doesn’t matter what internal mechanisms it uses, a sequence predictor is not, in itself, the kind of thing that could, even in principle, have communicative intent, and simply embedding it in a dialogue management system will not help. We know that the internal mechanisms of LLMs are not sensitive to things like the truth of the word sequences it predicts. It does not refer to any external “ground” for evaluating the meaning of these words.”
– Shanahn, Talking About Large Language Models, page 5.
One of the key points here is that humans can make a causal connection between words and phenomena in the real world. On the other hand, LLMs can only make a correlation between words and other words. For instance, when you ask ChatGPT “what country is to the east of Yemen,” it will answer correctly – “Oman.” However, this is not because it has built a sophisticated model of the geography of the world, or because it has developed the belief that Oman is east of Yemen. Rather, it’s just that tokens like ‘Yemen,’ ‘east,’ and ‘Oman’ were paired the most frequently in that order and context in ChatGPT’s text corpus. The model answers this question in the same way it would answer the prompt “twinkle twinkle” with “little star.” Both are simply correlation-based statistical predictions. To a human, these are two distinct types of question. One is about the real world, and one is simply pairing some text with the most likely completion. To an LLM, they’re not different.
These correlations are accurate enough that ChatGPT is almost always right. But there are hundreds of examples where the model simply confabulates, hallucinates, or otherwise bullshits the answer. It cannot think from first principles, make epistemic judgements based on experience, or compare its beliefs to a model of the world. This is why its text-based educated guessing of what word should come text become unhinged from reality. Language is learned by talking to other language-users while immersed in a shared world and engaged in joint activity. Without this, the LLM cannot develop human-level language abilities. One of the most dangerous abilities of ChatGPT is its ability to make up plausible-sounding bullshit, that looks right but is in fact wrong – for example, coming up with fake info security answers, or creating regex answers with subtle flaws.
However, it might be the case that LLMs alone can achieve language understanding. It is possible that in the process of trying to perform sequence prediction, the LLM stumbled upon emergent mechanisms that warrant higher-level descriptions like “knowledge,” “belief,” or “understanding.” Perhaps in all of this large-scale statistical learning, the LLM discovered that it could predict tokens better if it also represented the connections between words and stored a latent representation of the world described by these words. Maybe all that is needed to understand human language is contained within language itself. In other words, maybe human language is reducible to next token prediction. If we create a performant enough LLM with enough training data, it will be able to perfectly simulate language understanding. This essay argues that this outcome is unlikely for structural reasons. But it is still possible – we have been shocked by AI research before.
How can we tell if an LLM really does understand? There is no way to prove its ability to understand human language, as even millions of successful examples where it seems to understand could be disproven by a single edge case where it clearly does not understand. ChatGPT can clearly pass something like the Turing Test in most cases, but it fails (showing its true colors as an AI) in some other cases. What success rate is enough to justify calling it “understanding”? These are all important questions, and the answers are still unclear.
Works Cited
Granger, R. (2020). Toward the quantification of cognition. arXiv preprint arXiv:2008.05580.
Borghi, A. M., Scorolli, C., Caligiore, D., Baldassarre, G., & Tummolini, L. (2013). The embodied mind extended: using words as social tools. Frontiers in psychology, 4, 214.
Nye, M., Tessler, M., Tenenbaum, J., & Lake, B. M. (2021). Improving coherence and consistency in neural sequence models with dual-system, neuro-symbolic reasoning. Advances in Neural Information Processing Systems, 34.
Tenenbaum, J. B., Kemp, C., Griffiths, T. L., & Goodman, N. D. (2011). How to grow a mind: Statistics, structure, and abstraction. Science, 331(6022), 1279-1285.
Lake, B. M., Ullman, T. D., Tenenbaum, J. B., & Gershman, S. J. (2017). Building machines that learn and think like people. Behavioral and brain sciences, 40.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., … & Polosukhin, I. (2017). Attention is all you need. Advances in neural information processing systems, 30.
Bender, E. M., & Koller, A. (2020, July). Climbing towards NLU: On meaning, form, and understanding in the age of data. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (pp. 5185-5198).
Harnad, S. (1990). The symbol grounding problem. Physica D: Nonlinear Phenomena, 42(1-3), 335-346.
Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., … & Amodei, D. (2020). Scaling laws for neural language models. arXiv preprint arXiv:2001.08361.
Kuhl, P. K. (2011). Early language learning and literacy: neuroscience implications for education. Mind, brain, and education, 5(3), 128-142.
Kuhl, P. K. (2007). Is speech learning ‘gated’ by the social brain?. Developmental science, 10(1), 110-120.
Baldwin, D. A., & Moses, L. J. (1994). Early understanding of referential intent and attentional focus: Evidence from language and emotion. Children’s early understanding of mind: Origins and development, 133-156.
Ettinger, A. (2020). What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language models. Transactions of the Association for Computational Linguistics, 8, 34-48.
Minsky, M. L. (1974) A framework for representing knowledge. MIT-AI Laboratory Memo 306. [aBML]
Futrell, R., Wilcox, E., Morita, T., Qian, P., Ballesteros, M., & Levy, R. (2019). Neural language models as psycholinguistic subjects: Representations of syntactic state. arXiv preprint arXiv:1903.03260.
Fodor, J. A., & Pylyshyn, Z. W. (1988). Connectionism and cognitive architecture: A critical analysis. Cognition, 28(1-2), 3-71.
Fodor, J. A. (1998). Concepts: Where cognitive science went wrong. Oxford University Press.
Shanahan, M. (2022). Talking About Large Language Models. arXiv preprint arXiv:2212.03551.
Shen, X., Oualil, Y., Greenberg, C., Singh, M., & Klakow, D. (2017). Estimation of Gap Between Current Language Models and Human Performance. In INTERSPEECH (pp. 553-557).
Wang, A., Pruksachatkun, Y., Nangia, N., Singh, A., Michael, J., Hill, F., … & Bowman, S. (2019). Superglue: A stickier benchmark for general-purpose language understanding systems. Advances in neural information processing systems, 32.
Razeghi, Y., Logan IV, R. L., Gardner, M., & Singh, S. (2022). Impact of pretraining term frequencies on few-shot reasoning. arXiv preprint arXiv:2202.0720
Black, S., Biderman, S., Hallahan, E., Anthony, Q., Gao, L., Golding, L., … & Weinbach, S. (2022). Gpt-neox-20b: An open-source autoregressive language model. arXiv preprint arXiv:2204.06745. ↑
Imagining gives us our freedom. Or so I argue in this paper, which aims to describe the neural basis of imagination and its role in free will perceptions. First, I review imagination on all three of Marr’s levels of analysis: its computational function, its algorithmic structure, and its neural implementation (Marr, 1982). Then, I argue that the capacity to imagine alternative possibilities is essential to perceiving oneself as acting freely. I also show that the imagination is not free and unconstrained, but has systematic constraints, and these limit our ability to act volitionally and choose among possibilities. Further, expanding the imagination results in a greater perception of free will. By imagining we can make ourselves even more free.
1. What is Imagination?
The term “imagination” presents a challenge for researchers, since it is used colloquially in a variety of ways, its meaning is the subject of intense debates in philosophy and other fields, and “imagination” does not have an agreed-upon formalized definition in mathematics, computer science, or cognitive science. However, a minimal shared concept of imagination allows us gloss over the differences between the many sub-types of imagination, from mental imagery of visual scenes to propositional supposing, and focus on the commonalities.
Specifically, imagination is mental simulation: the ability to simulate non-occurrent possibilities, representing something in the mind without aiming to capture things as they actually are in the moment. In other words, imagination is to “represent without aiming at things as they actually, presently, and subjectively are” (Liao and Gendler, 2011). Thus, imagination can be understood as a form of “attention to possibilities,” where potential realities are projected, simulated, and operated upon internally (Williamson 2016, 4). In computational terms, imagination refers to a system’s processing or manipulation of information that is not directly present to the system’s sensors (Marques, 2009). This ‘imaginative’ processing occurs offline (when the agent is not receiving new sensory data or is not connected to its environment), covertly (without immediate consequences for the agent’s actions), and/or internally (occurring within the agent’s own latent models). Generally, the defining feature of imagination is the simulation of non-occurrent sensory states with internal representations or models.
The human capability to imagine is vital to a wide range of cognitive processes, including memory, predicting the future, conjuring alternate worlds, simulating and empathizing with other minds, spatial navigation, and inventing novel combinations of images and objects (Mullaly & Maguire, 2014). As expected of a process with so many diverse roles, imagination varies on several dimensions.[1] Specifically, imaginings can be voluntary (e.g. creative generation) or involuntary (e.g. daydreaming). Some even apply the dual-process framework to imagination, dividing imaginative processes into the (1) unconscious, uncontrolled, spontaneous, non-volitional, and effortless and (2) the conscious, controlled, volitional, and effortful (Stuart, 2021). Further, imagination can be sensory, as in mental imagery, or cognitive and non-sensory, as in imagining yourself with alternative beliefs or traits (Dokic, & Arcangeli, 2014). This sensory aspect can implicate any modality, from vision and touch to smell and sound. Finally, the complexity of the imagined object can vary dramatically. Imagination often involves representing a complete situation, “a configuration of objects, properties, and relations” rather than a single isolated object (Berto, 2018). It can even involve constructing an entire imaginary world. This paper will touch on imagination in all its aspects but will focus especially on the voluntary generation and manipulation of imagined possibilities.
2. The Computational Role of Imagination
At the computational level, we ask what imagination does, and why: the role imagination plays in the human information system and its adaptive function in our environment (Bechtel & Shagrir, 2013). As stated above, imagination’s primary role is world-simulation: it generates a model of the world or a part of the world and simulates it in internal experience. Imagination takes sensory data, memories, and our implicit and explicit models of the world as input, and it outputs an imagining – most often, a consciously experienced internal simulation.
Imagination has myriad adaptive functions. Critically and perhaps most obviously, imagining is crucial for decision-making. It allows us to simulate the consequences of actions, allowing our imaginings to ‘die for us’ so that we do not have to make choices by costly trial-and-error alone (Kielak, 2019). Further, the imagination can facilitate escaping local maxima in a decision environment: situations where no single choice could put you in a better position, but a series of choices (potentially through a long ‘desert’ of low reward) could improve your situation significantly. If we were unable to imagine the high-value oasis at the end of the low-reward desert, it would be far more difficult to escape these sub-optimal, least-bad situations. Gaesser (2013) also shows that imagination has a crucial role in enabling empathy and social cognition, supporting theory of mind, and in encouraging prosocial behavior, where more vivid and detailed imaginings of a person more effectively promoted altruistic actions toward them. Earlier in human evolution, imagination was likely crucial to creating novel tools and traps, mentally planning attack strategies, and improving tribal cohesion with religion, myth, and art (Vyshedskiy, 101). Imagination is therefore vital to human behavior.
More passive modes of imagining, like daydreaming and dreaming, may allow us to draw connections and integrate disparate information, incubate potentially creative ideas, and move experiences to long-term memory (Malinowski & Horton, 2015). Further, they may even serve a ‘defensive activation’ role: dreaming keeps visual areas in the brain active and engaged to prevent them from being reduced or replaced (Eagleman & Vaughn, 2021). Among the many functions of imagination, this paper focuses on two that are especially vital to making choices and perceiving oneself as free: imagination’s role in creativity and modal cognition.
1.2 Imagination and creativity
While imagination is closely connected to creativity, it is a separate process. Creativity is the process of producing ideas, artifacts, or concepts that are both novel and valuable. Imagination is the ability to produce and/or simulate new objects, sensations, or ideas in the mind, and can be understood as both a sub-process within creativity and as a semi-separate capacity that supports creative generation. As the initial generative step in creativity, imagination produces the creative possibilities that are then considered, evaluated, and implemented by other systems. Imagination produces internal representations that will not necessarily be novel or useful (creative), but that can provide a fertile starting point for the creative process. Ellamil et al (2012) finds that the two phases of generation and evaluation involved in creativity implicate distinct neural systems: creative generation recruits primarily medial temporal lobe regions like the hippocampus, while evaluation co-recruits the default mode and executive control networks. Further, these networks are competitive: “the more successfully [participants] were able to engage in creative generation while avoiding evaluative processes, the more they recruited MTL regions associated with creative generation” (p. 6). This neuroscientific evidence supports the hypothesized computational role of imagination in creativity: it generates loose, unrefined ideas to be evaluated, modified, and polished by other cognitive processes. Free choice relies on this relatively divergent, unstructured initial step to produce creative options, which can then be winnowed down and selected from in a convergent process.
1.2 Imagination and modal cognition
Imagination is closely tied to modal cognition – thinking about possibilities. Modal cognition relies on imagination to represent situations and generate potential alternatives. Just as in creativity, imagination is the initial step in modal cognition, as it generates the possibilities for consideration. The possibilities in the generated consideration set can then be partitioned into a more limited set of relevant possibilities, and ordered based on some criteria, like value or probability. Considering the ways a captain could have prevented a ship from sinking, for instance, requires mentally simulating this scenario and varying its features to produce alternative possibilities. If we were unable to generate and represent the alternative possibilities for a given situation, it would be difficult or impossible to see ourselves as free. Section 4 expands on the importance of imagination in modal cognition for free will perceptions.
3. The Algorithm of Imagination: Generative Models
The representational or algorithmic level asks how information is organized, encoded, and processed in the imagination, transforming representations into an imagined output. I argue that generative models serve as the fundamental algorithm of imagination. Imagination uses rules and implicit models of the world learned through perception to generate a limitless variety of possibilities. A generative model estimates the probability distribution of an observed variable given a target variable, in contrast to discriminative models that estimate a target variable’s probability distribution based on observed variables. In other words, a generative model simulates the interactions among unobserved variables that might generate the observed variables. Rather than just creating input-output mappings or categorizing a signal, generative algorithms attempt to figure out how the data was generated to classify it, asking which target category is most likely to have produced the observation. By understanding the methods of generation, these models can also create new data similar to the observed data.
Williams (2020) provides detailed arguments to show that imagination and perception are best described as generative models. Discriminative models are unable to explain top-down effects in perception (where higher-level representations impact processing of early info) or endogenously generated percepts like mental imagery and dreams (which have no clear inputs for classification). Generative models correlate to the widely-accepted predictive processing framework in neuroscience, as they are prolific expectation-generators that allow continuous predictions of incoming sensory information based on estimates of their external causes. The brain likely uses temporal generative models, which use current observations and perceptual history to make inferences and find dependencies in input patterns that appear in timed order, to predict its future sensory stream.
The imagination co-opts this predictive capacity of perception and re-uses its core representational architecture, modifying our implicit, learned representations of the dynamics of the real world to generate imagined worlds with new or altered dynamics. Extensive evidence supports the theory that the brain uses a hierarchical generative model to “minimize prediction error in the cascade of cortical processing,” and higher-level areas can use these generative models to drive lower neural populations into predicted patterns and produce internal perception (Clark, 2013). Thus, the cortex likely implements a generative model to explain, predict, and learn about sensory data, and then cross-applies this model to synthesize rich visual representations without external input.
Treating imagination as a generative model is valuable for a few additional reasons. First, imagination is generally governed by principles of generation: a set of (implicit or explicit) rules that guide our imaginings (Walton, 1990, p. 53). For example, in Harry Potter, “Latin words and wands create magic” is a principle of generation that readers can consistently use to simulate the imagined world. The imagination generates a set of possibilities guided by context-relevant principles, like graphics rendering algorithms that unfold an artificial world procedurally using algorithmic rules. Treating imagination as a generative model also explains imaginative mirroring: our imagination defaults to follow the rules of the real world unless prompted otherwise by principles of generation (Leslie, 1994). If a cup ‘spills’ in an imaginary tea party, the participants will treat the spilled cup as empty, following the normal physics of reality. This occurs because perception involves generative models, using processes we derive from experience to simulate the physical world and predict its behavior. Imagination involves running a generative model ‘alongside’ or ‘on top’ of this internal simulation of reality. Some processes are modified in the imagining, but the ones that are not modified are ‘filled in’ by our default generative model of the real world.
The efficacy of generative models for explaining the imagination is demonstrated by computational models that simulate imagination. Generative models based on artificial neural networks (ANNs) can visualize objects that the network has never seen before, replicating the correctness, coverage, and compositionality of the human imagination (Lee et al, 2008). An ANN can learn the structure of an environment and then simulate or hallucinate it internally, but this process relies on creating an efficient, compressed, thorough, and interpretable model of the world (Ha & Schmidhuber, 2018). For instance, Testolin & Zorzi (2016) show that human perception is analogous to graphical models implemented with generative ANNs, which build high-level representations and extract statistical regularities from the environment in an unsupervised way and use feedback connections to carry-top down expectations. These generative models have psychologically and biologically plausible properties, like unsupervised learning and interactions between feedback and feed-forward activity.
Reichert et al (2013) demonstrates that generative models can explain human internal imagery, showing that the cortical dynamics of spontaneous hallucinations in Charles Bonnett syndrome (CBS) can be simulated and explained by an ANN-based generative model. In CBS, partial blindness results in a deficiency of visual input in early processing stages, resulting in spontaneous activity in the cortex. The authors show that recurrent connections between layers in ANNs are similar to reciprocal synaptic connections between layers in the neural visual processing hierarchy and enable simulating the balance between bottom-up sensory information and top-down internal priors that occurs in the brain. When the trained ANN is given empty or corrupted input, this results in realistic artificial hallucinations that can be strikingly decoupled from the input image. These examples are fascinating demonstrations of the potential of using generative models to facilitate progress in understanding the mechanisms of hallucinations, mental imagery, and perception in the human brain.
Conclusively, the generative model framework offers a fruitful way to understand the imagination. It also suggests the algorithmic components that the imagination involves, which likely correspond to somewhat separate neural correlates. For instance, the imagination requires a sensory system to collect information about the world and support simulations of it. It also needs a memory system to consolidate these experiences into representations that can be accessed for future imaginings. Then, some sub-system must support compressing a huge number of observations of reality into a generative world model, so that imagination can use this model to create realistic and task-relevant simulations. Finally, there must be some internal workspace that allows the mind to produce, combine, and manipulate imagined objects.
3. The Neural Correlates and Mechanisms of Imagination
Any complete model of imagination must accurately and comprehensively describe how imaginings are produced by complex interactions of neuron assemblies, regions, and networks in the human brain. How do neural circuits create an experienceable representation of an object that is not currently present in the subject’s sensory environment?
3.1 Perception and imagination
Imagination has many parallels with perception. This is unsurprising given our theoretical framework that suggests perception and imagination both involve similar generative models. For instance, research showed that in people with visual disorders, imagination is disabled in the same way as perception – e.g. people with hemispatial neglect cannot imagine things on the neglected side, suggesting that imagery and perception use the same machinery (Koch, 2004, p. 99). Furthermore, more vivid imaginings dilate your pupil more, suggesting that the imagination activates very early perceptual processes (Laeng, 2014). The excitability of the visual cortex also predicts imagery strength (Keogh & Pearson, 2020). Additionally, binocular rivalry experiments conducted by Tartaglia et al (2009) demonstrate that just perceiving something (like an oriented line) can improve your visual sensitivity to that thing, imagining visual content improves your sensitivity to that content. This priming effect indicates that imagination involves processes similar to perception. Finally, the contents of imaginings can be mostly decoded with activity in the early visual cortices like V1 and V2 (Vetter et al, 2014), showing that the representations of imagined objects are partially realized in early sensory areas.
As described in Pearson (2017), top-down imagination functions like a weak version of perception with a “reverse visual hierarchy” (p. 2): imagining begins with an initial conscious choice to create a mental image in the frontal lobe, producing a cascade of activity that runs ‘backwards’ in the brain, retrieving stored info and memories in medial temporal areas, and then finally, sensory and spatial representations of the imagery are created in the parietal and occipital lobes.[2] Additionally, the hippocampus can recruit long-term memories to help give richness and spatial coherence to complex, large imaginings (Buckner, 2010). After all, imagination is closely linked to memory, and more vivid imagery is linked to better performance in visual working memory tasks (Keogh & Pearson, 2014). While perception involves feed-forward information propagating upward from early visual areas, imagination involves a feedback cascade that begins in frontal regions and then recruits memories and visual areas to produce imaginings.
Finally, frontal regions play an executive rule in guiding the imagination, but do not produce the actual imagined content. As the patterns of activity associated with imagination move up from V1 to frontal areas, they become increasingly similar to the neural patterns of perception (Pearson 2017, p. 3). This is likely because the executive control mechanisms and high-level processes involved in triggering imagination are nearly indistinguishable from the ones involved in processing, modeling, and manipulating feed-forward visual information. Attention to perceptual realities and attention to possibilities therefore seem to implicate the same neural mechanisms in frontal-parietal areas.
3.2 Imagination and the Default Mode Network
There is a growing consensus that remembering the past, imagining the future, counterfactual thinking, and simulating possible experiences, all involve similar neural mechanisms in the default mode network (DMN) (Hassabis & Maguire 2017; Mullaly & Maguire, 2014; Pearson, 2019; Addis et al, 2007; Spreng et al, 2009). The DMN is a collection of brain areas often activated during wakeful rest and internal mental activity, and includes the medial prefrontal cortex, the posterior cingulate cortex or precuneus, and the angular gyrus, among other regions (Raichle, 2015). Winlove et al (2018) review 40 neuroimaging studies in a meta-analysis of the correlates of visual imagery, and identified 11 consistently activated regions, finding that the superior parietal lobule was involved in top-down control of imagery, the inferior frontal sulcus semantic processing and working memory, and the frontal eye fields and V1 supported internal visual depictions. Further, Whittingstall et al (2014) show that the posterior cingulate cortex (PCC) is a crucial hub for integrating occipital, parietal, and temporal areas together during visuospatial imagery.
Imagining future events and prospective thinking involved the same generation processes and areas in the right frontopolar cortex and left ventrolateral prefrontal cortex, showing that the episodic memory system is involved in imagining the future and vice versa (Addis & Schacter, 2007). More specifically, future-oriented and counterfactual thinking engages the posterior DMN (pDMN), centered around the posterior cingulate cortex (Xu et al, 2016). Researchers showed this by asking participants in an fMRI scan to make choices about their present situation, and then prospective choices about their future. Their findings demonstrated that people often engage vivid mental imagery in future-oriented thinking, and that this process activates the pDMN while reducing its connectivity with the anterior DMN. This provides a candidate neural process that underlies imaginative generation of possibilities. However, imagination requires not just the DMN, but organized interactions between the DMN, executive control network (ECN), and salience networks to create controlled, meaningful, and actionable imaginings (Gotlieb et al, 2018). The DMN may be essential to generating images, ideas, and possibilities, while other networks allow us to modify, select amongst, and move our attention between them.
3.3 Imagining as binding-by-synchrony
A key cognitive ability that underlies imagination is prefrontal synthesis (PFS), the ability to create novel mental images by combining experienced or remembered objects. The binding-by-synchrony hypothesis claims that this process is performed in the lateral prefrontal cortex (LPFC), which likely acts as an executive controller that synchronizes a network of neuronal ensembles (NEs) that represent familiar objects, synthesizing these objects into a new imaginary experience (Vyshedskiy, 2019). Familiar objects are encoded in the brain by neuronal ensembles, and the sensory component of objects is physically encoded in “the posterior cortical hot zone” (Koch & Tononi, 2016). Remembering or imagining objects requires synchronous resonant activity of the object-encoding neuronal ensembles, and when this synchrony occurs in the posterior cortical hot zone it causes the object to come to consciousness.
Imagining novel things, then, is the processes of synchronizing independent object-NEs through conscious attention. Objects can then be imaginatively modified by desynchronizing parts of an object-NE from the whole (called prefrontal analysis). The LPFC acts as a puppeteer in this process, flexibly synchronizing object-NEs to manufacture an unlimited number of novel mental images (Vyshedskiyp. 99). For example, an ensemble representing Bill Clinton and one representing a lion can be synthesized by synchronizing their firing activity in the same phase, creating a mental image of Clinton holding a lion (Vyshedskiy & Dunn, 2015). Any arbitrary type and number of ensembles can be synchronized in the mental workspace, limited by working memory, experience, and focus. Imagination can be either top-down and intentional, driven by the prefrontal synchronization of lower-level neuronal assemblies, or bottom up and unintentional, when lower-level ensembles synchronize non-volitionally and without a puppeteer, spontaneously producing dreams, hallucinations, or sudden insights and images. Children acquire PFS around 3 to 4 years of age, along with other imaginative abilities like mental rotation, storytelling, and advanced pretend play (Vyshedskiy, 2019, p. 101). While further study is needed, it is plausible that development of PFS is associated with mature modal cognition, advanced creative abilities, and generating more sophisticated imaginings.
Creative thought relies on the ability to manipulate internal representations flexibly in a mental workspace. Schlegel et al (2013) confirm Winlove et al’s finding that 11 regions consistently are activated in imagination, including the occipital cortex, PPC, precuneus, posterior inferior temporal cortex, DLPFC, and frontal eye fields. However, this research also showed that maintenance and manipulation of imagined objects involved separate sub-networks, where maintaining involved a dense network integrated by the MTL and manipulation involved a sparser network with a hub in the precuneus. This supports the hypothesis for separate neural mechanisms for imaginative synthesis (forming and maintaining a mental image or object) and analysis (applying operations, filters, or decompositions to the imagined objects). Imagination relies on dynamically synchronizing neural assemblies in the mental workspace.
Finally, imagination is an example of type 3 qualia, which is the temporary binding of simple sensory objects (type 1 and 2 qualia) through endogenous attention (Tse, 2017). In daydreaming, mental imagery, and imagination, we can simultaneously experience the contents of the iconic buffer (our current sensory state) in the attentional background, and the contents of the working memory buffer in the foreground (Tse, 2017, p. 17). In contrast, dreams are an example of experiencing type 3 qualia alone, without external inputs or basic sensory. Binding-by-synchrony, and the idea that imaginings are type 3 quales, also explains why imagination only seems to represent an object while your attention is currently focused on it. Unlike perception, in which sensory areas are activated by external inputs, top-down imagination requires the constant, effortful synchronization of neural ensembles to maintain mental objects. When your attention moves, the synchronization collapses, and the imagined object vanishes.
4. Imagination and Free Will Perception
Imagination is fundamental to seeing oneself as a free agent. Here, I do not take a position on the complex and rife debates on free will, compatibilism, and determinism in philosophy. I do not argue that imagination is a literal precursor to free will in any deep metaphysical sense, but rather that it is indispensable to our perception of free will, bracketing away the question of whether this perception is an illusion or not. I support this position with several arguments. The most central argument claims that to represent or see oneself as choosing freely, one must be able to represent alternative possibilities for actions. Representing alternative possibilities requires imagination. Thus, imagination is required for free will perception. Seeing yourself as free requires representing or imagining alternative possibilities for action. Additionally, this implies that the systematic constraints on what possibilities we imagine restricts the choices we can make and limits our sense of free will.
“Free will also includes the creative ability to imagine,” as we can choose to apply our attention to internally generated qualia (Tse, 2013, p. 238). This enables freedom in two ways. First, imagining and mentally ‘playing out’ possible scenarios to form a plan for action is precisely what empowers us to make decisions. By creating an internal virtual reality, it allows us to pre-select and pre-experience actions, an essential part of human decision-making. Second, imagination itself involves choices among internal representations, even if this does not manifest in external actions. The imagination enables freedom in the sense that it supports generating a large number of possibilities with a “high degree of disorder or chance amounting to a kind of ‘freedom’” (Krausz & Bardsley, 2009, p. 133). In this sense, the very manner of representation and the kinds of mental operations involved in imagination are fundamentally intertwined with freedom. While memory and imagination involve similar processes, prospection (simulating the future) is less constrained and subject to ‘reality checks’ than retrospection (Kane et al, 2008, p. 132). Future-looking imagination is vital to volitional action.
4.1 Imagining alternative possibilities and its constraints
Research on modal cognition shows that imagining alternative possibilities is not free and boundless but has important constraints. By default, we only consider a systematically limited subset of the imaginable possibilities. Imagination produces a series of possibilities, and then during decision-making we sample from this distribution of imagined options in an adaptive way, constrained by relevant factors. The set of possibilities we consider is limited systematically by the sampling process (Morris, Phillips, and Cushman 2019). Under the theory of the psychological representation of modality developed by Phillips et al, the set of possibilities we consider is limited by the constraints of probability/normality, physics, and morality (Phillips & Knobe, 2018). For instance, both children and adults under severe time constraints tend to consider immoral options (e.g. stealing or lying) or unlikely and irregular options (e.g. painting polka dots on an airplane) as impossible (Phillips, Morris, & Cushman, 2017). Our perceptions of ourselves as freely acting are systematically limited by the possibilities we can imagine. Although we may be able to choose among possibilities, we do not have complete control over the pool of possibilities that are consciously available to us. Through imagination, we can modify and expand this pool of options, supporting a greater sense of agency and freedom.
We tend to judge agents as free when we can represent alternative possibilities for their action. In a sense, a failure to imagine can preempt free will perceptions, as one cannot choose to act upon a possibility that one does not represent, and one cannot see an option as ‘freely’ chosen if no other possibilities are represented. Indeed, people use judgements of possibility to inform judgements about whether an agent is free (Phillips & Knobe 2018). Generally, if we are able to imagine situations where the action could be different, we judge the agent as free. When participants generated more possibilities, imagining more alternative decisions a ship captain could have made, they were more likely to make the judgement that he was free and not forced (Phillips, Luguri, and Knobe, 2015). Unpublished data from the Dartmouth PhilLab supports this finding, suggesting that as people imaginatively generate more possibilities, these options become less constrained by the norms of probability, normality, morality, and rationality.[3] This may imply that possibilities become more divergent, unconventional, novel, or surprising as the quantity of ideas generated increases. Therefore, imagination is essential to free will judgements, and imagination enhances the sense of freedom by expanding the set of accessible options.
The ability to imaginatively project alternative possibilities may therefore underly individual differences in free will perceptions – if a person imagines many more available options, they see themselves as freer. Simply imagining more possibilities may engender a feeling of more freedom. Developmental research provides strong support for this claim.
Children tend to resist, or fail to generate, impossible and improbable imaginings. Kushnir (2018) also shows that free will beliefs originate in the ability to understand intentional action, inferring when agents are free to do otherwise and when they are constrained. Young children are often unable to imagine alternatives to improbable, irregular, or immoral events, and so tend to see them as impossible. Children’s imaginations are thus surprisingly reality-constrained: children (age 2-8) protest against pretense that contradicts their knowledge of regularity, expecting imaginary things to have ordinary properties (Friedman et al, 2017). Even when pretending, kids expect lions to roar and pigs to oink, and they resist imagining otherwise. Children also protest against pretense that contradicts their knowledge of regularity, expecting imaginary entities to have ordinary properties (Vandervoort and Friedman, 2017). Furthermore, 82% of the time, children extend fantasy stories with realistic events rather than fantastic events, while adults extend fantasy stories with fantastic events (Weisberg et al, 2013). Young children imagine along ordinary lines even when primed with fantastical contexts, filling in typical and probable causes for fantastical imaginary events (Lane et al, 2016). Children show a strong typicality bias in completing fictional stories, favoring additions to the story that match their regular experiences in reality (Thorburn et al, 2020). This evidence shows that children’s imaginations are limited by typicality, morality, and their understanding of the physical world. This suggests they are using simpler constraints, quick heuristics, and a more basic model of the world to effortlessly generate possibilities.
Most conclusively, an experiment by Flanagan and Kushnir (2019) found that performance on a task that involved generating ideas within an imagined fantasy world was the best predictor of children’s free will judgements: the more fluent the children were in this imagination task, the more likely they were to judge themselves as free. As the authors speculate, “one potential mechanism is a direct pathway from idea generation to judgments of choice and possibility” (p. 5). In my view, the pathway is not completely direct, as existing research indicates that after possibility-generation (imagination) we also evaluate the relevance of possibilities and rank them. However, the initial generation is crucial, and the nature and quantity of generated possibilities has demonstrable impacts on how people think about possibilities, freedom, and choice. Constraints on the imaginative process lead to downstream effects on our choices and our perceptions of freedom.
As children develop, they are able to soften these constraints and imagine more alternatives, and when they do endorse a choice as free rather than forced they often cite imagined alternatives to the scenario as an explanation (Kushnir, 2018). As children develop, the constraints on their imagination relax, leading to less restricted generation of possibilities. Older children are more likely to imagine improbable and physically impossible phenomena (Lane et al, 2016, p. 6). Explicitly prompting children to generate more possibilities leads them to imagine more like older children, producing possibilities less constrained by probability and regularity (Goulding & Friedman, 2020). Cultural contexts mediate this developmental process. For example, American children are more likely than Nepalese and Singaporean children to judge that they are free to act against cultural and moral norms (Chernyak and Kushnir, 2019). This is likely because children in cultures with stronger or more restrictive norms find it harder to generate evaluatively wrong possibilities or see these possibilities as relevant. As free will judgements depend on representing alternative possibilities, these children see themselves as less free to pursue possibilities that violate evaluative norms. When imaginative flexibility increases with age and experience, we can represent a wider range of possibilities for action and cultivate a broader conception of our own free will.
Viewing imagination as a generative model furthers fruitful interpretations of this research. When imagining, young children apply a generative model with the same rules of generation used in perception to produce expectations about reality. This early imagination may use simple constraints and empirical heuristics to allow effortless and rapid generation of possibilities. For instance, if the child regularly encounters an event, they are more likely to imagine this event (Goulding & Friedman, 2020). In later development and adulthood, the imagination generates possibilities in a more deliberative and analytical way. This suggests a dual process model of imagination (Stuart, 2019). Children may use a more uncontrolled, effortless, and unconscious imagination based on simple heuristics and experience-derived rules of generation. In contrast, adults use a more controlled, effortful and conscious imagination that generates possibilities based on relatively sophisticated and principled rules.
Conclusively, the default representation of imagination results in resistance to imagining possibilities that violate physical laws, irregular or unlikely possibilities, and immoral or evaluatively bad possibilities. Experimental results reveal that the imaginations of young children are limited by precisely these constraints. Adults are able to deliberately generate more and less constrained possibilities. With very limited time or significant cognitive pressure, adult imaginations may resemble the imaginations of young children. However, just as adults can treat immoral possibilities as irrelevant, imaginative resistance shows that the adult imagination is inhibited against immoral possibilities. Finally, individual differences in openness to experience, creativity, and imaginative ability may predict some of the variation in judgements of possibility and freedom. For instance, people who are naturally more imaginative (and thus generate more possibilities) will be more likely to judge agents as free rather than forced.
4.2 Imagination and existential freedom
To imagine enables free consciousness, because it allows you to get beyond the real, developing a broader perspective on the world by imagining beyond it and escaping from it to some degree (Turner, 1968). The imagination is a radical break from the surrounding world, a negation of present circumstances, making-present something that is not there by making-absent what is ‘really’ there (Sartre, 2010). As Husserl writes, free phantasy (imagination) allows one to see more possibilities and attain a wider-ranging knowledge of experience (Husserl 2012, §70). Our sensory lives give us access only to a small selection of possible experiences, and thus we need imagination to explore the immensity of conceivable configurations of experiences, choices, and perceptions. The imagination can supplement our experience, and in turn we can use experiences to pollinate the imagination and enable coherent world-simulations.
This sheds light on an important debate in existentialism. Sartre claimed that human consciousness is able to transcend any given situation by pursuing the possibilities we imagine. He thought that we are radically, infinitely free to choose our possibilities (2015, p. 112). We can define our identity with negation, through the set of possibilities we reject. In contrast, Heidegger had a much limited view of human freedom. He thought that our world, and our set of available possibilities, is defined by social structures that are out of our control. The They (Das Man), or basically our social context, limits the set of possibilities we are capable of considering (Heidegger, 2010, §27). Certain possibilities will never be available to us, not just because we cannot factually achieve them, but because we cannot even conceive them. For him, freedom is the process of personally appropriating one of these socially given options, and authenticity consists in becoming one’s possibilities. Both perspectives are true to some extent: we have immense freedom to generate and choose amongst our imagined possibilities, but these possibilities are also limited by our social context and our cognitive abilities.
Furthermore, imagination is essential to the process of flexible identity-construction: developing a sense of oneself, seeing aspects of one’s identity, and moving towards a hopeful future self (Gotlieb et al, 2018). A sometimes forgotten aspect of free will is that to perceive yourself as acting freely, you must already perceive yourself as an agent. By allowing us to picture ourselves in the future, counterfactually vary features of our identity, and imagine a constant thread of who we are through our lifetimes, imagination supports the cross-temporal identity that is essential to seeing oneself as a free agent. Finally, imagination may enable a kind of existential creativity: an individual’s attitude of exploring life’s possibilities and experimenting with life-plans and versions of herself (Loi & Plas, 2020). Imagination “permits one to take the paths of many varied and opposed ways of thinking,” creating “the excess that gives to the free spirit the dangerous privilege of living for experiments and of being allowed to offer itself to adventure” (Nietzsche, 1996, #4). Imagining alternative future tracks and ways of life, and narratively constructing an identity that persists through these disparate pasts and possibilities, is crucial to a person’s ability to forge a meaningful life.
4.3 Disorders of imagination
It may even be the case that certain disorders increase free will perceptions by amplifying imaginative abilities, facilitating unexpected connections and more unpredictable mental pathways. While at its extreme this can lead to psychosis, it also amplifies the exploratory processes essential to generating alternative possibilities for choice. Both bipolar and ADHD are associated with significantly higher openness to experience (Van Dijk et al, 2017; Quilty, 2009). Openness is linked to trait creativity, is even used as a measure of creativity, and is associated with higher volume in brain regions that inhibit control and reduce constraint (Li, 2015). The highly-open personalities of patients with disorders like ADHD and bipolar may facilitate highly associative, fluent, and originative brainstorming of possibilities.
Looser cognitive limitations, weakened top-down control, and more unconstrained thinking may also potentiate imagination and free will perceptions in certain disorders. Creative tasks benefit from a state of hypofrontality, in which reduced PFC activation enables more spontaneous, bottom-up thought patterns (Ramey & Chrysikou, 2014). Bipolar patients exhibit disruptions in the frontoparietal control network which reduce top-down constraints, and mania and involves hypofrontality, a “significant attenuation of task-related activation of right lateral orbitofrontal function” that results in disinhibition and distractibility (Altshuler et al, 2005). Further, individuals with ADHD have impaired executive inhibition, which reduces the person’s ability to suppress creative but unconventional ideas – and ADHD patients exhibit improved performance on tasks like the Unusual Uses Test (White & Shaw, 2006). People with mental disorders associated with impulsivity like bipolar, Tourette’s, and ADHD often have more fluent and vivid imaginations, and are biased toward generation over evaluation (Ellamil, 2012). Therefore, the disinhibited imaginations of people with certain disorders may allow them to brainstorm more actionable possibilities. However, this may also explain the pathological aspect of these disorders: it may be harder to select appropriate actions given a far larger pool of possibilities, including irrelevant, unfit, or harmful ones. Limiting the number of projected possibilities is therefore likely adaptive – imaginative constraints are the bonds that set us free.
Finally, aphantasia is a well-documented disorder that involves the absence of a ‘mind’s eye,’ where otherwise normal, healthy individuals report a complete lack of visual experience when they attempt to imagine something (Keogh & Pearson, 2018). There are also degrees of aphantasia – it can involve an impaired imagination with reduced strength, control, or vividness, rather than a complete lack of imaginings. At opposite end of the imaginative spectrum is hyperphantasia, an exceptional strength, control, and vividness of imagination. One of the testable predictions of my theory is that if imagination does indeed play a crucial role in the perception of freedom, then there will be significant differences in the free will perceptions and judgements of aphantasic and hyperphantasic individuals. Specifically, higher scores on tests of imaginative ability will correlate with greater perceptions of free will.
4.4 Imagining to increase agency
Imagination can also be thought of as a trainable ability, which can be practiced to improve self-efficacy, self-control, and agency. For example, individuals who are more skilled at counterfactual thinking are more easily able to self-restrain and delay gratification in the service of later reward (Mischel et al, 2011). Imagination breaks our constant present-orientation and task-focus, moving us into more flexible, open, and future-oriented mode of internal reflection that is crucial for long-term decision-making (Gotlieb et al, 2018). Being able to imagine the future allows us to resist current temptations and focus on long-term goals. Imagination supports the perception of free will, and in turn, an increased belief in free will changes the way persons imagine their futures – promoting a focus on personal agency and interpersonal connection in prospective imaginings (Nagelmann, 2019). Imaginative skill thus promotes a sense of freedom.
Finally, social norms are one of the most powerful sources of constraints on the imagination. Just as a child who is only given certain props and stories will naturally shape their pretend play around these objects and narratives, adults mold their imaginings by their socio-cultural environment. The existing structures of the world can congeal in our minds, ossifying until they seem almost unshakeable, and are not even realized as constraints. So perhaps, for example, “it is easier to imagine the end of the world than it is to imagine the end of capitalism” (Fisher, 2009, p. 8). Overcoming the constraints of collective imagination, the rigid social orthodoxies that tell us what is and is not possible, can have transformative force – liberating entire populations to act more freely because they realize their actions are not as constrained as they thought (Dey & Mason, 2018). Disruptive truth-telling, courageous speech, and utopian imagination in the style of MLK, Ghandi, or Mandela can therefore literally enhance our sense of free will by increasing the number of possibilities for action and boosting their cognitive availability.
Conclusion
The ability to imagine is a core component of consciousness. “Imagination is a specifically human form of conscious activity” (Vygotsky, 1967) which distinguishes us from other organisms, supporting our ability to generate complex mental representations and reconfigure them into innumerable combinations. Imagination dominates consciousness both in duration and degree. The average person spends between 30% and 50% of their waking time daydreaming (McMillan et al, 2013), and even more conscious time is occupied engaged in prefrontal synthesis, dreaming, operating in the mental workspace, or simulating the future. Further, imagination represents a peak of consciousness, where endogenous attention is actively and volitionally applied to synchronize lower-level neural ensembles into complex internal simulations. Research into the neural correlates of consciousness must therefore treat the imagination as a central question.
In this paper, I have reviewed the neural basis of the imagination, from its computational role and diverse functions in supporting creativity, decision-making, and modal cognition, to its algorithmic structure in generative models, to its implementation in the brain through reversed perceptual processes, the default mode network, and binding-by-synchrony. Further, I have argued that imagination is central to the perception of free will. Without imagination, free will is unimaginable. Without it, we would not be able to represent alternative possibilities, simulate the consequences of our actions, or construct an identity through time and envisage different ways of living and being. Imagination also explains some of the systematic ‘soft’ limitations on our free will, which prevent us from acting on options that we cannot or do not imagine. By building our imaginative capabilities we can transform our personal and societal futures.
Works Cited
Abraham, A. (Ed.). (2020). The Cambridge handbook of the imagination. Cambridge University Press.
Addis, D. R., Wong, A. T., & Schacter, D. L. (2007). Remembering the past and imagining the future: common and distinct neural substrates during event construction and elaboration. Neuropsychologia, 45(7), 1363-1377.
Altshuler, L. L., Bookheimer, S. Y., Townsend, J., Proenza, M. A., Eisenberger, N., Sabb, F., … & Cohen, M. S. (2005). Blunted activation in orbitofrontal cortex during mania: a functional magnetic resonance imaging study. Biological psychiatry, 58(10), 763-769.
Bechtel, W., & Shagrir, O. (2015). The non‐redundant contributions of Marr’s three levels of analysis for explaining information‐processing mechanisms. Topics in Cognitive Science, 7(2), 312-322.
Berto, F. (2021). Taming the runabout imagination ticket. Synthese, 198(8), 2029-2043.
Buckner, R. L. (2010). The role of the hippocampus in prediction and imagination. Annual review of psychology, 61, 27-48.
Chernyak, N., Kang, C., & Kushnir, T. (2019). The cultural roots of free will beliefs: How Singaporean and US Children judge and explain possibilities for action in interpersonal contexts. Developmental psychology, 55(4), 866.
Clark, A. (2013). Whatever next? Predictive brains, situated agents, and the future of cognitive science. Behavioral and brain sciences, 36(3), 181-204.
Cook, C., & Sobel, D. M. (2011). Children’s beliefs about the fantasy/reality status of hypothesized machines. Developmental Science, 14(1), 1-8.
Dokic, J., & Arcangeli, M. (2014). The heterogeneity of experiential imagination. Open MIND. Frankfurt am Main: MIND Group.
Eagleman, D. M., & Vaughn, D. A. (2021). The defensive activation theory: REM sleep as a mechanism to prevent takeover of the visual cortex. Frontiers in neuroscience, 15.
Ellamil, M., Dobson, C., Beeman, M., & Christoff, K. (2012). Evaluative and generative modes of thought during the creative process. Neuroimage, 59(2), 1783-1794.
Fisher, M. (2009). Capitalist realism: Is there no alternative?. John Hunt Publishing.
Flanagan, Teresa, and Tamar Kushnir (2019). “Individual differences in fluency with idea generation predict children’s beliefs in their own free will.” In CogSci, pp. 1738-1744.
Gaesser, B. (2013). Constructing memory, imagination, and empathy: a cognitive neuroscience perspective. Frontiers in psychology, 3, 576.
Gotlieb, R. J., Hyde, E., Immordino-Yang, M. H., & Kaufman, S. B. (2018). Imagination is the seed of creativity. The Cambridge Handbook of Creativity. New York, NY: Cambridge University press.
Goulding, B. W., & Friedman, O. (2020). Children’s beliefs about possibility differ across dreams, stories, and reality. Child development, 91(6), 1843-1853.
Ha, D., & Schmidhuber, J. (2018). Recurrent world models facilitate policy evolution. arXiv preprint arXiv:1809.01999.
Hassabis, D., Kumaran, D., & Maguire, E. A. (2007). Using imagination to understand the neural basis of episodic memory. Journal of neuroscience, 27(52), 14365-14374.
Heidegger, M. (2010). Being and time. Suny Press.
Hunt, M., & Fenton, M. (2007). Imagery rescripting versus in vivo exposure in the treatment of snake fear. Journal of Behavior Therapy and Experimental Psychiatry, 38(4), 329-344.
Husserl, E. (2012). Ideas: General introduction to pure phenomenology. Routledge.
Kane, J., McGraw, A. P., & Van Boven, L. (2008). Temporally asymmetric constraints on mental simulation: Retrospection is more constrained than prospection. The handbook of imagination and mental simulation, 131-149.
Keogh, R., & Pearson, J. (2014). The sensory strength of voluntary visual imagery predicts visual working memory capacity. Journal of vision, 14(12), 7-7.
Keogh, R., & Pearson, J. (2018). The blind mind: No sensory visual imagery in aphantasia. Cortex, 105, 53-60.
Keogh, R., Bergmann, J., & Pearson, J. (2020). Cortical excitability controls the strength of mental imagery. elife, 9, e50232.
Kielak, K. (2019). Generative Adversarial Imagination for Sample Efficient Deep Reinforcement Learning. arXiv preprint arXiv:1904.13255.
Koch, C., Massimini, M., Boly, M., & Tononi, G. (2016). Neural correlates of consciousness: progress and problems. Nature Reviews Neuroscience, 17(5), 307-321.
Koch, Christof. (2004). The quest for consciousness: a neurobiological approach. Roberts & Co.
Kushnir, T. (2018). The developmental and cultural psychology of free will. Philosophy Compass, 13(11), e12529.
Laeng, B., & Sulutvedt, U. (2014). The eye pupil adjusts to imaginary light. Psychological science, 25(1), 188-197.
Lane, J. D., Ronfard, S., Francioli, S. P., & Harris, P. L. (2016). Children’s imagination and belief: Prone to flights of fancy or grounded in reality?. Cognition, 152, 127-140.
Lee H, Ekanadham C, Ng Ay (2008). Sparse deep belief net model for visual area V2. Advances in Neural Information Processing Systems, 20.
Leslie, A. M. (1994). Pretending and believing: Issues in the theory of ToMM. Cognition, 50(1-3), 211-238.
Liao, S. Y., Strohminger, N., & Sripada, C. S. (2014). Empirically investigating imaginative resistance. British Journal of Aesthetics, 54(3), 339-355.
Liao, Shen-yi and Tamar Gendler (2011). “Imagination.” The Stanford Encyclopedia of Philosophy.
Loi, M., Viganó, E., & van der Plas, L. (2020). The societal and ethical relevance of computational creativity. arXiv preprint arXiv:2007.11973.
Malinowski, J. E., & Horton, C. L. (2015). Metaphor and hyperassociativity: the imagination mechanisms behind emotion assimilation in sleep and dreaming. Frontiers in Psychology, 6, 1132.
Marques, H. G., & Holland, O. (2009). Architectures for functional imagination. Neurocomputing, 72(4-6), 743-759.
Marr, D. (1982). Vision: A computational investigation into the human representation and processing of visual information. San Francisco: W.H. Freeman.
McMillan, R., Kaufman, S. B., & Singer, J. L. (2013). Ode to positive constructive daydreaming. Frontiers in psychology, 4, 626.
Mischel, W., Ayduk, O., Berman, M. G., Casey, B. J., Gotlib, I. H., Jonides, J., et al. (2011). Willpower over the life span: decomposing self-regulation. Soc. Cogn. Affect. Neurosci. 6, 252–256.
Mullally, S. L., & Maguire, E. A. (2014). Memory, imagination, and predicting the future: a common brain mechanism?. The Neuroscientist, 20(3), 220-234.
Nagelmann, Stina. “How do attitudes on free will relate to students’ imagination of the future?.” Bachelor’s thesis, University of Twente, 2019.
Nietzsche, Friedrich. (1996). Human, all too human: A book for free spirits. Cambridge University Press.
Pearson, J. (2019). The human imagination: the cognitive neuroscience of visual mental imagery. Nature Reviews Neuroscience, 20(10), 624-634.
Phillips, J., & Knobe, J. (2018). The psychological representation of modality. Mind & Language, 33(1), 65-94.
Phillips, J., Morris, A., & Cushman, F. (2019). How we know what not to think. Trends in cognitive sciences, 23(12), 1026-1040.
Phillips, Luguri, and Knobe (2015). “Unifying morality’s influence on non-moral judgments: The relevance of alternative possibilities.” Cognition 145, 30-42.
Quilty, L. C., Sellbom, M., Tackett, J. L., & Bagby, R. M. (2009). Personality trait predictors of bipolar disorder symptoms. Psychiatry Research, 169(2), 159-163.
Raichle, M. E. (2015). The brain’s default mode network. Annual review of neuroscience, 38, 433-447.
Ramey, C. H., & Chrysikou, E. G. (2014). “Not in their right mind”: the relation of psychopathology to the quantity and quality of creative thought. Frontiers in psychology, 5, 835.
Reichert, D. P., Series, P., & Storkey, A. J. (2013). Charles Bonnet syndrome: evidence for a generative model in the cortex?. PLoS computational biology, 9(7), e1003134.
Reichert, D. P., Series, P., & Storkey, A. J. (2013). Charles Bonnet syndrome: evidence for a generative model in the cortex?. PLoS computational biology, 9(7), e1003134.
Sack, A. T., van de Ven, V. G., Etschenberg, S., Schatz, D. & Linden, D. E. J. Enhanced vividness of mental imagery as a trait marker of schizophrenia? Schizophr. Bull. 31, 97–104 (2005).
Sartre, J. P. (2010). The imaginary: A phenomenological psychology of the imagination. Routledge.
Sartre, J. P. (2015). Being and nothingness. Central Works of Philosophy: The twentieth century: Moore to Popper, 4, 155.
Schlegel, A., Kohler, P. J., Fogelson, S. V., Alexander, P., Konuthula, D., & Tse, P. U. (2013). Network structure and dynamics of the mental workspace. Proceedings of the National Academy of Sciences, 110(40), 16277-16282.
Shine, J. M. et al. Imagine that: elevated sensory strength of mental imagery in individuals with Parkinson’s disease and visual hallucinations. Proc. R. Soc. B 282, 20142047 (2014).
Shtulman, A., & Phillips, J. (2018). Differentiating “could” from “should”: Developmental changes in modal cognition. Journal of Experimental Child Psychology, 165, 161-182.
Spreng, R. N., Mar, R. A., & Kim, A. S. (2009). The common neural basis of autobiographical memory, prospection, navigation, theory of mind, and the default mode: a quantitative meta-analysis. Journal of cognitive neuroscience, 21(3), 489-510.
Stuart, M. T. (2021). Towards a dual process epistemology of imagination. Synthese, 198(2), 1329-1350.
Tartaglia, E. M., Bamert, L., Mast, F. W., & Herzog, M. H. (2009). Human perceptual learning by mental imagery. Current Biology, 19(24), 2081-2085.
Testolin, A., & Zorzi, M. (2016). Probabilistic models and generative neural networks: Towards an unified framework for modeling normal and impaired neurocognitive functions. Frontiers in Computational Neuroscience, 10, 73.
Thorburn, R., Bowman-Smith, C. K., & Friedman, O. (2020). Likely stories: Young children favor typical over atypical story events. Cognitive Development, 56, 100950.
Tse, P. (2013). The neural basis of free will: Criterial causation. MIT Press.
Turner, R. (1968). Sartre and Ryle on the Imagination. South African Journal of Philosophy, 20-28.
Van de Vondervoort, J. W., & Friedman, O. (2017). Young children protest and correct pretense that contradicts their general knowledge. Cognitive Development, 43, 182-189.
Van de Vondervoort, Julia W., and Ori Friedman,” Young children protest and correct pretense that contradicts their general knowledge,” Cognitive Development 43 (2017): 182-189.
Van Dijk, F. E., Mostert, J., Glennon, J., Onnink, M., Dammers, J., Vasquez, A. A., … & Buitelaar, J. K. (2017). Five factor model personality traits relate to adult attention-deficit/hyperactivity disorder but not to their distinct neurocognitive profiles. Psychiatry research, 258, 255-261.
Vetter, P., Smith, F. W., & Muckli, L. (2014). Decoding sound and imagery content in early visual cortex. Current Biology, 24(11), 1256-1262.
Vygotsky, L. S. (2016). Play and its role in the mental development of the child. International Research in Early Childhood Education, 7(2), 3-25.
Vyshedskiy, A., & Dunn, R. (2015). Mental synthesis involves the synchronization of independent neuronal ensembles. Research Ideas and Outcomes, 1, e7642.
Vyshedskiy, Andrey. “Neuroscience of imagination and implications for human evolution” (2019). Preprint DOI: 10.31234/osf.io/skxwc.
Walton, K. L. (1990). Mimesis as make-believe: On the foundations of the representational arts. Harvard University Press.
Weisberg, D. S., Sobel, D. M., Goodstein, J., & Bloom, P. (2013). Young children are reality-prone when thinking about stories. Journal of Cognition and Culture, 13(3-4), 383-407.
White, H. A., & Shah, P. (2006). Uninhibited imaginations: creativity in adults with attention-deficit/hyperactivity disorder. Personality and individual differences, 40(6), 1121-1131.
Williams, D. (2021). Imaginative constraints and generative models. Australasian Journal of Philosophy, 99(1), 68-82.
Williams, D. (2021). Imaginative constraints and generative models. Australasian Journal of Philosophy, 99(1), 68-82.
Williamson, T. (2016). Knowing by imagining. Knowledge through imagination, 113-23.
Winlove, C. I., Milton, F., Ranson, J., Fulford, J., MacKisack, M., Macpherson, F., & Zeman, A. (2018). The neural correlates of visual imagery: A co-ordinate-based meta-analysis. Cortex, 105, 4-25.
Xu, Xiaoxiao, Hong Yuan, and Xu Lei, “Activation and connectivity within the default mode network contribute independently to future-oriented thought,” Scientific reports 6 (2016): 21001.
Figures and images
Figure 1 – the hierarchy of types of imagination, split by top-down and bottom-up imagination.
Figure 2 – an illustration of the ‘reverse perception’ process of imagination.
Figure 3 – An illustration of the binding by synchrony hypothesis.
Figure 4 – Chart showing that as more possibilities are generated, the possibilities increasingly deviate from the constraints of morality, normality, probability, and rationality. For instance, the 1st item generated is given a probability rating of about 6.3, while the 8th item generated is given a probability rating of about 5. This shows the importance of the quantity of ideas generated for escaping constraints and conceptual limitations during the brainstorming process. Based on unpublished data from the Dartmouth PhilLab, project by Jonathan Phillips, Eliza Jane, Margaret Garrard, and Maeen Arslan.
Heidegger's Being and Time will be cited as BT with marginal pagination.
Disclosing the world is a precondition for any engagement or concern with the world, as it makes the ready-to-hand “accessible for circumspective concern” (BT 76). Something must light up the world, making its totality of references, assignments, and tools available to us. But how is the world lit up or disclosed? Through the inseparably connected components of the care-structure, including attunement, understanding, fallenness, and discourse. This essay focuses on attunement, perhaps the most fundamental part of the care-structure, as it is what makes things to matter to Dasein in the first place (BT 137). Section 1 reconstructs Heidegger’s account of attunement and moods in the context of his broader existential analytic. Section 2 addresses some major methodological concerns for his account. Ultimately, Heidegger’s analysis of attunement illuminates key ontological structures of our experience and remains relevant even in a modern scientific context.
1. Attunement and Mood
Heidegger distinguishes between two concepts: an attunement or state-of-mind (Befindlichkeit), and a mood (Stimmung).[1] Unfortunately, Heidegger does not explicitly delineate these terms, and often uses them interchangeably. One interpretation is that attunement is the ontological existentiale, while mood is the ontic manifestation of attunement. In less technical terms, attunement is the fundamental condition that allows us to experience the world as meaningful and ‘mooded.’ Mood is the term for more specific modes of attunement, like fear, anxiety, joy, anger, or focus. Moods are therefore derivative from attunement. Perhaps Heidegger does not need to distinguish between the two. After all, we never experience some abstract, free-floating, or content-free attunement. Instead, we are always experiencing a specific, concrete mood. Attunement is a concept for describing the character of moods in general, as they all share a common structure. What are the characteristics of this structure?
An intuitive view is that moods are occasional, transient emotional experiences that affect us temporarily. One can be more or less moody, or feel a particularly strong mood, but moods are not constant features of our experience. For Heidegger, moods are far more fundamental. We are always already in a mood, and “we are never free from moods” (BT 136). Dasein is Being-in-the-world: it is always absorbed in and engaged with a web of references and assignments that make a totality of equipment ready-to-hand (BT 76). Moods make things accessible to us as equipment, making them meaningful. For instance, a mood like “focus” reveals this laptop as a tool for-the-sake-of the project of writing this essay. I am able to encounter only what a mood has already disclosed to me. Moods thereby disclose the worldhood of the world.
Moods allow us to “encounter something that matters to us” (BT 138). In this sense, moods color the world. However, this metaphor is misleading, as it suggests attunement simply tinges or tints objects that are already revealed. As Schopenhauer writes, “subjective mood—the affection of the will—communicates its color to the purely viewed surroundings.”[2] For Heidegger, moods are not just tinted lenses that give already-revealed objects some emotional color. Attunement, the structure of mood, is more like an atmosphere than a tinted lens: moods are always present, even if not visible, and are necessary for any experience of the world whatsoever.[3]Attunement is how the world opens up to me – whether it is opened up as a burden, a fearful place, or a wonderland. For instance, fearfulness is the mood which allows me to discover threatening objects (BT 138). Furthermore, a mood is not from inside or outside the mind, “but arises out of Being-in-the-world” (BT 176). Heidegger again rejects the distinction between subject and object, as it “splits the phenomenon asunder” (BT 132). Moods are neither inner nor outer, within nor without, objective nor subjective. Rather, moods condition the way we encounter things within the unitary phenomena of Being-in-the-world.
Lee, “Stillwinds #8”, acrylic on canvas. For Heidegger, art has a unique ability to communicate a mood.
Heidegger’s reasoning about attunement could fit into the pattern of a transcendental argument: (1) Being-in-the-world is the basic structure of experience as Dasein; (2) in Being-in-the-world, things are disclosed as meaningful and ready-to-hand; (3) there must be some way these things are disclosed and made meaningful; (4) attunement is a name for the way things are disclosed and made meaningful to Dasein.[4] Therefore, attunement is an ontological precondition for our experience of the world. As Heidegger puts it, “only because the ‘there’ has already been disclosed in a state of mind [attunement] can immanent reflection come across ‘experiences’ at all” (BT 136). Moods are not just a kind of experience or a way of being intentionally directed. Instead, moods are a condition that makes experience possible, making it “possible first of all to direct oneself toward something” (BT 137). This is why attunement is necessary for experience in general, and not just affective or emotional experience.
2. Methodological Problems for Heidegger’s Analysis
The first problem for Heidegger’s concept of attunement is a methodological one. If we are always already in a mood, it follows that even Heidegger’s existential analytic must be carried out in some mood. Therefore, we can ask what makes his mood, or any mood, existentially authoritative. Since moods condition experience in different ways, perhaps Dasein will reveal itself differently depending on the mood of the phenomenologist. Is there a ‘right’ mood for uncovering the real ontological structures of Dasein?
Initially, it is clear that Heidegger rejects the idea of a ‘pure’ phenomenology devoid of mood. For example, through the neutrality modification, Husserl aimed to “suspend everything connected to the will” to achieve a purer phenomenological method.[5] Heidegger argues that this is misguided. There is no pure, mood-free experience of objects, as mood is a precondition for being receptive to objects at all. Not “even the purest theory has left all moods behind it” (BT 138). We cannot get outside of moods and observe them from some external vantage point. Every investigation must have some mood that makes the objects of investigation accessible and meaningful.
Heidegger emphasizes that this does not mean we “surrender science ontically to ‘feeling’” (BT 138), but it does seem methodologically problematic for an existential analytic if ‘universal’ ontological structures are only visible in certain moods. One can understand why phenomenologists seek neutrality, to avoid this methodological subjectivity. A defender of Heidegger’s approach can make several responses. First, even if we only “see the ‘world’ unsteadily and fitfully in accordance with our moods” (BT 138), this may be the only way to analyze being as it truly manifests itself. If the investigation of being turns out to be mood-dependent and tumultuous, then so be it. We should not falsify our experience and create artificial uniformity, treating Dasein as always present-at-hand, just because this would make phenomenology seem more objective. Second, the existentiales Heidegger identifies are present regardless of mood: in “every state-of-mind…Being-in-the-world should be fully disclosed” (BT 191). Even if we are not explicitly aware of structures like understanding, Self, or the World, they still condition our experience. Indeed, Being will often be disguised and “covered up” to us (BT 35). Perhaps an in-depth analysis can reveal structures that are not visible in our average everydayness, but that are always present as ontological structures. Presumably, these structures will be recognizable in every mood, although in different ways and to different degrees.
Furthermore, not all moods are equal in their disclosure of Dasein. Information about Dasein is accessible to us through attunements, and more primordial attunements offer a greater possibility of accurately interpreting Dasein’s Being (BT 185). Heidegger argues that anxiety (angst) is the most primordial and disclosive attunement. Unlike fear in the face of some extant entity, we have anxiety in the face of Being-in-the-world as such, which is indefinite, unknown, and nowhere. Just as when our tools break, we become aware of them as present-at-hand objects, when our world breaks down, we become are aware of it as a world. Through anxiety, we see the networks of meaning we are normally absorbed in, realize our individuality and being-thrown, and recognize our freedom to live inauthentic or authentic possibilities. Anxiety also provokes feelings of uncanniness and homelessness in our once-familiar world. Thus, we usually flee from it, absorbing ourselves in projects and entities to “dim down” or tranquilize the anxiety (BT 189). Our ceaseless avoidance reveals the constant presence and primordiality of anxiety, showing that Dasein is anxious in the “very depths of its Being” (BT 190). Anxiety is therefore a primordial mood that can encourage authenticity and enable the analysis of Dasein.
Digital art by Kyle Kerr. Angst is a mood that can disclose our authentic being and open up our possibilities.
However, Heidegger leaves serious methodological questions unanswered. Despite using the term “primordial” 371 times in B&T, he never offers a method for determining whether a phenomenon is more primordial than another. His evidence that anxiety is a primordial attunement rests on the claim that we are always fleeing from it. However, even if this is accepted as a phenomenologically apt description, it is not clear why this implies that anxiety is more primordial. Even more critically, Heidegger suggests that anxiety as a primordial mood is more disclosive – it offers us privileged epistemic access to Dasein and the worldhood of the world. Why does the fact that we flee from an attunement imply that it is primordial, and why does its primordiality imply that the attunement is more disclosive? In claiming that anxiety discloses primordial Being, Heidegger seems to be begging the question: he presupposes some significant knowledge of primordial Being. Without this preexisting knowledge, it is hard to see how Heidegger could claim that anxiety discloses more of the reality or primordiality of Being.[6] While perhaps we have an implicit awareness of Being that enables us to begin an investigation of Dasein (BT 7), Heidegger is assuming a much richer understanding of Being here.
Furthermore, it is not clear why a phenomenon like fallenness is not more primordial than anxiety. After all, it almost universally present, and being-fallen is the mode of being that we occupy proximally and for the most part. In contrast, “‘real’ anxiety is rare” (BT 190). We flee toward fallenness, and away from anxiety (BT 189).Why should the phenomena we flee away from be more primordial than the phenomena we flee toward? Often, it seems that Heidegger labels a phenomenon “primordial” to communicate normative preferences rather than descriptive claims about the reality of Being. This leaves serious concerns: how can we resolve epistemic disputes about the primordiality of phenomena? More generally, why should we accept Heidegger’s characterizations of Being? The primary method he employs is a description of phenomena in our experience, and logical analysis to make conclusions about Being based on these phenomena. At least to some degree, Heidegger relies on the aptness and explanatory power of his descriptions of our experience. Thus, the validity of his “fundamental ontology” is dependent on the resonance of his words in describing the human condition, and seems to be an aesthetic activity analogous to that of a novelist or fiction writer.
Shoes, Van Gogh (painting). Heidegger describes this painting as disclosing an entire life-world. Perhaps his own theory can be taken as an artistic depiction of the nature of Being, and not a rigorous ontological investigation.
Finally, in Heidegger’s time, the “psychology of moods” was a new, undeveloped field which “still lies fallow” (BT 134). Today, it has grown into the far more mature field of affective science. However, Heidegger would likely criticize even a more advanced, scientific, and explanatorily successful psychology as having critical problematic assumptions and a deeply flawed starting point. The sciences treat Dasein as a present-at-hand object which can be understood in a detached theoretical attitude, and this approach inevitably falsifies the phenomena. Empirical science is a restricted mode of disclosing being, and it is not epistemologically prior. Indeed, the existentiales that Heidegger elucidates are “a priori conditions for the objects which biology takes for its theme,” and the structures examined by any science can only be understood if they are first seen as structures of Dasein (BT 58). For instance, attunements are the fundamental conditions that render the world intelligible to us, making possible logical or theoretical investigation. Ontological structures like attunement must be presupposed by the sciences and can never be fully explained by present-at-hand analysis.
As it happens, many of Heidegger’s explanations of Being have proved fruitful in the sciences, and his work influences entire research areas like embodied cognition. The existential analytic of Dasein has been ‘naturalized,’ tested, and applied as a model of the extant human brain. For example, Ratcliffe (2002) argues that Heidegger’s account is “actually required as an interpretive backdrop for neuropsychological cases,” and provides a powerful framework for modern affective science.[7] Recent findings show that moods determine how the world is opened up to us, enabling cognitive processing, decision-making, and successful reasoning. These findings show that Heidegger’s analysis has explanatory power in science as well as phenomenology. Additionally, as they reveal the inextricability of emotion from cognitive processes like logic, these findings challenge the ‘purity’ of many theoretical methods and undermine the epistemological assumptions of the sciences.
However, attempting to use science to add credibility to Heidegger’s views implicitly accepts that his claims are legitimately interpretable and even testable in a scientific context. This implies that empirical sciences can offer meaningful knowledge about Dasein, a claim Heidegger would likely reject. If the existential analytic truly has ontological priority, then it does not require empirical validation through the study of present-at-hand beings, and it cannot be treated as a merely ontic science. In the process of applying Heidegger’s ideas, the sciences therefore may violate some of his most essential philosophical principles. However, the problems discussed above raise questions for Heidegger’s own methods. These methods may not be able to fulfill his own desiderata, as they do not reveal the phenomena in a sufficiently originary way and are not clearly epistemologically prior. Instead, Heidegger’s approach, insofar as it aims for explanatory power in its description of consciousness and being, could be interpreted as continuous with the natural sciences. After all, a strict division between the study of Dasein and the present-at-hand would commit a cardinal Heideggerian sin by splitting up unitary phenomena. Just as the sciences are not a privileged conduit to reality, perhaps the existential analytic of Dasein is just one limited but insightful way of disclosing Being.
Bibliography
Elpidorou, Andreas, and Lauren Freeman. “Affectivity in Heidegger I: Moods and emotions in Being and Time.” Philosophy Compass 10, no. 10 (2015): 661-671.
Heidegger, Martin. The fundamental concepts of metaphysics: World, finitude, solitude. Indiana University Press, 1995.
Heidegger, Martin. Basic Problems of Phenomenology. Albert Hofstadter, trans. Indiana University Press, 1988.
Heidegger, Martin. Being and Time. Trans. John Macquarrie & Edward Robinson. Harper Reprint, 2008.
Husserl, Edmund. Ideas for a pure phenomenology and phenomenological philosophy: First book: General introduction to pure phenomenology. Hackett Publishing, 2014.
Schopenhauer, Arthur. The World as Will and Idea – Vol. 2. Project Gutenberg, 2015.
Polt, Richard. Heidegger: an introduction. Routledge, 2013.
Ratcliffe, Matthew. “Heidegger’s attunement and the neuropsychology of emotion.” Phenomenology and the Cognitive Sciences 1, no. 3 (2002): 287-312.
I will use “attunement” for Heidegger’s term Befindlichkeit, and “mood” for Stimmung. Many translators agree these English terms most accurately communicate Heidegger’s concepts. See Andreas Elpidorou and Lauren Freeman, “Affectivity in Heidegger I: Moods and emotions in Being and Time,” Philosophy Compass 10, no. 10 (2015): 661-671. ↑
Arthur Schopenhauer, The World as Will and Idea-Vol. 2, Project Gutenberg, 2015. Pg. 400. ↑
Heidegger, The fundamental concepts of metaphysics, pg. 45. ↑
Of course, attunement is not the only way things are disclosed – it is part of the whole care-structure. ↑
“Almost everywhere it was madness which prepared the way for the new idea, which broke the spell of a venerated usage and superstition. Do you understand why it had to be madness which did this?”
— Nietzsche, Daybreak #14
How could ‘madness’ be helpful for idea-generation, brainstorming, artistic expression, and other creative processes? Mental disorders, as neural-cognitive differences that often misalign with a social context, may enable the kinds of divergence that contribute to creativity. I argue that conditions like bipolar disorder, Tourette’s syndrome, and ADHD (the ‘C-disorders’) share features that substantially increase generative creativity. Although they may not have a common etiology, the C-disorders have important shared cognitive styles and neural patterns. Part 1 provides a theoretical framework, describing generative creativity within a dual-process model, defending its value, and considering how it can be effectively studied. Part 2 analyzes the empirical evidence indicating that the cognitive styles and neural correlates of generative creativity are exceptionally exhibited in the C-disorders. I conclude by tying together these threads and calling for a new approach to treating the C-disorders that takes these findings into account.
1. The Dual-Process Model and Generative Creativity
1.1 — What is Creativity?
Creativity lies at the intersection of novelty and value. To be creative, an idea, invention, artwork, or other product must be both useful and new. Of course, this definition is vague and subject to difficult questions. For instance, what does it mean for a creative product to be valuable? This question is subject to social and evaluative norms. Often, the standards used are social consensus, scientific-technological innovation, or material-economic benefits, but it is not clear these are necessary or sufficient. The definition of creativity, and the best construct to measure and describe it, remains hotly disputed (Ford & Harris, 1992). In part to bypass some of these theoretical issues, this essay will be restricted to a specific sub-component: generative creativity.
1.2 — The Dual Process Model of Creativity
Diagram from “A Dual-state Model of Creative Cognition for Supporting Strategies that Foster Creativity in the Classroom” by Howard-Jones (2002).
Creative thinking proceeds in phases—an initial phase of unconstrained generation or brainstorming, and a subsequent more-constrained and systematic evaluation. Under this dual-process model, creativity starts by generating a wide range of initial ideas and associations and finishes by exploring these crude options with evaluation and testing. Generative creativity is the first process. Computationally, a generatively creative system is one that creates new patterns regardless of their estimated benefit to the system, while evaluative or adaptive creativity involves creating patterns that fulfill established value-functions (Bown, 2012, pg. 364). Evidence suggests that the dual-process model has a basis in the brain, as the two phases of generative and evaluative creativity involve distinct neural systems: creative generation recruits primarily medial temporal lobe regions like the hippocampus, while evaluation co-recruits the default mode and executive control networks (Ellamil et al, 2012). Furthermore, this study finds that the generative and evaluative networks were somewhat competitive: “the more successfully [participants] were able to engage in creative generation while avoiding evaluative processes, the more they recruited MTL regions associated with creative generation.” These phases are both vital to successful creative production, but they are underpinned by diverging cognitive styles and neural correlates.
Indeed, the two processes often conflict. Brainstorming demonstrates the importance of quarantining the generative process from critical, evaluative, goal-directed, convergent mental processes. Listening to the critic in one’s head is the fastest way to make a brainstorming session crash on the runway. Unfettered generation is especially critical because “the more creative concepts you have to choose from, the better” (Adams, 2001, pg. 22). If one does not take the time for unconstrained, generation-focused, divergent thinking, it is far more likely that the creative process will be prematurely mired in conceptual blocks and arbitrary limitations. Effective brainstorming entails avoiding premature evaluations and quality checks, and instead focusing on ideational speed and fluency – producing a large number of new concepts, designs, or ideas. When it comes to creativity, quantity has a quality all its own.
1.3 — Modal Cognition and Generative Creativity
Research on modal cognition also has important theoretical import for generative creativity. Under the theory of the psychological representation of modality developed across multiple papers by Phillips et al, the initial set of possibilities we consider is limited by the constraints of probability, physics, and morality (Phillips & Knobe 2018). With limited time, we default to only considering a systematically limited subset of possibilities. For instance, both children and time-constrained adults tend to consider immoral options (e.g. stealing or lying) or unlikely and irregular options (e.g. painting polka dots on an airplane) as impossible (Phillips, Morris, & Cushman, 2017). Indeed, experimental data from the PhilLab suggests that as people generate more options, these options become less constrained by norms of probability, normality, morality, and rationality.[1] This may imply that possibilities become more divergent, unconventional, novel, or surprising as the quantity of ideas generated increases.
Figure 1: Chart showing that as more possibilities are generated, the possibilities increasingly deviate from the constraints of morality, normality, probability, and rationality. For instance, the 1st item generated is given a probability rating of about 6.3, while the 8th item generated is given a probability rating of about 5. This shows the importance of the quantity of ideas generated for escaping constraints and conceptual limitations during the brainstorming process. Based on unpublished data from the Dartmouth PhilLab, project by Jonathan Phillips, Eliza Jane, Margaret Garrard, and Maeen Arslan.
Using simple heuristics to delimit the most relevant and useful possibilities is computationally cheap, quick, and often adaptive. But for generative creativity, one must minimize constraint and mental friction to produce maximal options. Phillips theorizes that there are two processes in modal cognition: the default and the deliberative representations of possibility (2017). Perhaps generative creativity relies on the deliberative representation: as more possibilities are generated in a creative flow state, the ordinary restraints loosen, and the consideration set expands. Mental disorders may facilitate surpassing the default modal limitations, allowing unconstrained generation.
1.4 — Methodological Considerations in Creativity Studies
Empirical research on mental disorders and generative creativity should keep a few important considerations in mind. First, research should be constrained to adults. Including children and adolescents would introduce too many confounding variables, as neuroplasticity and other features of the developing brain likely influence generative creativity and interfere with attempts to isolate the effect of the C-disorders. Second, it should avoid an excessive focus on the DSM-V constructs — which are unlikely to map perfectly onto brain differences, are subject to change, and have serious conceptual and methodological problems (Hadfield, 2020). Instead, I emphasize the neuro-cognitive patterns exemplified in these disorders.
Finally, a key problem with creativity research is its focus on ‘demonstrated creativity’: concrete observable outcomes valued in a social context. For instance, creative professions, eminence, and forms of creative output are used as proxies for creativity. However, this paper is more concerned with creative processes than outcomes. Demonstrated creativity is a very ‘noisy’ measure, as actual generative creativity is filtered through social, economic, and pragmatic pressures. Therefore, it may be systematically biased against people with socioeconomic disadvantages, the mentally ill, and others for whom it is particularly difficult to conform to social criteria and fit within existing systems. Similarly, it would be misguided to measure intelligence (g) by financial or academic achievements alone. I will concentrate on measures of generative creativity that are process-based rather than outcome-based.
2. Review of Empirical Evidence
This cannot be a comprehensive research review. Rather, it is a sampling of some available evidence to provide preliminary support for the view that C-disorders increase generative creativity. The C-disorders are united by being approach-based rather than avoidance-based psychopathologies (like anxiety and depression), and meta-analyses have shown that approach disorders are associated with creativity (Baas et al, 2016). Compellingly, a DTI analysis found that there is “specific white matter architecture underlying the normal variance of divergent thinking, openness, and psychotic-spectrum traits,” which supports the idea of a continuum between creativity and psychopathology (Jung et al, 2010). The C-disorders share some specific cognitive styles and neural correlates connected to generative creativity.
2.1 — The C-Disorders and Ideational Speed
First, the C-disorders are associated with increased ideational fluency, racing thoughts, and some measures of cognitive speed. This could result in a higher pace of generation that outputs more ideas. A subjective acceleration of consciousness and an overproduction of ideas are involved in both adult ADHD and hypomania (Martz, 2021). ADHD symptoms like hyperactivity and impulsivity are associated with enhanced divergent thinking, originality, and cognitive flexibility, and improved performance on open-ended generation tasks (Boot et al, 2017). People with ADHD also generated more original ideas than controls when under competition, although they had trouble constraining ideas by practicality (Boot et al, 2020). Additionally, manic patients exhibited higher fluency scores, producing more novel word associations, and their associational fluency increased after discontinuing lithium (Johnson et al, 2012, pg. 8). A catalytic combination of ideational speed, fluency, and an excess of thought could allow people with C-disorders to brainstorm at an exceptional allegro-like tempo. The neural correlates of these processes are unclear, but possible candidates are dopaminergic hypersensitivity and potentially even a higher rate of synaptic transmission throughout the brain.
2.2 — The C-Disorders, Openness, and Divergent Thinking
Second, the C-disorders increase divergent thinking and openness, resulting in unexpected connections and more unpredictable mental pathways. While at its extreme this can lead to psychosis, it also amplifies the exploratory processes essential to generativity. Both bipolar and ADHD are associated with significantly higher openness to experience (Van Dijk et al, 2017; Quilty, 2009). Openness is linked to trait creativity, is even used as a measure of creativity, and is associated with higher volume in brain regions that inhibit control and reduce constraint (Li, 2015). The highly-open personalities of C-disorder patients seem to facilitate highly associative, fluent, and originative brainstorming.
Painting “Mania” by Florencio Yllana
Furthermore, mania risk is associated with divergent thinking (Johnson et al, 2012). The more adaptive symptoms of mania – reduced need for sleep, hyperactivity, excitement, motivation, and enhanced mental speed – are particularly related to generative creativity, while more damaging symptoms like hypersexuality, anger, and poor judgement were not helpful (Johnson et al, pg. 12). However, even seemingly negative symptoms of mania like impulsivity and distractibility can be essential to generative creativity, as they can enable expression with reduced constraint and cognitive control. Bipolar also correlates with many measures of demonstrated creativity: this review finds that mean occupational creativity and lifetime ratings of creative accomplishment are significantly higher in bipolar patients, and the disorder is over-represented in eminent creatives like famous writers and artists (Johnson, pg. 6). As a whole, the kinds of cognitive and neural divergence seen in the C-disorders are valuable for generative creativity.
2.3 – The C-Disorders and Weakened Constraints
Third, the C-disorders are linked to looser cognitive limitations, weakened top-down control, and more unconstrained thinking. Creative tasks benefit from a state of hypofrontality, in which reduced PFC activation enables more spontaneous, bottom-up thought patterns. Bipolar I patients exhibit disruptions in the frontoparietal control network which reduce top-down constraints (Ramey & Chrysikou, 2014). Mania involves hypofrontality, a “significant attenuation of task-related activation of right lateral orbitofrontal function” that results in disinhibition and distractibility (Altshuler et al, 2005). Further, individuals with ADHD have impaired executive inhibition, which reduces the person’s ability to suppress creative but unconventional ideas – and ADHD patients exhibit improved performance on tasks like the Unusual Uses Test (White & Shaw, 2006). All of the C-disorders involve similar neuro-cognitive disinhibitions.
2.4 – The C-Disorders and Creative-Expressive Motivation
Fourth, one important driver of creativity in the C-disorders may simply be motivation: a desire to express and be creative. My personal experience with bipolar has involved strange, unusual, and difficult-to-explain conscious experiences like free-wheeling hallucinations, the sense that my imagination is bleeding into reality, and profound states of inspired joy. This has instilled an intense motivation to try and communicate these experiences and convert the imaginative richness of mania into some real, sharable artifact. For instance, Tourette’s syndrome is also highly correlated with musical creativity, perhaps in part because artistic expression is an enjoyable and effective way to manage tics (Espert et al, 2017). Sacks describes how, for one friend, “the half-convulsive excitement of Tourette’s continually stimulates his perception and imagination, producing a ceaseless stream of extraordinary images” (1992). A rushing river of creative thought can evoke an inspired motivational state that drives people to actualize ideas. Indeed, a desire to act creatively is connected to dopaminergic modulation of a mesolimbic pathway altered in ADHD (Boot et al, 2017). Often, those with C-disorders pursue generative creativity as an autonomous interpretative response to their experiences.
2.5 – A Note on Tourette’s
Tourette’s syndrome (TS) is mentioned sparingly here because it is the least-studied of the three — the most comprehensive review to date called Tourette’s connection to creativity an ‘uncharted topic’ (Colautti et al, 2021). Although it is understudied, this review still shows that TS results in higher generative creativity and is associated with higher openness to experience and divergent thinking. The neural structures implicated in TS correspond to the systems involved in creativity, and “it has been postulated that the excess of dopamine characterizing TS can enhance creative thinking” (Coluatti et al). In short, it seems that Tourette’s facilitates rapid mental associations through hypersensitivity in postsynaptic dopamine receptors and reduced executive control via altered PFC circuitry.
“Spectrum Of Tourettes,” by Kevin Gavaghan
3. Conclusion
Viewed holistically, this evidence establishes the initial plausibility of the hypothesis that the C-disorders (TS, BD, and ADHD) involve similar mental and neural mechanisms that result in enhanced generative creativity. Specifically, the disorders are connected to an increased rate of ideational production, augmented divergent thinking, reduced constraints, and higher motivation toward creative expression. These cognitive styles and brain differences form a loosely grouped cluster of traits that are remarkably valuable for the generation-focused initial steps of the creative process, like brainstorming.
While the core aim of this paper is to construct a hypothesis and ground it in existing empirical evidence, the findings reviewed here also have important practical implications. First, the C-disorders are not entirely pathological and have demonstrable and impactful benefits. This provides support for a neurodiversity approach, where psychiatry seeks to support patients with managing their conditions, channeling their creativity, and adjusting to society, rather than trying to ‘cure’ the disorders. However, existing treatments for ADHD, BD, and TS result in a state of diminished creativity that many patients find unpleasant. For instance, lithium, one of the most common medications for bipolar, produces well-documented creativity deficits (Rothenberg, 2001). Antidopaminergic medications for Tourette’s syndrome have also been documented to reduce creativity (Thenganatt & Jankovic, 2016). Psychiatric treatment should not be exclusively oriented toward mitigating all symptoms. Instead, it should aim to enhance the positive and creative features of these disorders, while minimizing the negative symptoms in line with patient’s wishes.
Second, this research suggests that cooperative, neurodiverse communities are essential for a maximally fruitful creative process. This is fundamentally based on the fact that the systems underlying generative and evaluative creativity are rivalrous. Exceptional generative and evaluative creativity, or remarkable talent in both divergent and convergent thinking, are therefore very unlikely to appear simultaneously in a single brain. The best creative solutions will be social. Highly generative, unconstrained thinkers can help break the ice of social norms, shatter conceptual blocks, and produce a gamut of novel ideas, but they will need the help of more structured, analytic, evaluative thinkers to turn the ideas into something valuable. Attempting to confine the creative process to a single individual’s mind is outdated, misguided, and mythologically rather than scientifically rooted. Instead, creativity operates in an extended way through multiple minds and in connection with external tools. Combining generative and evaluative processes through interpersonal synergy mixes together sparks of novelty and value that can light an inferno of creativity.
Photo by Hari Nandakumar. An artistic representation of neurodiverse cooperation?
Works Cited
Adams, J. L. (2019). Conceptual blockbusting: A guide to better ideas. Hachette UK.
Altshuler, L. L., Bookheimer, S. Y., Townsend, J., Proenza, M. A., Eisenberger, N., Sabb, F., … & Cohen, M. S. (2005). Blunted activation in orbitofrontal cortex during mania: a functional magnetic resonance imaging study. Biological psychiatry, 58(10), 763-769.
Baas, M., Nijstad, B. A., Boot, N. C., & De Dreu, C. K. (2016). Mad genius revisited: Vulnerability to psychopathology, biobehavioral approach-avoidance, and creativity. Psychological bulletin, 142(6), 668. https://doi.org/10.1037/bul0000049
Boot, N., Baas, M., van Gaal, S., Cools, R., & De Dreu, C. K. (2017). Creative cognition and dopaminergic modulation of fronto-striatal networks: Integrative review and research agenda. Neuroscience & Biobehavioral Reviews, 78, 13-23. https://doi.org/10.1016/j.neubiorev.2017.04.007
Boot, N., Nevicka, B., & Baas, M. (2017). Subclinical symptoms of attention-deficit/hyperactivity disorder (ADHD) are associated with specific creative processes. Personality and Individual Differences, 114, 73-81. https://doi.org/10.1016/j.paid.2017.03.050
Boot, N., Nevicka, B., & Baas, M. (2020). Creativity in ADHD: goal-directed motivation and domain specificity. Journal of attention disorders, 24(13), 1857-1866. doi: 10.1177/1087054717727352.
Bown, O. (2012). Generative and adaptive creativity: A unified approach to creativity in nature, humans and machines. In Computers and creativity (pp. 361-381). Springer, Berlin, Heidelberg.
Ellamil, M., Dobson, C., Beeman, M., & Christoff, K. (2012). Evaluative and generative modes of thought during the creative process. Neuroimage, 59(2), 1783-1794. https://doi.org/10.1016/j.neuroimage.2011.08.008
Espert, R., Gadea, M., Alino, M., & Oltra-Cucarella, J. (2017). Neuropsychology of Tourette’s disorder: cognition, neuroimaging and creativity. Revista de neurologia, 64(s01), S65-S72.
Johnson et al (2012). Creativity and bipolar disorder: touched by fire or burning with questions?. Clinical psychology review, 32(1), 1-12. https://doi.org/10.1016/j.cpr.2011.10.001
Jung, R. E., Grazioplene, R., Caprihan, A., Chavez, R. S., & Haier, R. J. (2010). White matter integrity, creativity, and psychopathology: disentangling constructs with diffusion tensor imaging. PloS one, 5(3), e9818. https://doi.org/10.1371/journal.pone.0009818
Li, W., Li, X., Huang, L., Kong, X., Yang, W., Wei, D., … & Liu, J. (2015). Brain structure links trait creativity to openness to experience. Social cognitive and affective neuroscience, 10(2), 191-198. https://doi.org/10.1093/scan/nsu041
Martz, E., Bertschy, G., Kraemer, C., Weibel, S., & Weiner, L. (2021). Beyond motor hyperactivity: racing thoughts are an integral symptom of adult Attention Deficit Hyperactivity Disorder. Psychiatry Research, 113988. https://doi.org/10.1016/j.psychres.2021.113988
Partridge, D., & Rowe, J. (1994). Computers and creativity. Intellect Books.
Phillips, J., & Cushman, F. (2017). Morality constrains the default representation of what is possible. Proceedings of the National Academy of Sciences, 114(18), 4649-4654. https://doi.org/10.1073/pnas.1619717114
Phillips, J., & Knobe, J. (2018). The psychological representation of modality. Mind & Language, 33(1), 65-94. https://doi.org/10.1111/mila.12165
Phillips, J., Morris, A., & Cushman, F. (2019). How we know what not to think. Trends in cognitive sciences, 23(12), 1026-1040. https://doi.org/10.1016/j.tics.2019.09.007
Poincaré, H. (1908). L’invention mathématique, conférence faite à l’Institut général psychologique. Au siège de la Société.
Quilty, L. C., Sellbom, M., Tackett, J. L., & Bagby, R. M. (2009). Personality trait predictors of bipolar disorder symptoms. Psychiatry Research, 169(2), 159-163. https://doi.org/10.1016/j.psychres.2008.07.004
Ramey, C. H., & Chrysikou, E. G. (2014). “Not in their right mind”: the relation of psychopathology to the quantity and quality of creative thought. Frontiers in psychology, 5, 835. https://doi.org/10.3389/fpsyg.2014.00835
Rothenberg, A. (2001). Bipolar illness, creativity, and treatment. Psychiatric Quarterly, 72(2), 131-147.
Sacks, O. (1992). Tourette’s syndrome and creativity. BMJ: British Medical Journal, 305(6868), 1515.
Thenganatt, M. A., & Jankovic, J. (2016). Recent advances in understanding and managing Tourette syndrome. F1000Research, 5.
White, H. A., & Shah, P. (2006). Uninhibited imaginations: creativity in adults with attention-deficit/hyperactivity disorder. Personality and individual differences, 40(6), 1121-1131.
Van Dijk, F. E., Mostert, J., Glennon, J., Onnink, M., Dammers, J., Vasquez, A. A., … & Buitelaar, J. K. (2017). Five factor model personality traits relate to adult attention-deficit/hyperactivity disorder but not to their distinct neurocognitive profiles. Psychiatry research, 258, 255-261. doi: 10.1016/j.psychres.2017.08.037
Imagining plays a key role in thinking about possibilities. Modal terms like “could,” “should,” and “might” prompt us to imagine possible scenarios. I argue that imagination is the first step in modal cognition, as it generates the possibilities for consideration. The possibilities in the consideration set can then be partitioned into a more limited set of relevant possibilities, and ordered on some criteria, like value or probability.[1] Yet even imagination is not free, boundless, and unlimited. There are systematic constraints on imaginings. The three considerations that determine which possibilities are considered — physical possibility, probability or regularity, and morality — also influence which scenarios are imaginable or easier to imagine.
Ultimately, the evidence indicates that imagination uses a representation similar to the psychological representation of modality,[2] and operates under the constraints that apply to modal cognition in general. This paper has two key goals: (1) to strengthen the theory of a common underlying psychological representation of modality by applying it to imagination, and (2) to understand the imagination and its constraints better by illuminating the psychological representation it has in common with modal cognition.
1. Imagination as the Initial Generative Step of Modal Cognition
Modal cognition will be used as an umbrella term for any kind of thinking about possibility, including counterfactual thinking, causal selection, free will judgements, and more. Imagination is a sub-concept under modal cognition, as it is a form of “attention to possibilities.”[3] There are many types of imagination, but we can afford to gloss over most of the distinctions and instead use a broad definition. Imagination is to “represent without aiming at things as they actually, presently, and subjectively are.”[4] In other words, imagination is mental simulation. Since imagination is about non-occurrent possibilities – like fictional scenarios, images of the future, or counterfactuals – it is necessarily modal. But is modal cognition necessarily imaginative? In short: yes.
After all, we cannot represent possibilities based on a single proposition. Merely varying some proposition’s meaning or truth-value is a simple logical process that cannot characterize modal cognition in general, especially the rich kind of modal cognition involved in decision-making, causal judgements, and counterfactual reasoning. In modal cognition, we must conceive of a full scenario and then consider alternatives (possible worlds) for that scenario. This sounds a lot like imagination, which involves representing a situation: “a configuration of objects, properties, and relations.”[5] Considering the ways a captain could have prevented a ship from sinking, for instance, requires mentally simulating this scenario and varying its features to produce alternative possibilities.[6]Modal cognition relies on imagination to represent situations and generate their alternatives.
More precisely, imagination fits into modal cognition as the initial generative step: it produces the possibilities that are later considered and evaluated. This is inspired by the distinction between discriminative models and generative models in machine learning.[7] A discriminative model uses observed variables to identify unobserved target variables – for example, to find the probable causes of sensory inputs. These models often use a hierarchy of mappings between variables to represent an overall input-output mapping. In contrast, a generative model simulates the interactions among unobserved variables that might generate the observed variables. For example, graphics rendering programs can follow a set of processes to simulate a physical environment. Williams (2020) provides detailed evidence showing that both perception and imagination are best described as generative models.
While I will not repeat William’s arguments here, treating imagination as a generative model is valuable for a few additional reasons. First, imagination is governed by principles of generation: a set of (implicit or explicit) rules that guide our imaginings.[8] For example, in Harry Potter, “Latin words and wands create magic” is a principle of generation that readers can consistently use to simulate the imagined world. Rather than a graphics rendering program that deterministically yields a given outcome by following certain processes, the imagination generates a set of possibilities guided by the relevant principles of generation. However, imagination, like rendering, is a generative model that uses certain processes to produce (and explain) a set of phenomena.
Second, treating imagination as a generative model explains imaginative mirroring: unless prompted otherwise by principles of generation, our imagination defaults to follow the rules of the real world. If a cup ‘spills’ in an imaginary tea party, the participants will treat the spilled cup as empty, following the physics of reality.[9] In perception, we are always running a generative model of reality, using processes we derive from experience to simulate the physical world and predict its behavior.[10]Imagination involves running a generative model on top of this simulation of reality. Some processes are modified in the imagining, but the ones that are not modified are ‘filled in’ by our default generative model of reality. Further, we quarantine imaginative models from perceptual models, so that events in the imagining are not taken to have effects in the real world – imagined spills do not make the real table wet. Treating imagination as a generative model running separately but based upon a reality-based perceptual model is useful in explaining these effects.
Finally, the generative model view explains the systematic constraints on imagination and their function. Imaginings are not utterly free and boundless. Rather, imagination changes some aspects of the world, and then unfolds the impacts of these changes in a constrained way based on specific rules of generation. Later, I will show that imagination by default follows our world’s laws of physics and probability. We also resist imaginings that break normative limitations set by morality. If imagination is a generative model, then the constraints are the rules that determine how the generation process is carried out, analogous to rendering algorithms in animations or games. Imagination’s constraints allow it to serve a valuable and adaptive function in generating possibilities relevant to our real world.
In Kratzer semantics, a modal anchor is the element from which a set of possible worlds is projected.[11] In simpler terms, the anchor is the thing held constant in modal projection. For example, in the statement “people could jump off this roof,” the modal anchor is the situation of the roof. We project a domain of possible worlds that all include this roof and determine if people jump off the roof in at least one possible world. Imagination is the cognitive function that carries out modal projection, as it generates the possibilities prescribed by the modal anchor and its context. The modal anchor defines the processes of the generative model. Alternatively, modal anchors correspond to “props” in the philosophy of imagination, where a prop is the thing that prescribes what is to be imagined and the principles of generation to be used in imagining.[12] The modal anchor functions as a linguistic prop, prescribing an imagining that generates a set of possibilities relevant to the anchor.
This sets the stage for a comprehensive picture of modal cognition. First, some prop or modal anchor elicits thought about possibilities and triggers the start of the process. Second, imagination acts as a generative model, creating a set of possibilities based on the rules of generation prescribed by the modal anchor. This produces the consideration set, the group of possibilities under consideration. Third, the generated possibilities can then be narrowed down further and partitioned into a relevance set.[13] Finally, the possibilities are ordered according to some criteria, so the possibilities most relevant to the task at hand are ranked the most highly.
While it is conceptually helpful to separate these steps, I do not claim the steps occur in a sequential, discrete order. These components can happen synchronously and are often blurred together. Steps two and three are especially entangled, as I will show that generation through imagination also involves constraints that winnow down the considered possibilities. The rest of this paper will examine step two in detail. I will focus on how the imagination is constrained, and how its constraints indicate that it involves the psychological representation of modality.
2. The Psychological Representation of Modality and Imagination
2.1 Constraints in Modality
A growing body of research shows that a common psychological representation underlies many kinds of thinking about possibilities. Using certain constraints, this representation supports quick, effortless, computationally cheap, and often unconscious modal cognition. The constraints of physics, morality, and probability influence which possibilities are considered relevant.[14] For instance, in counterfactual reasoning, we mostly consider probable events, evaluatively good events, and physically normal events. Evidence also indicates that a common psychological capacity underlies our judgements of moral permissibility and physical possibility.[15] Evaluative concerns and prescriptive norms play an especially critical role in constraining possibilities.
Phillips, Luguri, and Knobe (2015) show that morality plays a key role in limiting the set of relevant possibilities for many types of judgement. For instance, people are less likely to agree that a captain on a sinking ship was forced to throw his wife overboard than that he was forced to throw cargo overboard. With the added support of several other studies, researchers demonstrated this effect occurs because immoral possibilities are considered less relevant. Critically for my thesis, the researchers also showed that prompting participants to generate more possibilities led to significant effects on their judgements.[16] When participants imagined decisions the captain could have made, they were more likely to judgements that he was free and not forced. This demonstrates the importance of the initial generative step.
Further, Phillips and Cushman (2017) found that both children and adults under time constraints tend to judge immoral events as impossible.[17]Non-reflective modal judgements are “ought-like,” and exclude immoral possibilities from consideration. Given time to deliberate, adults can differentiate types of modal judgment and make more reasoned judgements about possibility. In this study, participants were presented with events and were asked to judge which events were possible. For example, for the person stuck at an airport, participants are asked if he can hail a taxi, teleport, sell his car, or sneak onto public transit. Importantly, the generative step is performed by the researchers. The participants do not have to imaginatively generate the options. Instead, they are given the options and asked to evaluate their possibility. This skips step 2 of modal cognition, and instead focuses on step 3. However, in most natural situations, we have to generate the available options ourselves.
In general, research on modal cognition overlooks the mechanism that generates possibilities. Existing studies often ask participants to evaluate already-generated possibilities. This experimental design systematically misses the effects of the process that generates possibilities in the first place. One exception is Kushnir and Flanagan (2019), which tested whether a person’s ability to generate possibilities predicted their judgement that they have free will.[18] We tend to judge agents as free when we can represent alternative possibilities for their action. Thus, simply generating more possibilities may lead us to judge that agents are freer. Indeed, this experiment found that children’s fluency in generating ideas predicted their evaluation of their own free will. Performance on a task that involved generating ideas within an imagined world was the best predictor of a child’s judgements: the more fluent the children were in this imagination task, the more likely they were to judge themselves as free.
The researchers speculated that there may be a “direct pathway from idea generation to judgments of choice and possibility.”[19] In my view, the pathway is indirect, as existing research indicates that after possibility-generation we also evaluate the relevance of possibilities and rank them. However, the studies discussed above underscore the importance of the imagination as the initial generative step. The nature and quantity of the generated possibilities has demonstrable impacts on modal judgements. Furthermore, there may be important constraints on this generation process that lead to downstream effects on later processes in modal cognition.
2.2 Constraints in Imagination
The same constraints apply to both modality and imagination. This is surprising, as intuitively imagination seems far freer and more limitless than normal reasoning. We can easily imagine worlds where magic violates physical laws or where improbable events occur often. However, I argue that the default representation of imagination results in resistance to imagining possibilities that violate physical laws, irregular or unlikely possibilities, and immoral or evaluatively bad possibilities. Experimental results reveal that the imaginations of young children are limited by precisely these constraints. Adults are able to deliberately generate more and less constrained possibilities. However, just as adults can treat immoral possibilities as irrelevant, imaginative resistance shows that the adult imagination is inhibited against immoral possibilities. Conclusively, the imagination shows a startling resemblance to the psychological representation of modality.
Investigations of modal cognition often use developmental research to show constraints on children’s reasoning about possibilities, indicating a default representation of modality that is especially visible during early childhood.[20] Similarly, the imaginations of young children (ages 2-8) are surprisingly reality-constrained. Children tend to resist, or fail to generate, impossible and improbable imaginings. When prompted to imagine hypothetical machines, children judge that familiar machines could be real, but are reluctant to imagine possible machines that operate very differently from any object they have regular experience with.[21] Children also protest against pretense that contradicts their knowledge of regularity, expecting imaginary entities to have ordinary properties.[22] Even when pretending, kids expect lions to roar and pigs to oink, and they resist imagining otherwise.
Furthermore, 82% of the time, children extend fantasy stories with realistic events rather than fantastic events, while adults extend fantasy stories with fantastic events.[23] Young children imagine along ordinary lines even when primed with fantastical contexts, filling in typical and probable causes for fantastical imaginary events.[24]Children show a strong typicality bias in completing fictional stories, favoring additions to the story that match their regular experiences in reality.[25] For example, even if an imagined character can teleport or ride dragons, a young child will say the character gets to the store by walking and arrives at school on a bus. Children’s bias toward adding regular events persisted even after experimental manipulations designed to encourage children to notice a pattern of atypicality in the story.[26] This is surprising: popular wisdom dictates that children are exceptionally and fantastically imaginative. However, this research shows that children have simple, limited, and relatively mundane imaginations that are constrained by regularity, probability, and typical reality.
The imaginations of young children are not as free & creative as you might expect. (Image source: Kelly Sikkema)
Evaluative concerns are an additional constraint on the imagination. My theory predicts that children’s imaginations will show a bias toward generating evaluatively good possibilities, and a resistance to imagining possibilities that they see as evaluatively wrong. Some studies indicate that this is the case. For example, American children are more likely than Nepalese and Singaporean children to judge that they are free to act against cultural and moral norms.[27] This is likely because children in cultures with stronger or more restrictive evaluative norms find it harder to generate evaluatively wrong possibilities or see these possibilities as relevant. As free will judgements depend on representing alternative possibilities, these children see themselves as less free to pursue possibilities that violate evaluative norms. This means that morality is an additional constraint on the imagination, especially in early childhood. However, more research is needed to validate this hypothesis.
As children develop, the constraints on their imagination relax, leading to less restricted generation of possibilities. Older children are more likely to imagine improbable and physically impossible phenomena.[28] Explicitly prompting children to generate more possibilities leads them to imagine more like older children, producing possibilities less constrained by probability and regularity.[29] This shows that the initial generative step may underlie observed developmental changes in modal cognition. The imaginations of older children generate more total possibilities, including more irregular possibilities, and they are therefore more likely to judge irregular events as possible.
Viewing imagination as a generative model allows productive interpretations of this research. When imagining, young children apply a generative model with the same rules of generation used in perception to produce expectations about reality. This early imagination may use simple constraints and empirical heuristics to allow effortless and rapid generation of possibilities. For instance, if the child regularly encounters an event, they are more likely to imagine this event.[30] In later development and adulthood, the imagination generates possibilities in a more deliberative and analytical way. This suggests a dual process model of imagination.[31] Children may use a more uncontrolled, effortless, and unconscious imagination based on simple heuristics and experience-derived rules of generation. In contrast, adults use a more controlled, effortful and conscious imagination that generates possibilities based on relatively sophisticated and principled rules.
Although adults can more easily imagine irregular events or events that violate physical laws, the developed imagination is still constrained by moral norms. Imaginative resistance refers to a phenomenon where people find it difficult to engage in prompted imaginative activities. For example, if a fiction prompts us to imagine that axe-murdering is morally good, we resist this imagining. Unfortunately, there are few empirical tests of imaginative resistance. In one study conducted by Liao, Strohminger, and Sripada (2014), participants exhibited resistance to imagining morally deviant scenarios.[32] For example, participants reported difficulty in imagining that it was morally right for Hippolytos to trick Larisa in the Greek myth “The Rape of Persephone,” even though Zeus declared the trickery was morally right.[33] Their imaginative difficulty was significantly correlated with their evaluation that this trickery was morally wrong. This effect was replicated in a second experiment with a different story. The experiments also showed that imaginative resistance was modulated by context and genre. Participants more familiar with Greek myth were less likely to resist imagining that Hippolytos’ trickery was right, and participants were more willing to imagine that child sacrifice is permissible in an Aztec myth than in a police procedural. Context-specific variation in imaginative resistance may explain some of the variation in modal judgements.
Further research has demonstrated the empirical reality of imaginative resistance. In one study, adults were asked to imagine morally deviant worlds, where immoral actions are morally right within the imagined world.[34] Most participants found morally deviant worlds more difficult to imagine than worlds where unlikely events occurred often, but easier to imagine than worlds with conceptual contradictions. Participants classified these morally deviant worlds as improbable, not impossible, although a subset reported an absolute inability to imagine a morally deviant world. Another study employed a unique design to avoid the effects of authorial authority and variation in prompts, asking participants to create morally deviant worlds themselves and describe these imagined worlds in their own words.[35]Participants still exhibited resistance to imagining moral deviant worlds, even when they were the authors of the world. Disgust sensitivity was correlated with imaginative resistance, while need for cognition and creativity were correlated with ease of imagining. Finally, Black and Barnes (2017) constructed an imaginative resistance scale to support future research on this phenomenon and its correlations with individual differences.
Taken as a whole, the research discussed above provides strong support for the view that imagination and thinking about possibilities involve the same psychological representation. This default representation is most visible in early childhood, but it still operates in adulthood, especially under time-constraints or in scenarios involving immoral possibilities. Imaginative resistance shows that the primacy of morality in limiting the imagination corresponds to the primacy of morality in limiting which possibilities are considered relevant. Overall, this shows that generation of possibilities through imagination and evaluations of possibility relevance both involve a common psychological representation that is present at all stages of modal cognition.
2.3 Neuroscience of Imagination & Modal Cognition
This paper primarily aims to describe imagination and modal cognition on Marr’s computational and algorithmic levels of analysis, without delving into the neural implementation. However, any complete model of modal cognition will describe the neural implementational details. Furthermore, an implication of my view is that interactions between imagination and modal cognition will be visible on a neural level. One falsification of my view could show that these two processes do not interact or involve very distinct neural pathways. As such, the limited review of the neuroscientific evidence below is meant only to establish the plausibility of two key claims: (1) modal cognition involves imagination, and (2) imagination and modal cognition use similar neural mechanisms.
Neuroscientific evidence shows that modal cognition and imagination involve the same neural correlates. There is a growing consensus that remembering the past, imagining the future, and counterfactual thinking all involve similar neural mechanisms in the default mode network (DMN).[36] Several studies show that the DMN is involved in simulating possible experiences, imagining, and counterfactual thinking.[37] At the outset, this indicates that modal cognition and imagination use the same parts of the brain. But more specifically, future-oriented and counterfactual thinking engages the posterior DMN (pDMN), centered around the posterior cingulate cortex.[38] Researchers showed this by asking participants in an fMRI scan to make choices about their present situation, and then prospective choices about their future. Their findings demonstrated that people often engage vivid mental imagery in future-oriented thinking, and that this process activates the pDMN while reducing its connectivity with the anterior DMN. This provides a candidate neural process that underlies imaginative generation of possibilities.
Furthermore, a key cognitive ability that underlies imagination is prefrontal synthesis (PFS), the ability to create novel mental images. This process is performed in the lateral prefrontal cortex (LPFC), which likely acts as an executive controller that synchronizes a network of neuronal ensembles that represent familiar objects, synthesizing these objects into a new imaginary experience.[39] Children acquire PFS around 3 to 4 years of age, along with other imaginative abilities like mental rotation, storytelling, and advanced pretend play.[40] Similarly, young children tend to lack a distinction between immoral, impossible, and irregular counterfactuals – they often conflate “could” and “should.”[41] While further study is needed, it is plausible that development of PFS is associated with mature modal cognition, making modal distinctions, and generating more sophisticated imaginings.
3. Conclusion
This essay constructs a broad theory of modal cognition in which imagination plays a critical role. Namely, imagination serves as an initial step that generates the possibilities for consideration for later steps. Imagination is best described algorithmically as a generative model which operates based on rules of generation prescribed by a modal anchor. Furthermore, the evidence discussed in section 2 indicates that imagination and thinking about possibilities both use a default psychological representation with the same fundamental constraints. While this psychological representation is not always visible in adulthood, it is clear in early childhood, and it still has observable effects in adult cognition. The psychological representation of modality and imagination enables us to think about possibilities in rapid, effortless, and useful ways.
This theory also yields testable predictions that could be explored by future empirical research. For example, it predicts that young children will exhibit more imaginative resistance to violations of morality than adults. They will be more likely to classify morally deviant worlds as impossible or show a total inability to imagine these worlds.[42]Under time pressure, adults will exhibit more imaginative resistance, and they will be more likely to imagine valuable scenarios than dis-valuable scenarios – just as people are more likely to generate valuable possibilities under time pressure.[43] Correspondingly, people given more time and opportunity to engage the imagination might exhibit more willingness to imagine morally deviant worlds. With very limited time or significant cognitive pressure, adult imaginations may resemble the imaginations of young children. Finally, individual differences in openness to experience, creativity, and imaginative ability may predict some of the variation in possibility judgements, through differences in the generation of possibilities. For instance, people who naturally generate more possibilities will be more likely to judge agents as free rather than forced.
Existing research has not explicitly drawn this connection between the imagination and the psychological representation of modality. Even if this proposed model is not correct as a whole, I hope this paper can help integrate disconnected research projects on modal cognition and imagination in cognitive science, neuroscience, and philosophy.
Bibliography
Addis, Donna Rose, Alana T. Wong, and Daniel L. Schacter. “Remembering the past and imagining the future: common and distinct neural substrates during event construction and elaboration.” Neuropsychologia 45, no. 7 (2007): 1363-1377.
Barnes, Jennifer, and Jessica Black. “Impossible or improbable: The difficulty of imagining morally deviant worlds.” Imagination, Cognition and Personality 36, no. 1 (2016): 27-40.
Black, Jessica E., and Jennifer L. Barnes. “Measuring the unimaginable: Imaginative resistance to fiction and related constructs.” Personality and Individual Differences 111 (2017): 71-79.
Black, Jessica E., and Jennifer L. Barnes. “Morality and the imagination: Real-world moral beliefs interfere with imagining fictional content.” Philosophical Psychology 33, no. 7 (2020): 1018-1044.
Berto, Francesco. “Taming the runabout imagination ticket.” Synthese (2018): 1-15.
Bowman-Smith, Celina K., Andrew Shtulman, and Ori Friedman. “Distant lands make for distant possibilities: Children view improbable events as more possible in far-away locations.” Developmental psychology 55, no. 4 (2019): 722.
Cook, Claire, and David M. Sobel. “Children’s beliefs about the fantasy/reality status of hypothesized machines.” Developmental Science 14, no. 1 (2011): 1-8.
Cushman, Fiery. “Action, outcome, and value: A dual-system framework for morality.” Personality and social psychology review 17, no. 3 (2013): 273-292.
Gaesser, Brendan. “Constructing memory, imagination, and empathy: a cognitive neuroscience perspective.” Frontiers in psychology 3 (2013): 576.
Goulding, Brandon W., and Ori Friedman. “Children’s beliefs about possibility differ across dreams, stories, and reality.” Child development (2020).
Kind, Amy. “Imagining under constraints.” Knowledge through imagination (2016): 145-59.
Kratzer, Angelika, “Modality for the 21st century,” In 19th International Congress of Linguists, pp. 181-201. 2013.
Lane, Jonathan D., Samuel Ronfard, Stéphane P. Francioli, and Paul L. Harris. “Children’s imagination and belief: Prone to flights of fancy or grounded in reality?” Cognition 152 (2016): 127-140.
Leslie, Alan M. “Pretending and believing: Issues in the theory of ToMM.” Cognition 50, no. 1-3 (1994): 211-238.
Liao, Shen-yi and Tamar Gendler. “Imagination.” The Stanford Encyclopedia of Philosophy (Summer 2020 Edition). Edward N. Zalta (ed.). <https://plato.stanford.edu/archives/sum2020/entries/imagination/>.
Liao, Shen-yi, Nina Strohminger, and Chandra Sekhar Sripada. “Empirically investigating imaginative resistance.” British Journal of Aesthetics 54, no. 3 (2014): 339-355.
Moulton, Samuel T., and Stephen M. Kosslyn. “Imagining predictions: mental imagery as mental emulation.” Philosophical Transactions of the Royal Society B: Biological Sciences 364, no. 1521 (2009): 1273-1280.
Parikh, Natasha, Luka Ruzic, Gregory W. Stewart, R. Nathan Spreng, and Felipe De Brigard. “What if? Neural activity underlying semantic and episodic counterfactual thinking.” NeuroImage 178 (2018): 332-345
Pearson, Joel. “The human imagination: the cognitive neuroscience of visual mental imagery.” Nature Reviews Neuroscience 20, no. 10 (2019): 624-634.
Phillips, Jonathan, Adam Morris, and Fiery Cushman. “How we know what not to think.” Trends in cognitive sciences 23, no. 12 (2019): 1026-1040.
Phillips, Jonathan, and Fiery Cushman. “Morality constrains the default representation of what is possible.” Proceedings of the National Academy of Sciences 114, no. 18 (2017): 4649-4654.
Phillips, Jonathan, and Joshua Knobe. “The psychological representation of modality.” Mind & Language 33, no. 1 (2018): 65-94.
Phillips, Jonathan, Jamie B. Luguri, and Joshua Knobe. “Unifying morality’s influence on non-moral judgments: The relevance of alternative possibilities.” Cognition 145 (2015): 30-42.
Schubert, Torben, Renée Eloo, Jana Scharfen, and Nexhmedin Morina. “How imagining personal future scenarios influences affect: Systematic review and meta-analysis.” Clinical Psychology Review 75 (2020): 101811.
Shtulman, Andrew, and Jonathan Phillips. “Differentiating “could” from “should”: Developmental changes in modal cognition.” Journal of Experimental Child Psychology 165 (2018): 161-182.
Shtulman, Andrew, and Lester Tong. “Cognitive parallels between moral judgment and modal judgment.” Psychonomic bulletin & review 20, no. 6 (2013): 1327-1335.
Spreng, R. Nathan, Raymond A. Mar, and Alice SN Kim. “The common neural basis of autobiographical memory, prospection, navigation, theory of mind, and the default mode: a quantitative meta-analysis.” Journal of cognitive neuroscience 21, no. 3 (2009): 489-510.
Stuart, Michael T. “Towards a dual process epistemology of imagination.” Synthese (2019): 1-22.
Thorburn, Rachel, Celina K. Bowman-Smith, and Ori Friedman. “Likely stories: Young children favor typical over atypical story events.” Cognitive Development 56 (2020): 100950.
Van de Vondervoort, Julia W., and Ori Friedman. “Preschoolers can infer general rules governing fantastical events in fiction.” Developmental psychology 50, no. 5 (2014): 1594.
Van de Vondervoort, Julia W., and Ori Friedman. “Young children protest and correct pretense that contradicts their general knowledge.” Cognitive Development 43 (2017): 182-189.
Vyshedskiy, Andrey. “Neuroscience of imagination and implications for human evolution.” (2019). Preprint DOI: 10.31234/osf.io/skxwc.
Weisberg, Deena Skolnick, and David M. Sobel. “Young children discriminate improbable from impossible events in fiction.” Cognitive Development 27, no. 1 (2012): 90-98.
Weisberg, Deena Skolnick, David M. Sobel, Joshua Goodstein, and Paul Bloom. “Young children are reality-prone when thinking about stories.” Journal of Cognition and Culture 13, no. 3-4 (2013): 383-407.
Williams, Daniel. “Imaginative Constraints and Generative Models.” Australasian Journal of Philosophy (2020): 1-15.
Williamson, Timothy. “Knowing by imagining.” Knowledge through imagination (2016): 113-23.
Winlove, Crawford IP, Fraser Milton, Jake Ranson, Jon Fulford, Matthew MacKisack, Fiona Macpherson, and Adam Zeman. “The neural correlates of visual imagery: A co-ordinate-based meta-analysis.” Cortex 105 (2018): 4-25.
Xu, Xiaoxiao, Hong Yuan, and Xu Lei. “Activation and connectivity within the default mode network contribute independently to future-oriented thought.” Scientific reports 6 (2016): 21001.
Phillips, Jonathan, Adam Morris, and Fiery Cushman, “How we know what not to think,” Trends in cognitive sciences 23, no. 12 (2019): 1026-1040. ↑
Phillips, Jonathan, and Joshua Knobe, “The psychological representation of modality,” Mind & Language 33, no. 1 (2018): 65-94. ↑
Williamson, Timothy, “Knowing by imagining,” Knowledge through imagination (2016): 113-23. Pg. 4. ↑
Liao, Shen-yi and Tamar Gendler, “Imagination,” The Stanford Encyclopedia of Philosophy. ↑
Phillips, Luguri, and Knobe. “Unifying morality’s influence on non-moral judgments: The relevance of alternative possibilities,” Cognition 145 (2015): 30-42. ↑
The difference between discriminative and generative models is (roughly) similar to the distinction between model-free and model-based reinforcement learning – see Cushman (2017). ↑
Walton, Kendall L, Mimesis as make-believe: On the foundations of the representational arts, Harvard University Press, 1990. Pg. 53. ↑
Leslie, Alan M, “Pretending and believing: Issues in the theory of ToMM,” Cognition 50, no. 1-3 (1994): 211-238. ↑
Williams, “Imaginative Constraints and Generative Models,” 2020. ↑
Kratzer, Angelika, “Modality for the 21st century,” In 19th International Congress of Linguists, pp. 181-201. 2013. ↑
Flanagan, Teresa, and Tamar Kushnir, “Individual differences in fluency with idea generation predict children’s beliefs in their own free will,” Cognitive Science, pp. 1738-1744. 2019. ↑
For instance, see Shtulman, Andrew, and Jonathan Phillips, “Differentiating “could” from “should”: Developmental changes in modal cognition,” Journal of Experimental Child Psychology 165 (2018): 161-182. ↑
Cook and Sobel, “Children’s beliefs about the fantasy/reality status of hypothesized machines,” Developmental Science 14, no. 1 (2011): 1-8. ↑
Van de Vondervoort, Julia W., and Ori Friedman,” Young children protest and correct pretense that contradicts their general knowledge,” Cognitive Development 43 (2017): 182-189. ↑
Weisberg et al, “Young children are reality-prone when thinking about stories,” Journal of Cognition and Culture 13, no. 3-4 (2013): 383-407. Pg. 386. ↑
Lane et al, “Children’s imagination and belief: Prone to flights of fancy or grounded in reality?,” Cognition 152 (2016): 127-140. Pg. 131. ↑
Thorburn, Bowman-Smith, and Friedman, “Likely stories: Young children favor typical over atypical story events,” Cognitive Development 56 (2020): 100950. ↑
See Lane et al, pg. 8; Goulding and Friedman, “Children’s beliefs about possibility differ across dreams, stories, and reality,” Child development (2020); and Bowman-Smith et al, “Distant lands make for distant possibilities: Children view improbable events as more possible in far-away locations,” Developmental psychology 55, no. 4 (2019): 722. ↑
Barnes and Black (2016), “Impossible or improbable: The difficulty of imagining morally deviant worlds,” pg. 8. ↑
Black, Jessica E., and Jennifer L. Barnes, “Morality and the imagination: Real-world moral beliefs interfere with imagining fictional content,” Philosophical Psychology 33, no. 7 (2020): 1018-1044. ↑
Mullally, Sinéad L., and Eleanor A. Maguire, “Memory, imagination, and predicting the future: a common brain mechanism?” The Neuroscientist 20, no. 3 (2014): 220-234. ↑
Pearson (2019); Gaesser (2013); Addis et al (2007); Spreng et al (2009); and Winlove et al (2018). ↑
Xu, Xiaoxiao, Hong Yuan, and Xu Lei, “Activation and connectivity within the default mode network contribute independently to future-oriented thought,” Scientific reports 6 (2016): 21001. ↑
Vyshedskiy, Andrey. “Neuroscience of imagination and implications for human evolution.” (2019). Preprint DOI: 10.31234/osf.io/skxwc. ↑
Shtulman, Andrew, and Jonathan Phillips. “Differentiating “could” from “should”: Developmental changes in modal cognition.” Journal of Experimental Child Psychology 165 (2018): 161-182. ↑
Phillips, Jonathan, and Fiery Cushman, “Morality constrains the default representation of what is possible,” Proceedings of the National Academy of Sciences 114, no. 18 (2017): 4649-4654. ↑
