COMMENT

https://doi.org/10.1057/s41599-024-03553-w OPEN

A clarification of the conditions under which Large

language Models could be conscious
Morten Overgaard1 ✉ & Asger Kirkeby-Hinrup1,2

With incredible speed Large Language Models (LLMs) are reshaping many
aspects of society. This has been met with unease by the public, and public
1234567890():,;

discourse is rife with questions about whether LLMs are or might be conscious.
Because there is widespread disagreement about consciousness among scien-
tists, any concrete answers that could be offered the public would be con-
tentious. This paper offers the next best thing: charting the possibility of
consciousness in LLMs. So, while it is too early to judge concerning the possi-
bility of LLM consciousness, our charting of the possibility space for this may
serve as a temporary guide for theorizing about it.

1 Department of Clinical Medicine - Center of Functionally Integrative Neuroscience, Aarhus University, Aarhus, Denmark. 2 Department of Philosophy –

Theoretical Philosophy, Lund University, Lund, Sweden. ✉email: morten.storm.overgaard@cfin.au.dk

HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | (2024)11:1031 | https://doi.org/10.1057/s41599-024-03553-w 1

COMMENT HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | https://doi.org/10.1057/s41599-024-03553-w

L
arge Language Models (LLMs) are sophisticated artificial It is a strong intuition both in science and common sense that
neural networks whose weights are trained on hundreds of being conscious of something makes a cognitive difference for the
billions of words from the internet, including language subject. Yet, predominant models in cognitive neuroscience have
conversations between conscious humans with ‘real’ agency. not been able to conceptually — or empirically — identify a
Users that interact with LLMs are provided with a fascinating particular cognitive function (or set of functions) for which
language-based simulation of a natural language interaction. consciousness is necessary. This also goes for language and lin-
Because LLMs have been trained on conversations, in which guistic capabilities. So, at present, there is no objective way of
(actual) humans describe and express in different ways the determining whether any given function or action an LLM may
peculiar inner life we associate with conscious experience, the perform in fact is associated with consciousness, making this
LLMs are capable of giving descriptions and expressions of such approach unfeasible (see also Bayne et al. 2024).
an inner life that are practically indistinguishable from the that of The brief analysis above seems to show that the debate is stuck.
humans. To the public, this has made manifest the lack of clarity There is no empirical method available to determine if LLMs are
about what it means to have agency and to be conscious. In public conscious, and a theoretical conclusion on the matter will be
discourse on LLMs an uncertainty about whether they could be based on a choice or an assumption, thus either depending on
conscious drives many of the worries expressed by politicians, the arbitrary assumptions or ending as a circular argument. The
public audience, and laypeople alike. This uncertainty thrives in problem is familiar to consciousness researchers but is echoed in
part because we — as a scientific field — have yet to understand previous debates about consciousness in e.g. insects, animals,
consciousness as well. infants, non-communicating patients in coma or vegetative state,
In interdisciplinary consciousness studies, researchers are and even in neurotypical human adults, as exemplified in the
today far from consensus about how to explain consciousness “other minds problem” from the philosophy of mind.
theoretically. In fact, there is an extended and ongoing debate in If there is a way forward to directly measuring consciousness,
the field about what the words we use to describe and theorize we must identify the questions that need answering before we can
about consciousness even mean. Therefore, we have no strong find it. For instance, it must be determined whether the core
theoretical guidance to understand whether LLMs are — or can correlate of consciousness is biological/structural in nature or
be — conscious either (Aru et al. 2023; Chalmers, 2023b). Several rather functional/computational. Naturally, there will always be
recent scientific articles have assumed that LLMs are not con- biological and functional correlates at the same time, yet any
scious (Chalmers, 2023a; Colombatto and Fleming, 2023; Dodig- theory must argue that consciousness exists because of something
Crnkovic, 2023) and that we therefore can conclude that the that is either biological or functional in nature, so that other
ability to converse can happen unconsciously. At the same time, correlates are spurious or secondary. In recent years, much
others, as mentioned above, have suggested the exact opposite. attention has been given to classifying explanatory targets and
However, any such assumption is a theoretical choice not sup- mechanisms of extant theories in various, but similar terms
ported by any empirical evidence. (Doerig et al. 2020; Fahrenfort and van Gaal, 2021; Sattin et al.
Recently, it has been suggested in media as well as in the 2021; Schurger and Graziano, 2022; Signorelli et al. 2021).
scientific literature that there is evidence to suggest that con- Two of the important questions we need to raise are expressed
sciousness is common – not just in the biological domain but in in the matrix below: Consciousness is either realized by certain
any domain where information is integrated (Tononi et al. 2016). biological structures or by certain functions/computations, and
It is however very premature to make such a claim based on consciousness is either realized by simple/low-level phenomena
empirical science. This goes not only for the integrated infor- or by complex/higher-level phenomena.

mation theory, but for any contemporary theory of consciousness. Segmenting the landscape according to the functional-
How to measure consciousness remains one of the most promi- biological and simple-complex distinctions gives us a handle on
nent unsolved problems around (Bayne et al. 2024). Since con- the conditions under which LLMs may be conscious (now or in
sciousness seems to be a central component of human life, we the future). Because this way of segmenting the theoretical
have a vested interest in finding objective and reliable biomarkers landscape has narrowed the possible positions into a two-by-two
of consciousness in humans (not the least for clinical reasons). matrix, where each space predicts the prospects of artificial
Regarding the topic at hand, clearly, if we only understood how consciousness, it allows for better (but still underdetermined)
consciousness comes about in humans, it would be much easier to generalization than when considering each theory in isolation.
determine what it would take for a machine to be conscious, and This approach has the advantage of offering a theory-neutral
whether this is even possible in the first place. But we currently do mapping of the possibility space for LLM consciousness. There-
not know how consciousness comes about in humans, therefore fore, we will next briefly consider each of the two dimensions in a
this is not a feasible approach. little more detail.

2 HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | (2024)11:1031 | https://doi.org/10.1057/s41599-024-03553-w

HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | https://doi.org/10.1057/s41599-024-03553-w COMMENT

The biological-functional distinction example of the former would be a theory positing the need for
Fundamentally, either consciousness is associated with a physical exascale quantum computation at ambient temperatures in
structure or consciousness is associated with function. These two combination with specific properties of cortical neurons and the
“types” of theories come in many versions, depending on the neuronal membranes (Stoll, 2022). An example of the latter
specifics of what consciousness is taken to be reducible to, would be the ability to integrate information (Tononi, 2005).
identical with, or different from, and if or how it is anchored in
some specific structure or function. Conclusions
Researchers who associate consciousness with biological struc- While the above presents the available options of a highly com-
tures often have one or more neural structure(s) in mind. From this plicated and diverse theoretical landscape in a simple matrix, we do
perspective, an organism is conscious under the condition that it not suggest that answering the question is simple. We have a dif-
has a specific neural structure (biological foundation). This thinking ficult and long journey ahead. It should be clear, however, that it is
is evident in several currently influential theories, e.g. in integrated premature to draw any conclusions about the possibility of LLM
information theory, where human consciousness is literally identical consciousness. Furthermore, while our ability to reach a final
to the most complex cluster of interconnected information in a conclusion may still be far in the future, it is possible that we may be
brain (Tononi et al. 2016)1. If consciousness depends on biological able to rule out some positions in the matrix before that (if
structures, LLMs will never be conscious because they are not incoming and future data is serendipitous). In this sense, the matrix
instantiated in the ‘right’ material (c.f. Searle 1980). Now, one might may serve as a best-we-have-so-far understanding and may be
object that if an LLM was instantiated in the right biological useful to parse incoming data. Nevertheless, until serious progress is
material (a biological computer of the future) then it would in fact made, and as long as theories rooted in all four positions in the
be conscious. This, however, is misguided with respect to the point matrix can explain all or most of the available scientific data (for
we are making here. Yes, in such a case it would certainly be discussion, see Butlin et al. 2023), it will be an unscientific enterprise
conscious, but it would not be conscious in virtue of that which to draw conclusions about consciousness in LLMs or any other
makes it an LLM, it would be conscious in virtue of the material in artificial system. In the words of Uriah Kriegel: “When two theories
which it was instantiated. are perfectly empirically equivalent, there is an important sense in
Researchers who associate consciousness with functional which choosing among them on the basis of superempirical virtues
properties typically conceive of consciousness as analogous to is a nonscientific endeavor.” (Kriegel, 2020, p. 273).
computer software that needs some hardware to run (c.f. what
Chalmers (2011) calls the “thesis of computational sufficiency”2). Received: 30 April 2024; Accepted: 1 August 2024;
From this perspective, any physical structure (e.g. brains or arrays
of silicon chips) with the necessary – currently unknown –
characteristics to run the ‘right’ software will be able to realize
consciousness. Accordingly, if consciousness depends on func-
tional characteristics, LLMs can be conscious if they run the
‘right’ software. Notes
1 Observe, this position may still be compatible with Panpsychism.
2 Observe, Chalmers himself is sceptical about LLM consciousness.

Complexity matters
The other parameter in our chart maps whether consciousness References
Aru J, Larkum ME, Shine JM (2023) The feasibility of artificial consciousness
depends on complex biological structures or functions, or whether through the lens of neuroscience. Trends Neurosci 46(12):1008–1017
it merely requires simple structures or functions. Bayne T, Seth AK, Massimini M, Shepherd J, Cleeremans A, Fleming SM, Malach
Some researchers propose that one or a few functions are able to R, Mattingley JB, Menon DK, Owen AM, Peters MAK, Razi A, Mudrik L
realize consciousness. One such example is higher-order thought (2024) Tests for consciousness in humans and beyond. Trends Cogn Sci
28(5):454–466. https://doi.org/10.1016/j.tics.2024.01.010
theory which argues that consciousness merely requires an (itself Brown R, Lau H, LeDoux JE (2019) Understanding the higher-order approach to
unconscious) thought about a first-order content (such as a visual consciousness. Trends Cogn Sci 23(9):754–768. https://doi.org/10.1016/j.tics.
stimulus) (Brown et al. 2019). Consequently, any system with the 2019.06.009
right kind of metacognitive abilities may be conscious. Nothing Butlin, P, Long, R, Elmoznino, E, Bengio, Y, Birch, J, Constant, A, Deane, G,
indicates that LLMs have actual metacognition. In principle, how- Fleming, SM, Frith, C, & Ji, X (2023). Consciousness in artificial intelligence:
insights from the science of consciousness. arXiv preprint arXiv:2308.08708
ever, nothing would prevent an artificial system from having Chalmers DJ (2011) A computational foundation for the study of cognition. J Cogn
higher-order thought-like states, so consciousness in artificial sys- Sci 12(4):325–359
tems is not ruled out from this perspective. Chalmers, DJ (2023a) Could a large language model be conscious? arXiv preprint
If consciousness is correctly associated with any kind of arXiv:2303.07103
function, LLMs may already be close to being conscious, and Chalmers DJ (2023b) Does thought require sensory grounding? From pure thin-
kers to large language models. Proc Address Am Philos Assoc 97:22–45
future developments are likely to lead to conscious artificial sys- Colombatto, C, & Fleming, S (2023) Folk psychological attributions of con-
tems. If consciousness is associated with extremely simple func- sciousness to large language models
tions, without knowing it, we may have created consciousness in Dodig-Crnkovic, G (2023) How GPT Realizes Leibniz’s Dream and Passes the
artificial systems long before LLMs appeared. Turing Test without Being Conscious. Computer Sciences & Mathematics
The complexity aspect plays out differently if consciousness is Forum
Doerig A, Schurger A, Herzog MH (2020) Hard criteria for empirical theories of
in fact associated with biological structures. On this view our consciousness. Cogn Neurosci 12(2):41–62
current LLMs will never be conscious because they are not Fahrenfort JJ, van Gaal S (2021) Criteria for empirical theories of consciousness
instantiated in the ‘right’ material. However, it is possible that the should focus on the explanatory power of mechanisms, not on functional
‘right’ material (e.g. brains) exists not only in humans, so con- equivalence. Cogn Neurosc 12(2):93–94. https://doi.org/10.1080/17588928.
sciousness may be very widespread in nature. On this view, the 2020.1838470
Kriegel U (2020) The Oxford Handbook of the Philosophy of Consciousness. Oxford
complexity aspect maps onto how widespread consciousness in University Press
fact is. If the structure must be very complex, fewer species will be Sattin D, Magnani FG, Bartesaghi L, Caputo M, Fittipaldo AV, Cacciatore M,
conscious, if the structure is very simple, consciousness will be Picozzi M, Leonardi M (2021) Theoretical models of consciousness: a scoping
abundant in biological beings (Wiese and Friston, 2021). An review. Brain Sci 11(5):535

HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | (2024)11:1031 | https://doi.org/10.1057/s41599-024-03553-w 3

COMMENT HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | https://doi.org/10.1057/s41599-024-03553-w

Schurger A, Graziano M (2022) Consciousness explained or described? Neurosci Informed consent

Conscious, 2022(1). https://doi.org/10.1093/nc/niac001 Informated consent was not required as the study did not involve human participants.
Searle JR (1980) Minds, brains, and programs. Behav Brain Sci 3(03):417–424
Signorelli CM, Szczotka J, Prentner R (2021) Explanatory profiles of models of
consciousness - towards a systematic classification. Neurosci Conscious,
Additional information
Correspondence and requests for materials should be addressed to Morten Overgaard.
2021(2). https://doi.org/10.1093/nc/niab021
Stoll E (2022) Modeling electron interference at the neuronal membrane yields a
Reprints and permission information is available at http://www.nature.com/reprints
holographic projection of representative information content. bioRxiv,
2022.2012.2003.518989. https://doi.org/10.1101/2022.12.03.518989
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in
Tononi, G (2005) Consciousness, information integration, and the brain. In L
published maps and institutional affiliations.
Steven (Ed.), Progress in Brain Research (150, pp. 109–126). Elsevier. https://
doi.org/10.1016/S0079-6123(05)50009-8
Tononi G, Boly M, Massimini M, Koch C (2016) Integrated information theory:
from consciousness to its physical substrate. Nat Rev Neurosci
17(7):450–461. https://doi.org/10.1038/nrn.2016.44 Open Access This article is licensed under a Creative Commons
Wiese W, Friston, KJ (2021) The neural correlates of consciousness under the free Attribution-NonCommercial-NoDerivatives 4.0 International License,
energy principle: From computational correlates to computational explana- which permits any non-commercial use, sharing, distribution and reproduction in any
tion. Philos Mind Sci, 2 medium or format, as long as you give appropriate credit to the original author(s) and
the source, provide a link to the Creative Commons licence, and indicate if you modified
the licensed material. You do not have permission under this licence to share adapted
material derived from this article or parts of it. The images or other third party material
Author contributions
in this article are included in the article’s Creative Commons licence, unless indicated
MO and AKH contributed equally to all aspects of the paper.
otherwise in a credit line to the material. If material is not included in the article’s
Creative Commons licence and your intended use is not permitted by statutory
regulation or exceeds the permitted use, you will need to obtain permission directly from
Competing interests the copyright holder. To view a copy of this licence, visit http://creativecommons.org/
The authors declare no competing interests.
licenses/by-nc-nd/4.0/.

Ethical approval © The Author(s) 2024

Ethical approval was not required as the study did not involve human participants.

4 HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | (2024)11:1031 | https://doi.org/10.1057/s41599-024-03553-w