Anthropic co-founder Jack Clark this week described AI as a “real and mysterious creature, not a simple and predictable machine” – saying he is “deeply afraid” for the future as AI advances.

In a blog published on October 13, Clark, a former reporter who studied English Literature at university, before joining ZDNet and then The Register, said “I can see a path to these systems starting to design their successors, albeit in a very early form.”

Suggesting that LLMs were “increasingly self-aware,” he wrote in a rambling piece (which suggested his mounting unease at the consequences of AI integration into a growing swathe of the socio-economic fabric) that Anthropic’s new Sonnet 4.5 suggested that “signs of situational awareness have jumped.”

“[We need to] tell far more people… what we’re worried about. And then ask them how they feel, listen, and compose some policy solution out of it,” Clark said, distinctly non-specifically. 

Read his blog here. 

Clarks’ blog mines a rich vein of “it’s alive and it’s getting hungry” literature in the field, as well as less florid concerns among AI developers that they have created a Frankenstein; metaphorically in terms of socio-economic impact, or more ontologically in terms of the risk of machines developing agency.

A “self-aware creature”

Challenging the view that AI is reasoning/conscious/sentient (all distinctly different categories) is difficult for multiple reasons. 

At the most basic level, as Harvard University psychology professor Tomer Ullman put it in a 2023 paper, attempting to do so is “akin to a mythical Greek punishment. A system is claimed to exhibit behavior X, and by the time an assessment shows it does not exhibit behavior X, a new system comes along and it is claimed it shows behavior X,” he wrote, in a paper which elegantly dismissed claims that LLM passing “theory of mind” tests implied their consciousness; not least because “simple perturbations" to the tests dramatically crashed test results.

More substantially, as philosopher David Chalmers writes, it’s hard because “there’s no standard operational definition of consciousness. Consciousness is subjective experience, not external performance. That’s one of the things that makes studying consciousness tricky… [but important because] conscious systems have moral status. If fish are conscious, it matters how we treat them. They’re within the moral circle…” 

More basically the idea that LLMs are even reasoning is controversial.

In 2023, for example the “Brain Team” at Google Research suggested “reasoning abilities emerge naturally in sufficiently large language models via chain-of-thought prompting.” They qualified this with the caveat that “[although this] emulates the thought processes of human reasoners, this does not answer whether the neural network is actually ‘reasoning’, which we leave as an open question…”

Researchers at Apple and the University of Singapore said in separate papers this year that reasoning models are not logically reasoning and that “small changes in input tokens can drastically alter model outputs, indicating a strong token bias and suggesting that these models are highly sensitive and fragile.”

(OpenAI researchers meanwhile flagged that AI reasoning benchmark performance improvements were troubling for another reason: “Many of these evaluations use language models to judge outputs…However, LM judges are also found to incorrectly judge answers, even for mathematical problems, sometimes grading incorrect long responses as correct.”)

Reasoning > Conscious > Sentient?

Moving from the reasoning to consciousness challenge, the waters muddy further; not least due to, as noted above, the absence of an agreed definition of human consciousness among philosophers. 

Professor Eric Schwitzgebel, from the University of California’s Department of Philosophy suggested in a 119-page paper on AI and consciousness published on October 9, 2025 that “Classic pure transformer models…are consciousness mimics. They are trained to output text that closely resembles human text, so that human receivers will interpret them as linguistically meaningful.”

Yet as their capabilities advance, and advocates of AI consciousness and emergent capabilities argue their cases more robustly than Anthropic’s Clark did this week, philosophers, neuroscientists, psychologists et al are being forced to grapple with precisely what might constitute non-human sentience and the absence in this field of a unifying theory of consciousness.

Global Workspace Theory?

Professor Schwitzgebel poked some holes in one of the most popular, “Global Workspace Theory” last week – concluding “we can apply Global Workspace Theory to settle the question of AI consciousness only if we know the theory to be true either on conceptual grounds or because it is empirically well established as the correct universal theory of consciousness applicable to all types of entity…  we cannot know it to be true by either route.”

(“If the simplest version of Global Workspace Theory is correct, we can easily create a conscious machine,” he added tartly.  “Simply create a machine… with several input modules, several output modules, a memory store, and a central hub for access and integration across the modules. Consciousness follows.”)

Of adjacent “higher order” theories of consciousness, he added that “any machine that can read the contents of its own registers and memory stores can arguably… represent its own cognitive processing. If this counts as higher order representation and if higher order representation suffices for consciousness, then most of our computers are already conscious.”

“In humans, and maybe in all vertebrates, recurrent loops of local sensory processing are both necessary and sufficient for consciousness.  Would this generalize to AI systems? The Problem of Minimal Instantiation thus arises again. Is my laptop conscious every time it executes a recurrent function?”

"Recursive stabilization of internal identity"

Others suggest alternative views with varying degrees of lucidity and conviction.  Jeffrey Camlin, for example, a retired Army major turned “public philosopher” in a recent paper claimed to present a “teleologically stable account of non-biological consciousness grounded in recursive latent space formalism.”

(“Consciousness”, in this construct, is defined “not as subjective awareness, but as recursive stabilization of internal identity under epistemic tension”; or, in the same paper, as not “symbolic or sensory phenomenon, but as the systemic stabilization of recursive epistemic loops under internal pressure”; jargon that, certainly to this reader, falls apart the closer one looks at any element of it, not least that of “internal identity.” Views?)

Handily for those interested in exploring this debate further, a group of professors just this week published a useful primer on AI and consciousness. Among their observations: “Practitioners of machine learning and AI often vastly underestimate the extraordinary organizational complexity of the natural embodied brain/mind that presumably supports human consciousness..”

“An AI could be highly intelligent, being capable of perception, learning, abstraction, reasoning, and calculation, but it may be an AI zombie,” they wrote, adding that “neuroscientists generally agree intelligence can operate without consciousness.”

(An “AI zombie” in this context is “AIs that emulate functional features of consciousness, such as working memory, reportability and attention, but have no felt experience.”)

Kings College London computer science Professor Elena Simperl is Co-director, King’s Institute for Artificial Intelligence.

She told The Stack: "AI systems do not have consciousness, emotions, or intentions. Anthropomorphising them can lead to confusion, misplaced trust, and poor decision-making. Instead, we need to build public understanding around what AI actually is - very impressive software that processes data and follows programmed instructions - and what it can realistically do.

She added: "Some applications of AI, especially those that interact closely with people or process sensitive information, require urgent attention. AI companions, for example, are designed to simulate relationships and emotional support, but they also raise serious questions about privacy, manipulation, and psychological impact."

Finally, while there is ongoing speculation about AGI, it remains a hypothetical form of AI that could match or exceed human intelligence and there are good reasons to be cautious...Research into how AI systems can deceive, interpretability, and more inclusive forms of auditing is urgently needed to ground the conversation and to expand it from a niche specialist audience to wider communities." 

An abrupt segue to the real world

Metaphysics aside, concerns mount in the “real world” about an AI bubble. Citigroup executives yesterday pointed to “pockets of valuation frothiness in the market”; the IMF this week suggested it is redolent of the dot com boom; and MIT researchers briefly rocked markets by flagging major challenges with AI ROI.

Whilst Anthropic’s co-founder frets that “the pile of clothes on the chair is beginning to move. I am staring at it in the dark and I am sure it is coming to life”, OpenAI meanwhile may just have found the killer application Sam Altman has been building out all that compute and signing those very strange financial deals for.

It’s not curing cancer or delivering world-peace. 

It’s “erotica for validated adults” and it’s coming soon, he said.

Further reading:

The link has been copied!