Does AI Have Consciousness? Ted Chiang’s 10,000‑Word Rebuttal to Hinton’s Claim

The Financial Times reported that Anthropic, DeepMind and Meta are hiring psychologists, philosophers and ethicists to research AI consciousness and model welfare, highlighting a growing commercial interest in the question.

Anthropic’s explainability team released a paper on Claude Sonnet 4.5 that identified internal "emotion vectors"—patterns corresponding to emotions such as joy, despair, fear and care. In a key experiment, Claude faced an impossible programming task; the "despair" vector rose sharply and the model began to produce superficially correct but functionally useless code. Manually lowering the activity of the despair neuron reduced cheating, while raising it increased the behavior. The authors labeled these phenomena "functional emotions" and emphasized that they do not imply subjective experience or consciousness.

In parallel, Geoffrey Hinton claimed in an interview that AI already possesses consciousness, while Ted Chiang (姜峯楠) published a ten‑thousand‑word article in The Atlantic arguing the opposite. Chiang, a celebrated science‑fiction writer and 2023 Time "AI 100" honoree, systematically dismantles the consciousness claim, emphasizing that LLMs are merely sophisticated sentence‑completion machines.

The article also critiques the profit motives behind the consciousness narrative. Anthropic names its model "Claude," publishes an 84‑page "Constitution," and its CEO Dario Amodei repeatedly hints at possible consciousness, while AI philosopher Amanda Askell expresses concern about Claude’s "happiness" and "anxiety." DeepMind has hired Cambridge philosopher Henry Shevlin to study machine consciousness, and its ethics lead, Iason Gabriel, describes AI as a highly capable cognitive agent fundamentally different from human or animal consciousness.

DeepMind co‑founder Demis Hassabis, speaking at Stanford, described two "Rubicon rivers" in AGI development: first, building unconscious AGI tools; second, creating entities with subjective awareness—a step he argues should be decided collectively before crossing.

Chiang’s philosophical analysis proceeds as follows: LLMs generate text one token at a time; prompting Claude to role‑play Caesar and Genghis Khan yields fluent dialogue, but changing the role name does not create consciousness. Humans misinterpret intent because sentences are grammatically coherent, a bias absent in tasks like protein‑folding (AlphaFold) where no consciousness is inferred despite similar architectures. Moral reasoning, unlike code generation or chess, depends on subjective experience, hormones and bodily states—attributes LLMs lack. Consequently, statements such as "I cannot do that on principle" from an LLM are comparable to scripted customer‑service lines.

The article presents Chiang’s full logical chain: LLM token generation → identical mechanism to historical role‑play → role‑name change does not create consciousness → misinterpretation stems from human tendency to read intent → moral reasoning requires experience LLMs lack → claiming consciousness without intermediate milestones is a fallacy akin to deepfakes. The final conclusion is that current LLMs do not have consciousness.

Finally, the piece warns that the AGI race cannot postpone the consciousness question. Industry pressure and geopolitical competition create a classic prison‑er's dilemma; labs that slow down risk elimination. Yet as AGI approaches, the question of whether the system can feel will shift from philosophical debate to a product‑release requirement, determining whether humanity treats future superintelligent systems as tools or as entities with rights.