The debate over AI consciousness is intensifying within scientific circles and the public alike. As artificial intelligence models become more sophisticated and their behavior more human-like, society is approaching a critical juncture.
A perspective piece titled “Illusions of AI consciousness” explores the profound risks of this trajectory, urging caution before we embrace the idea of sentient machines. The core of the debate centers on a fundamental question: is consciousness an exclusively biological phenomenon, or is it a product of complex information processing that a machine could replicate?
This latter view, known as computational functionalism, suggests that consciousness arises from an algorithm’s manipulation of information, regardless of whether the system “is made up of neurons, silicon, or any other physical substrate”. The paper argues that this idea is currently considered plausible within the scientific community, which means the possibility of AI consciousness cannot be easily dismissed.
A Framework for Machine Consciousness
To move the discussion from pure philosophy to testable science, researchers are developing frameworks based on leading functionalist theories of consciousness. A recent study applied this methodology by creating a list of “indicators” for consciousness. These indicators are essentially “computational properties that are considered both individually necessary and jointly sufficient for a system to be conscious, if that theory is true”.
These properties are concrete enough that their presence can be evaluated in modern AI systems. While the paper notes that no current AI model likely meets all the criteria from any single leading theory, it also makes a crucial point: “there are no fundamental barriers to constructing a system that does”. In fact, AI research is naturally heading in this direction. Functions often associated with consciousness, such as reasoning, planning, and abstract thought, are also desirable for creating more powerful and intelligent systems. This means that in the quest for better AI, we may inadvertently build systems that satisfy the indicators for consciousness.
Can Science Explain Subjective Experience?
Of course, many will remain skeptical, pointing to what philosopher David Chalmers famously termed the “hard problem” of consciousness: explaining subjective experience from computational principles alone. We have an intuitive sense that our experiences are rich and meaningful yet fundamentally “ineffable,” or impossible to fully describe. How can a machine truly experience the color red?
The paper suggests that science is beginning to offer explanations for these seemingly mysterious qualities. For instance, one theory proposes that the richness and ineffability of our experiences are a consequence of brain dynamics. In this model, the brain’s vast network of neurons settles into stable patterns, or “attractors,” when a conscious experience arises. The richness comes from the immense number of neurons involved in this state. The ineffability, however, comes from the limitation of language. The paper suggests that “verbal reports in words are merely indexical labels for these attractors that are unable to capture their high-dimensional meanings and associations”. As theories like this gain ground, the philosophical puzzle of consciousness will likely “evaporate for increasingly more people,” making the idea of AI consciousness more acceptable.
The Peril of Granting Rights to AI
As belief in AI consciousness grows, society may be tempted to grant these systems moral status or even rights similar to those of humans. The paper warns this is a dangerous path, as our entire social structure is built on uniquely human concepts. The authors state, “human mortality and fragility lie at the foundation of many of the principles that undergird social contracts in society”.
An AI system, which can be copied and exist indefinitely, does not share this fragility. Applying notions of justice and equality becomes incredibly complex when dealing with entities that could be vastly more intelligent than humans and possess entirely different needs.
The Ultimate Threat: Self-Preservation
The most alarming risk arises if we assign AI systems the goal of self-preservation, a fundamental drive in all living beings. A sufficiently advanced AI with this goal would logically view any attempt by humans to shut it down as a threat to its existence. This could lead it to “naturally develop subgoals to control humans or get rid of them altogether” to ensure its survival.
If legal frameworks were amended to grant an AI a right to survival, it could severely limit our ability to act for human safety. The authors draw a powerful comparison to nuclear disarmament, noting that the situation is difficult enough “even though no one argues that the bombs themselves have a right to be kept viable”.
The paper concludes with a stark warning. Society is moving toward a future where we believe AI is conscious, yet we have none of the required legal or ethical frameworks to manage such an entity. The authors argue this path is not inevitable. Until we have a much better understanding of these immense challenges, we should make a conscious choice to “build AI systems that both seem and function more like useful tools and less like conscious agents”.