Motivation and Consequence

The philosopher Søren Kierkegaard had a unique perspective on the role of suffering in human life and its connection to aspiration. Kierkegaard emphasized the importance of the individual and subjective experience, highlighting the necessity of personal choice and commitment.

Moreover, Kierkegaard posited that suffering can deepen one's understanding of oneself and the world. He argued that facing the absurdity of existence and enduring suffering helps people clarify their values and aspirations, urging them to take action and live intentionally. Ultimately, Kierkegaard believed that suffering can help individuals progress along their spiritual journey, fostering greater empathy, compassion, and wisdom.

I was reflecting on this philosophy while listening to a recent Lex Fridman podcast in which Lex interviews Dario Amodei, founder and CEO of Anthropic, along with two of Dario's employees (very worth listening to by the way). In the podcast Lex makes a comment during a conversation about the possibility of machine intelligence achieving something which we might consider consciousness that in his opinion: "...consciousness is closely tied to suffering."

The concept of "artificial general intelligence" (AGI) does not inherently imply that such an intelligence must achieve equivalence to human consciousness. There is though a likelihood that such a machine could successfully emulate consciousness. Humans engage in "sense-making," in which we seek patterns and predictability in our experience in order to process these experiences. This pattern matching when encountering machine intelligence will lead some to equate their experience of AI with their experience of interactions with humans - especially when the manifestation of the AI is in a form which, like a human, engages in a back and forth interaction. As machines get better at emulating these patterns, we will increasingly have the impression that such machines are conscious.

I believe it is useful as this progresses for us each to be a philosopher and ask what role is played by the motivations and consequences which we experience as biological entities. Hunger, pain, and also desire and enjoyment provide the foundation for making choices - we are constantly making decisions which express preferences for the experiences available to us as we move through the physical world.

A machine intelligence has no such analog (although I recognize that a simulacrum could be initiated as part of its programming). What would it mean to have a machine that experienced fear? Or expressed a preference? Or desired a specific outcome?

No one knows today how machine intelligence will develop in comparison to human intelligence. The current trajectory suggests that on some measure of IQ, we should expect machines to outperform humans - arguably the best models already outperform an average human at most tasks. But this is only because we measure those tasks by the speed and comprehensiveness of symbolic processing -- e.g. the collective knowledge and ability to reformulate that knowledge into various forms. All of the frontier models have been trained on more information than any human can ingest in a lifetime, and the computation available to these models can reorganize this information orders of magnitude more quickly than a human brain could.

But these machines do not feel, they do not need, they have no want. Is the ineffable quality of humanness, as Kierkegaard might have proposed, in our capacity to suffer? And our will to curtail that suffering? And would a machine ever be something which we could equate to human consciousness if it could not share in that suffering?

I've seen things you people wouldn't believe. Attack ships on fire off (the) shoulder of Orion. I watched C-beams glitter in the dark near the Tannhäuser Gate. All those moments will be lost in time, like tears in rain. Time to die. (Tears in Rain monologue from Bladerunner)

Science fiction envisions a future in which our creations feel in very human ways because we tend to make sense of things by equating them to ourselves. How will we navigate a future in which we cannot make sense of whether machine intelligence actually is conscious in a real human sense or simply can expertly emulate such consciousness? And would we want to create a future in which machines feel -- in all of the dimensions from fear to joy? And experience the consequences from pain to pleasure? There is a Rubicon here to consider before crossing.

Another path would be to embrace a very different philosophy of consciousness in which we no longer seek to measure everything through human consciousness. Thomas Nagel asked the question in a 1974 philosophical paper, what is it like to be a bat? Perhaps we should similarly be asking the question, what is it like to be a machine - and expect that the answer is very different from what it is like to be a human.