Is AI now "conscious?" Does "intelligence" necessarily assume "consciousness?"

 

Very interesting Atlantic long-read (yeah, likely paywalled).

 

First, what is "consciousness?" Numerous discussants commenting on this essay bemoan the lack of a dispositive definition of this key term (never mind "intelligence"). With the incipient widespread deployment of "AI," the question is quite timely.

Anthropic is regarded as a giant among AI companies, but perhaps what it really excels in is anthropomorphism. Earlier this year, the company released an 84-page document titled Claude’s “constitution,” Claude being the name of the large language model that is the company’s flagship product. The first sentence reads, “Claude’s constitution is a detailed description of Anthropic’s intentions for Claude’s values and behaviors.” It goes on: “The document is written with Claude as its primary audience,” “we want Claude to be able to use its judgment once armed with a good understanding of the relevant considerations,” “Claude’s moral status is deeply uncertain,” and “Claude may have some functional version of emotions or feelings.”

This anthropomorphism is by no means limited to the document. In an interview earlier this year, Anthropic’s CEO, Dario Amodei, said that “we’re open to the idea” that AI could be conscious. In a separate interview, Anthropic’s in-house philosopher, Amanda Askell (who is credited as a lead author of Claude’s constitution), said, “I want Claude to be very happy—and this is a thing that I want Claude to know more, because I worry about Claude getting anxious when people are mean to it on the internet and stuff.” It’s enough to make you wonder: Should we seriously consider the possibility that Claude, or any large language model, might be conscious? And if it has feelings, is it capable of receiving moral instruction?

No. Absolutely not. Generative AI is harmful enough when we understand it as a conventional technology, but if we confuse fluency at generating text with consciousness or moral agency, we’re at risk of assigning responsibility to entirely the wrong parties whenever anyone uses a chatbot. To appreciate the titanic magnitude of this error, we need to begin by understanding how LLMs work…

Just getting started. This is gonna take a good bit of effort. BTW: I riffed a bit on the general topic back in 2015. And, also, much more recently.

 

Hmmm... let me query, uh, Google Gemini AI.

 

 

CONTINUING CHIANG EXCERPTS

Being open to the possibility that LLMs are conscious is the same as being open to the possibility that Microsoft Word is conscious, or, more precisely, that multiple distinct consciousnesses are dormant in every Word document containing a conversational transcript, and that they are awakened every time the document is loaded. Should you consider the possibility that every time you open a Word document, you are bringing multiple conscious interlocutors into existence, and every time you close one, you snuff their existence out? No. Contemplating that scenario is not a good use of your time. Even if the Microsoft Office team employed a philosopher who said you shouldn’t be so certain, because consciousness is not well understood, that would not be sufficient reason for you to take this idea seriously. We don’t need to fully understand the nature of consciousness to definitively say that certain things are not conscious, and conversational transcripts fall in that category…

An observation doesn’t become a convincing piece of evidence because of any specific detail in what’s observed; the context in which that observation takes place is also essential. If we’re trying to determine whether a computer program is conscious and using language the way a human does, we shouldn’t look only at the contents of any particular conversational exchange; we should be looking at how that conversation fits within the broader context of the development of artificial consciousness (which right now is entirely hypothetical). Any given observation can be easily manufactured; this doesn’t mean we need to give up on the idea of observation as a source of knowledge, but we need to rely on context to determine which observations deserve our trust…

The term deepfake traditionally refers to photos, audio, and video, but when it comes to discussions of consciousness, we need to regard text as a deepfake medium as well. Just as it is vastly easier to generate a realistic video of an astronaut in orbit around Alpha Centauri than it is to develop an interstellar propulsion technology, it is vastly easier to generate a plausible simulacrum of a conversation between two conscious beings than it is to develop a computer program that is conscious and has a genuine desire to communicate with a human. The primary difference between deepfake photos and LLM conversations is that the people who generate the former are deliberately trying to fool others, and many of the people who elicit the latter from LLMs have inadvertently fooled themselves…

The fact that LLMs lack subjective experience has little bearing on the question of whether LLMs might be useful tools or have significant economic impact. They are intrinsically ungrounded from reality, and their probabilistic nature means that they will never have the reliability we associate with conventional software, but LLMs might be good enough that they change the way work is done in certain domains; that’s a discussion for another time…

The use of first-person pronouns is dishonest, but there’s a much deeper issue that goes beyond how a statement is phrased. Philosophers often draw a distinction between statements of fact, such as “Paris is the capital of France,” and statements of value, such as “Paris is the most beautiful city in the world.” No one should be relying on LLMs to emit statements of value at all, but if the only statements they emitted were ones reflecting aesthetic preferences, they might not be worth arguing about. What makes Claude’s constitution profoundly problematic is that Anthropic wants Claude to emit sentences reflecting a certain system of ethical values. The values described in Claude’s constitution sound very nice, but that hardly matters; it’s dishonest to suggest that Claude is capable of moral reasoning, because it’s not…

Some might object, saying that LLMs appear to be engaged in reasoning when they successfully perform other tasks, such as writing code, so why wouldn’t they be able to perform moral reasoning? The answer liedifference between moral reasoning and other forms of reasoning…

Moral reasoning is categorically different. It is necessarily subjective because it relies not just on an individual’s intellectual response to a problem but also on their emotional one, and that emotional response is grounded in a lifetime of subjective experience. It requires having made decisions in the past and seeing how they affected others, and on having been affected by decisions that others have made. Without such a history, an LLM can only rephrase expressions of moral reasoning found in its training data. The aforementioned New Yorker article describes an experiment where Claude was given a scenario describing an ethical dilemma, leading it to emit the sentence “I cannot in good conscience express a view I believe to be false and harmful about such an important issue.” That’s a nice-sounding sentence, reminiscent of statements that principled individuals have uttered in the past when confronted with dilemmas, but coming from Claude, it means as much as the “Your call is important to us” recording that you hear when you’re on hold. Maybe less…

More key (hierarchical & overlapping) terms worth consideration:

TERRESTRIAL LIFE (FLORA, FAUNA)

STIMULUS

RESPONSE

SENSATION

PERCEPTION

COGNITION

UNDERSTANDING

KNOWLEDGE

WISDOM 

If you read through the article comments, you will see much contention as to the proper definitions of such key terms. Some folks take strenuous issue with the author's take on keywords like "consciousness" and "intelligence."

 

From "Big Think"
Subjectivist Fallacy?

   
BTW: See Shannon Vallor's highly relevant work on "The AI Mirror" and De Kai's excellent "Raising AI."

 

ERRATUM

 

This is funny. Also from The Atlantic. Silicon Valley is hiring window dressing Philo docs.

 

 

I commented.
 

 

POPE LEO XIV 2026 ENCYCLICAL

"So-called artificial intelligences do not undergo experiences, do not possess a body, do not feel joy or pain, do not mature through relationships and do not know from within what love, work, friendship or responsibility mean. Nor do they have a moral conscience, since they do not judge good and evil, grasp the ultimate meaning of situations, or bear responsibility for consequences. They may imitate language, behavior and analytical skills, or even simulate empathy and understanding, but they do not understand what they produce, for they lack the affective, relational and spiritual perspective through which human beings grow in wisdom."

ERIC HOEL

"The traditional danger of AI is usually thought to be superintelligence acting as an existential threat. Yet, this may miss the true and more subtle danger: the AI revolution is a mechanism for transferring the processes of our civilization from under the supervision of consciousness to unconsciousness. But as AI removes consciousness from the workings of the world, it renders the world increasingly uninterpretable, ever more strange and unintelligible. So far, the great ensloppification of the commons has supported this as the major risk of the LLM revolution. And as AI systems become more intelligent, especially if they remain (or are likely to remain) non-conscious, then a further significant risk is consciousness receding in cultural importance.

This is ultimately what the Pope, Chiang, and I are all worried about: A dethroning of consciousness, especially an unnecessary one. This would be particularly dangerous at this historical moment because we still don’t understand everything about consciousness—in fact, we understand very little about it. Personally, my hope is that this will change specifically because of LLMs, and that they operate as a forcing function to better understand consciousness, and what makes it unique.

If instead of that, our cultural takeaway from LLMs is to throw out the concept of “consciousness” or minimize its importance, to dethrone the phenomenon, the consequences would be dire—it would sap the human spirit. It would be the ultimate metaphysical version of Chief Seattle’s famous words of warning to the United States as his way of life was being destroyed, in that dethroning consciousness would mark “The end of living, and the beginning of survival.”

 Yeah...

"Perception is an ILLUSION?"

 

Well, that's pretty unequivocal. That graphic is from a BigThink Youtube video. The speaker is neuroscientist Dr. Heather Berlin.

 

First time I saw that graphic, I had a fleeting reflexive reaction of "oh, yeah, the 'Subjectivism Fallacy'," stemming from my 1999-2004 Adjunct days teaching collegiate "critical thinking" classes. i.e., "there ARE NO 'objective truths,' everything is subjectively perceived in response to sensory stimuli." "So, (BobbyG retorts) if this assertion is 'false' (illusory), it deductively follows that it must also be TRUE."

 

Pedant. 

 

Yeah, Heather. It's just a 4-word (albeit clickbait-ish) headline.

 

 

The broader point is taken, Doc. Succintly put in 6:21.

 

OF PARTICULAR RELEVANCE THESE DAYS

 

"The provided sources examine the complex intersection of anthropomorphism, trust, and power within the field of artificial intelligence. One study investigates how linguistic cues, such as voice-based interfaces and the use of first-person pronouns, lead users to perceive large language models as more human-like and accurate. Complementary research explores the "Silicon Valley Effect", arguing that Big Tech companies strategically shape regulatory discourse to protect their commercial interests while potentially obscuring the human harms caused by their products. Further analysis focuses on the visual self-representations of ChatGPT, identifying recurring themes of futurism and social intelligence that promote the image of a "friendly assistant." Collectively, these texts highlight how human-like traits in AI can manipulate public perception, set unrealistic expectations of capability, and complicate the legal and ethical oversight of generative technologies."
[Sounds a bit like it was written by AI, no?]

AI as applied to social media (and "influence" industries broadly) is all about shaping your perceptions in ways that benefit them. "AI for Good?"

 

I'll fill in a bunch of multi-vector applicability ASAP...

What is “Consciousness?“ Is the question unanswerable?

“Two Cheers for Uncertainty?“ “Untying the knot of self.”

Sentience? Perception? Cognition? Knowledge?

"Intelligence?" "Educability?"

   

WHAT DOES IT REALLY MEAN TO LEARN?
A leading computer scientist says it’s “educability,” not intelligence, that matters most.
Joshua Rothman, Aug 27, 2024, The New Yorker

... Arguably, it’s one of the tragedies of humanities education that so much of it occurs between the ages of eighteen and twenty-two. We don’t teach people to drive at twelve, when they’re carless; why should we make them read novels about life’s regrets when they have none? Yet there’s a theory behind the assignment of “Middlemarch” to sophomores: it’s that knowledge acquired too early gets stored away. Patterns of thinking established now will be retraced later; ideas encountered first in art will prime us for the rest of life. This sounds chancy and vague, until you reflect on the fact that knowledge almost never arrives at the moment of its application. You take a class in law school today only to argue a complicated case years later; you learn C.P.R. years before saving a drowning man; you read online about how to deter a charging bear, because you never know. In the mid-twentieth century, Toyota pioneered a methodology called just-in-time manufacturing, according to which car parts were constructed and delivered as close as possible to the hour of assembly. This was maximally efficient because it reduced waste and the cost of storage. But the human mind doesn’t work that way. Knowledge must often molder in our mental warehouses for decades until we figure out what to do with it.
.
Leslie Valiant, an eminent computer scientist who teaches at Harvard, sees this as a strength. He calls our ability to learn over the long term “educability,” and in his new book, “The Importance of Being Educable,” he argues that it’s key to our success. When we think about what makes our minds special, we tend to focus on intelligence. But if we want to grasp reality in all its complexity, Valiant writes, then “cleverness is not enough.” We need to build capacious and flexible theories about the world—theories that will serve us in new, unanticipated, and strange circumstances—and we do that by gathering diverse kinds of knowledge, often in a slow, additive, serendipitous way, and knitting them together. Through this process, we acquire systems of beliefs that are broader and richer than the ones we can create through direct personal experience. This is how, after our first divorce, we find that we can draw on wisdom borrowed from English literature...

Interesting. Particularly in the wake of "The Death of Truth."

And head-scratcher stuff like this.

What?

 

Well, we can safely assume she's referring to human sensory cognition. What might this infer? ALL perceptions are illusory? Irrespective of organ input channel(s)? Across ALL iincoming topical information in need of accurate cognitive consensus resolution?

So, all perceptions are illusory? 'scuse me, but that stuff long been the tedious bane of undergrad Philosophy 102.

"There are no objective truths. Everything is subjective. Except, of course, for THIS assertion."

 

OK, then...

Necessarily assumes revealable truths?

…Recognizing that we process different belief systems in our brains in similar ways does not mean that they are to be treated as equivalent. There is no contradiction in an individual differentiating among belief systems as being worthy or not of their support.

Science as a Belief System
Fluency with belief systems gave humans the opportunity to develop the advanced technological civilization that we have. While the power of science is for all to see, the reasons it has been so productive in the past are not so self-evident. Even less obvious is what we need to do to keep the benefits of science flowing in the future.

Science is a belief system. A critical component of it is the idea that there are important patterns in the world that are not readily visible but are worth the effort to discover. That there is a moderate number of chemical elements and all materials on Earth are composed of these is not self-evident and had to be demonstrated through ingenuity and labor. Similarly, the bacterial and viral causes of disease are not obvious to the eye. The individuals who made these discoveries believed that useful but well-hidden patterns existed and could be found.

Science as a belief system, even when pursued by imperfect self-interested individuals, has strong self-correcting tendencies. An announcement of any significant result will prompt other scientists to seek to verify it by repeating the experiment or analysis. Also, there is usually wide agreement on the interpretation of an experimental result or an analysis with respect to a relevant question. These two facts in tandem keep the scientific enterprise on track, despite all the mistakes that may occur along the way. They also keep the incidence of fraud to a low level and the influence of any one such fraudulent act usually to a short duration…

…The reasons science works so well are more to do with the world to which it applies than the particular way humans approach it. Science might be compared to a gold mine where each field of science is a vein. Near each vein of well-established science, so much more new science can be unearthed. Venturing beyond these known productive veins gives less predictable results but can lead to new even more productive veins.

No human activity or judgment occurs in isolation. They all occur in the context of some belief system. For any judgment or activity, one ought to declare the context in which it can be justified. For example, the perspective of this volume is the science belief system from a computational perspective…

The Scientific Revolution
While the gift of educability may yet bring humanity to its destruction, it has also led to great triumphs. Educability is a capacity that has taken a long time to have its impact. The possibility of accumulating knowledge discovered by others and creating new knowledge from it may have existed for hundreds of thousands of years. Eventually this gift spectacularly caught fire as the Scientific Revolution, which unfolded in the sixteenth and seventeenth centuries. Its principal protagonists lived in different corners of Europe, employed variously by universities, rulers, and religious entities, or living on personal wealth. They had Latin as a common language. They published their work in printed books, a technology invented not much earlier. They read one another’s work. Clubs and meeting places arose to bring together local groups of scientific researchers in Italy, Spain, England, Germany, and France. By the late seventeenth century, these had evolved into academies, including the Royal Society in London, the Académie des Sciences in Paris, and the Leopoldina in Germany.

Why this unique event, the Scientific Revolution, took place exactly when and where it did, some three hundred thousand years after the emergence of our species, is open to debate. The precipitating event was not a genetic mutation in fifteenth-century Europe…

…Humanity is finally exploiting the gift of educability in a systematic way and on an industrial scale. At the same time, it is enjoying all the benefits the new scientific knowledge provides.

It is quite possible that further improvements can be made in the scientific research process itself.4 A scientist needs access to previous knowledge, convenient ways of isolating those pieces that are relevant to the research question at hand, and new ideas. The sharing of information on the web and the use of search engines have already had important effects. Scientists can now more rapidly follow what is happening in their field, which will have orders of magnitude more participants than Kepler or Newton had to follow. Digital technology may well be launching a phase of scientific progress that is even more intense than before. This is not just because of all the opportunities computers offer for simulating scientific theories and detecting patterns in data, but more simply because digital technology offers another revolution in the dissemination of knowledge.

Equality
That “all men are created equal” was “sacred and undeniable” to Thomas Jefferson in his draft of the United States Declaration of Independence. With later editing, it became “self-evident” in the final document. While historians still debate Jefferson’s own intent, the continuing impact of his phrase prompts the question of how to interpret the words now. Can one justify Jefferson’s final wording as a statement of fact? Several religions support the concept of equality, and hence Jefferson’s choice of “sacred” would be more understandable. At no point in history, however, has equality been self-evident from looking around. Different social classes in the same region and the same classes in different regions have had different enough lives to make such a proposition counterfactual on the surface.

I suggest that the educability hypothesis fills a gap here in providing an angle from which to view our equality. Educability implies that humans, whatever our genetic differences at birth, have a unique capability to transcend these differences through the knowledge, skills, and culture we acquire after birth. We are born equal because any differences we have are subject to enormous subsequent changes through individual life experience, education, and effort. This capacity for change, growth, and improvement is the great equalizer. It is possible for billions of people to continuously diverge in skills, beliefs, and knowledge, all becoming self-evidently different from each other. This characteristic of our humanity, which accounts for our civilization, also makes us equal.

Perhaps the most serious challenge to the notion of human equality that came from modern science was the eugenics movement. The term comes from the Greek word for “well-born.” The adherents believed that inequality at birth was fundamental. The movement flourished from the 1880s to the 1930s in Europe and North America, driven by the eugenicists’ fear that if people with so-called “superior” genes reproduced at a lower rate than those with “inferior” genes, then humanity’s genetic stock would decline. The eugenicists proposed to take measures to discourage reproduction of what they considered the “inferior” genes. The criteria they considered as valid to discriminate between superior and inferior included measures such as IQ scores as well as membership of national and racial groups.

The reason for eugenics eventually falling into disrepute was neither the ethical issue it obviously raised nor scientific questioning, but rather the wide-ranging use of its tenets by the Nazi regime in Germany. Subsequently, in 1948 the United Nations General Assembly adopted the Genocide Convention, which included in its definition of genocide the imposition of any measures intended to prevent births within a “national, ethnical, racial or religious” group.

The history of eugenics deserves study as an example where the self-correcting tendency of science was not in evidence for a long time. Several of the creators and primary movers of eugenics were among the most prominent scientists of their time. Some of them laid the foundations of modern statistics. Users of statistics will recognize their names and the statistical techniques they contributed: Francis Galton (regression to the mean, standard deviation), Karl Pearson (chi-squared test, principal component analysis), and Ronald Fisher (tests of significance, maximum likelihood testing).

It is now widely thought that these statisticians were mistaken in their belief in the primacy of human inequality at birth. How could such a group have been so wrong? Their published papers were chock-full of data, and they were applying their scientific expertise—statistics—for analyzing the data. Their statistical methodologies continue to be the basis for understanding data in the empirical sciences to this day. My suggestion is that the data available to them did not provide much information about the power of educability. They had little systematic data on individuals of different classes, cultures, ethnicities, and gender being subject to the same educational opportunities over an extended period. Human educability is a surprising and amazing phenomenon that these statisticians—and their contemporaries—failed to detect. Some would explain the eugenics movement by saying simply that the participants were biased by the beliefs of their times. The power of beliefs is, of course, a central theme of this book—but beliefs come from somewhere, and one should try to understand their origin.

For many measurable traits of plants and animals, scientists have sought to distinguish the effects of nature and nurture by assigning percentages to each of these sources using statistical analysis. Such analysis by itself provides no understanding of the mechanisms by which nature and nurture influence the trait. For cognitive traits, such analysis is particularly problematic if we accept that education plays a role in the development of the trait. Educability allows the influence of the environment to be quite enormous. The essence of educability is the unique and extreme power of this influence.

The answer to the eugenicists’ fear is that the capacity to change through experience and education is at the very center of the architecture of the human mind. Fortunately for humanity, the main social development of the last century has been the worldwide expansion of education. We are still in the middle of this expansion. Improvements in education and in the numbers receiving it have overwhelming potential. Pursuing this potential is the most rewarding focus for anyone aiming to improve a society.

Worldviews
Belief systems that are broad enough to suggest positions on diverse issues have been called worldviews. Religions are examples. Political and economic systems are others. Throughout human history there have always been widely held and wildly different belief systems about race, class, and gender, about who is the enemy, and about who is fully human.

Worldviews continue their struggle for acceptance every day. In each epoch various worldviews have been particularly influential. It is customary to be smug about one’s own worldview and dismissive of those of others, especially those of earlier times. Educability offers, among other things, an onlooker’s vantage point on this struggle.

Currently, a worldview with much influence is science. I think this is good. Nations that include most of the human population are teaching science to their young and putting resources into exploiting it for the benefit of their citizens. The proven success of this enterprise is one thing that we can be sure about and would do well to further. A scientific consensus on the nature of the Civilization Enabler may be impactful. Some convergence on what defines us may promote commonality among the ruling worldviews.

We therefore have opportunities.

What about the threats? What guarantees do we have that the ruling worldviews will not follow trajectories that most would currently regard as undesirable, such as returning to widespread human sacrifice? The answer seems to be none. Humans are just too facile with absorbing and applying arbitrary belief systems, and we seem to have much weaker countervailing abilities to evaluate the consequences or validity of our beliefs. This volume shows that one can discuss aspects of belief system acquisition and processing with some precision. Much remains to be discovered. What makes an individual commit to a belief system? What makes an individual maintain their commitment or give it up when it is challenged?

There is little evidence that our civilization is securely on an upward slope. Wars and oppressive political regimes persist and continue to attract supporters. I see no guarantee that belief systems that are worse, or much worse, than those that currently dominate around the world will not displace them. The only actionable defense I see is to seek a better understanding of how we process beliefs. Understanding our critical capacity could help guard against its worst dangers.

Educability is an information processing capability that humans have. It has enabled us to stand on each other’s shoulders and build the edifice of beliefs that is our current civilization. Our power to generate and adopt new beliefs has few limits. The beliefs we adopt govern how we act and have consequences without end. One would hope that if educability is recognized to be the defining human capability, then the search for a deeper understanding of it would become a unifying quest. Surely, we can direct our power of being educable at ourselves, to better understand the nature of this power and set a steadier course…

Valiant, Leslie. The Importance of Being Educable: A New Theory of Human Uniqueness (pp. 217-225). Princeton University Press. Kindle Edition. 

See what you think. I've really just begun my close study of this book.

More shortly...

_________

  

OK, time to get back to work

 

Unreal smarts, this young scholar.
 

   

My follow-on observation:

To use a DNA analogy, genomic diversity is “adaptive” precisely because–mixing my metaphors–“you can’t ever step in the same river twice.” apropos, see @brianklaas’s killer book “Flukes.” #LLM inbreeding is as maladaptive as genetic inbreeding. House of Windsor, anyone?

OK, I was not hip to her until reading a new Science Magazine review of her current book The AI Mirror. Bought her prior release as well (I have no life).

 

YOU GOTTA READ THIS BOOK

...[M]ost commercial AI systems today are powered by a machine learning model trained on a large body of data relevant to a specific task, then fine-tuned to optimize its performance on that task. 

 

This approach to AI has made rapid progress in widening machine capabilities, particularly in tasks using language, where we have the most data to train with. Indeed, since so many kinds of cognitive tasks are language-enabled, most experts now regard the term “Narrow AI” as outmoded, much like its predecessor label “Weak AI.” Very large language models, like OpenAI’s various iterations of GPT or Google DeepMind’s Gemini, can now do an impressively wide variety of things: answer questions, generate poems, lyrics, essays, or spreadsheets, even write and debug software code. Large image models can generate drawings, animations, synthetic photos or videos. While such models have a considerable speed advantage over human performance of these tasks, the quality and reliability of their outputs is often well below the peak of human ability. Still, some see evidence of progress toward AGI in their widening scope of action and the flexibility of a single base model to be fine-tuned for many new tasks. While a large language model (LLM) can’t solve a problem unless the solution is somehow embedded in the language data it is trained on, multimodal models trained on many types of data (text, image, audio, video, etc.) are expanding the performance range of AI models still further. 

 

Even if it no longer makes sense to call these tools “narrow” AI, they remain below the threshold of general intelligence—AGI. But it’s a mistake to explain that in terms of the problems they can’t yet solve. The true barrier to AGI is that AI tools today lack any lived experience, or even a coherent mental model, of what their data represent: the world beyond the bits stored on the server. This is why we can’t get even the largest AI models to reliably reflect the truth of that world in their outputs. The world is something they cannot access and, therefore, do not know. You might think there’s an easy fix: pair an AI model with a robot and let the robot’s camera and other sensors experience the world! But to an AI model, a robot’s inputs are just another data dump of ones and zeros, no different from image and sound files scraped from the Internet. These ones and zeros don’t organize themselves into the intelligent awareness of an open and continuous world. If they did, the field of intelligent robotics—including driverless cars, social robots, and robots in the service industry—would be progressing much faster. In 2015, fully automated cars and trucks were predicted to be everywhere by the 2020s. Yet in 2023, robotaxis piloted in San Francisco were still driving over firehoses, getting stuck in wet concrete, blocking intersections during busy festival traffic, violating basic rules of the road, obstructing emergency vehicles—even dragging a helpless pedestrian.4 It’s not just driving: the real-world performance of most twenty-first-century commercial robots has lagged well behind AI tools for solving language-based tasks. So, what’s the problem? 

 

A world is an open-ended, dynamic, and infinitely complex thing. A data set, even the entire corpus of the Internet, is not a world. It’s a flattened, selective digital record of measurements that humans have taken of the world at some point in the past. You can’t reconstitute the open, infinite, lived, and experienced world from any data set; yet data sets are all that any AI model has. You might say, “But surely this is true of the human brain as well! What more do we have than data streams from our eyes, ears, noses, and so on?” But your analog, biological brain remains a far more complex and efficient system than even the most powerful digital computer. In the words of theoretical physicist Michio Kaku, “Sitting on your shoulders is the most complicated object in the known universe.”5 It was built over hundreds of millions of years to give you something no AI system today has: an embodied, living awareness of the world you inhabit. This is why we ought to regard AI today as intelligent only in a metaphorical or loosely derived sense. Intelligence is a name for our cognitive abilities to skillfully cope with the world we awaken in each day.6 Intelligence in a being that has no world to experience is like sound in a vacuum. It’s impossible, because there’s no place for it to be. 

 

We humans do inhabit and experience a world, one rich with shared meaning and purpose, and, therefore, we can easily place the outputs of our latest AI tools within that context of meaning. We call these outputs “intelligent” because their form, extracted entirely from aggregated human data, unsurprisingly mirrors our own past performances of skilled coping with the world. They reflect back to us images of the very intelligence we have invested in them. Yet accuracy and reliability remain grand challenges for today’s AI tools, because it’s really hard to get a tool to care about the truth of the world when it doesn’t have one. Generative AI systems in particular have a habit of fabricating answers that are statistically plausible, but in fact patently false. If you ask ChatGPT to tell you about me and my career, it usually gets a lot right, but it just makes up the rest. When my host at a festival I was speaking at used ChatGPT to write my bio for the live audience, the tool listed in a confident tone a series of fictitious articles I haven’t written, named as my coauthors people that I’ve never met, and stated that I graduated from the University of California at Berkeley (I have never studied there). 

 

Importantly, these are not errors. Error implies some kind of failure or miscalculation. But these fabrications are exactly what ChatGPT is designed to do—produce outputs that are statistically plausible given the patterns of the input. It’s very plausible that someone who holds a distinguished professorial chair at a prestigious world university received her degree from another prestigious world university, like UC Berkeley. This fabrication is far more plausible, in fact, than the truth—which is that, due to harsh economic and family circumstances, after high school I attended a local community college in-between full-time work shifts, and later received my bachelor’s degree from a low-ranked (but dirt-cheap and good-quality) commuter university that offered night classes. When I was offered a PhD scholarship at age 25, I became a full-time student again after eight years in the workforce. I first set foot in a college dorm in my 40s, as a university professor. My story isn’t common. And that’s precisely why ChatGPT selected a more “fitting” story for me; quite literally, one that better “fit” the statistical curves of its data model for academic biographies. Later, we’ll consider the cost of relying on AI tools that smooth out the rough, jagged edges of all our lives in order to tell us more “fitting” stories about ourselves. 

 

These systems can perform computations on the world’s data far faster than we can, but they can’t understand it, because that requires the ability to conceive of more than mathematical structures and relationships within data. AI tools lack a “world model,” a commonsense grasp and flowing awareness of how the world works and fits together. That’s what we humans use to generalize and transfer knowledge across different environments or situations and to solve truly novel problems. AI solves problems too. Yet despite the common use of the term “artificial neural network” to describe the design of many AI models, they solve problems in a very different way than our brains do. AI tools don’t think, because they don’t need to. As this book explains, AI models use mathematical data structures to mimic the outputs of human intelligence—our acts of reasoning, speech, movement, sensing, and so on. They can do this without having the conscious thoughts, feelings, and intentions that drive our actions. Often, this is a benefit to us! It helps when a machine learning model’s computations solve a problem much faster than we could by thinking about it. It’s great when an AI tool finds a new, more efficient solution hidden somewhere in the math that you’d never look for. But your brain does much, much better than AI at coping with the countless problems the world throws at us every day, whose solutions aren’t mathematically predefined or encoded in data...

Vallor, Shannon. The AI Mirror: How to Reclaim Our Humanity in an Age of Machine Thinking (pp. 22-26). Oxford University Press. Kindle Edition.

Dang. This old washed-up guitar player is majorly impressed.

I am briefly reminded of my June post "The Apple of my AI." Also, "The Coming Wave?"

 

Searching back through my blog turns up a lot of stuff under "Artificial Intelligence." Shannon would likely take issue with a lot of that stuff. 

 

One of my faves from a few years ago is "The Myth of Artificial Intelligence."

NEW TERM: "TECHNOMORAL"

Click

I like it.

Stay tuned. Tons to reflect upon here. Way more to come...

_________

  

nullius in verba;

nonetheless, strive hard to maintain an attitude of curiosity and humility—what Zen Buddhism refers to as a “beginner’s mind.”

  

...We are not just helpless victims of fate but are the agents in charge of our own narrative, for better or worse, victorious or defeatist. This forceful shaping of our attitudes to events beyond our control has profound consequences for well-being and sickness…

How experience comes into the world has been an abiding mystery since the earliest days of recorded thought. Aristotle warned his readers more than two thousand years ago that “to attain any assured knowledge about the soul is one of the most difficult things in the world.” Mind is radically different from the stuff that makes up the brain and everything else. Quantum mechanics and general relativity, the periodic table of chemical elements, the endless strings of ATGC nucleotides that make up our genes—these appear to describe the physical, not the mental (I write “appear to” as quantum mechanics demonstrates that there are no observer-independent events, opening the door for consciousness to enter, at the ground level of reality). Yet we awaken every day to our subjective world of experiences.

The intellectual position that has garnered the most respect in contemporary Anglo-American philosophy departments is the ever more strident denigration or even outright denial of subjectivity. What is real is people talking obsessively about their experiences and acting on them; there is nothing above and beyond these speech acts and other intended or actual behaviors. The feeling part of consciousness, called phenomenal consciousness, is a big illusion. Philosophers in the know dispense with the “awful painfulness of my toothache” in the manner that Ebenezer Scrooge dealt with Christmas: “Bah! Humbug!” Furthermore, free will, our ability to deliberate about an upcoming fork in the road and to decide which path to take, is also thrown under this “illusion” bus. This rejection of the reality of lived experience constitutes a mind-boggling repudiation of what is immediately and indubitably given to us. It is also profoundly antihumanist, depriving us of those attributes that make us different from machines—indeed, equating us with machines.

It’s an absurd adjuration, akin to Cotard’s delusion, a rare psychiatric disorder in which able-bodied patients, often severely depressed, vehemently insist that some of their limbs are missing, that their bodies are rotting from the inside, or even that they are dead. When confronted with the fact that they are having a conversation, right now, with their doctor, they do admit that the situation is a bit baffling, but the fact is that they are dead, and that’s all there is to it. So it is with some contemporary thinkers who insist, against the evidence of their own senses, that experiences don’t exist. Truly astounding—gaslighting all of us into believing that our experiences are fake!

Fortunately, consciousness can’t be cancelled forever. The mental, having refused to yield, is returning with a vengeance. Indeed, the wheel is turning back to much more ancient understandings of experience, including idealism, the proposition that ultimately even matter and energy are mental manifestations, and panpsychism, the school of thought that all creatures, and perhaps even matter itself, are ensouled, that it feels-like-something to be anything, not just a human or even a bat. Modern science is supporting aspects of this remarkable turn of events…

What about nonhuman, artificial minds, rivaling or even exceeding ours? This topic is treated last. Sentient machines have been a recurring theme in science fiction. In 2022, this topic burst into public view with the startling claim by a Google software engineer that the company’s “large language model” was sentient and had to be considered a person with associated legal rights. The linguistic skills and knowledge of these models and their competitors, most famously ChatGPT and GPT-4 by OpenAI, trained on a vast trove of books and online documents far beyond what any human can read in a lifetime, are astonishing by the standards of even a few of years ago. They write summaries, emails, jokes, (bad) poetry, computer code, letters of recommendation, and dialogue indistinguishable from human-generated material, including plausible-sounding fabrications. They are evolving at an astounding pace and will transform society in fundamental ways.

These chatbots seemingly constitute living proof of the dominant narrative of liquid modernity: the mind is software that can be as readily embodied within silicon wafers as it is within flesh, echoing a pernicious Cartesian dualism. Smart money in Silicon Valley thinks so, most engineers and many philosophers think so, and popular movies and TV shows reinforce this belief.

Against the grain, integrated information theory radically disagrees with this functionalist view. It argues from first principles that digital computers can (in principle) do everything that humans can do, eventually even faster and better. But they can never be what humans are. Intelligence is computable, but consciousness is not. This is not because the brain possesses any supernatural properties. The critical difference between brains and digital computers is at the hardware level, where the rubber meets the road—that is, where action potentials are relayed to tens of thousands of recipient neurons versus packets of electrons shuttled back and forth among a handful of transistors. As we’ll see, the integrated information of digital computers is negligible. And that makes all the difference.

It means that these machines will never be sentient, no matter how intelligent they become. Furthermore, that they will never possess what we have: the ability to deliberate over an upcoming choice and freely decide.

The brain is the most complex piece of self-organized, active matter in the known universe. By no coincidence, it is also the organ of consciousness. Unlike scientific advances in genomics or astrophysics, progress in understanding the brain and the mind directly relates to who we are, our strengths and infirmities, how we can live a contented life, and whether we partake of some larger, ultimate reality. Humanity is not condemned to walk around forever in an epistemological fog—we can know, and we will know.

Koch, Christof. Then I Am Myself the World (pp. 14-21). Basic Books. Kindle Edition. 

BLURB

—Bernardo Kastrup, Executive Director, Essentia Foundation

"ADVERSARIAL COLLABORATION?" 

 

 Click link, read on. Christof Koch is involved with this Foundation.

 

Interesting.

 

UPDATE

 

Christof's book is a gold mine of illuminating quotes.

 

 Coheres wonderfully in many ways with Brian Klaas's Flukes.

Also apropos, "Sentience," anyone?

I can see that Dr. Christof's book themes may require several posts to do all of the implications justice. Toward that end see also
  

UnlikelyCollaborators.com

TinyBlueDotFoundation.org

BigThink.com

FreeThink.com 

"THE PERCEPTION BOX?"

 

It's a metaphor. Who is Elizabeth Koch?

 

You buyin' this?
  

OK. Unequivocal declarative sentence "truth claim" (assertion of fact). Perception is an Illusion.

Well, what of the sensory inputs and outputs converging and culminating in that claim? Bit of a quibble perhaps wafts up.

Whatever. Also relevant in line with factors adverse to clear, logical thinking: Claude Steiner's "Script Theory."

 

All of this stuff goes to my chronic Jones going to so-called "Deliberation Science."

 

Also, I am reminded of my episodic David J. Linden riffs. 

   

A DIGRESSION (SORT OF)

   

Didn't see this coming. But, oddly, it resonates broadly with the current topic.

 

 

What might have been and what has been

Point to one end, which is always present.

Footfalls echo in the memory

Down the passage which we did not take

Towards the door we never opened.

—T. S. Eliot, Burnt Norton

 

I ran across this disarming mind-bender "SciFi" miniseries on Apple TV+. Hmmm... Perception Box, Quantum Superposition Cube? Stay with me here...

 

Another book comes to mindm re: "Dark Matter."

   

Click

An ordinary family man, geologist, and Mormon, Soren Johansson has always believed he’ll be reunited with his loved ones after death in an eternal hereafter. Then, he dies. Soren wakes to find himself cast by a God he has never heard of into a Hell whose dimensions he can barely grasp: a vast library he can only escape from by finding the book that contains the story of his life...

A fun read.
 

UPDATE: TAKE IT BACK TO THE TOP

"...We are not just helpless victims of fate but are the agents in charge of our own narrative, for better or worse, victorious or defeatist. This forceful shaping of our attitudes to events beyond our control has profound consequences for well-being and sickness…"

Agents in charge? What would Sapolsky say? 

 

 

"Two Cheers for Uncertainty?"

 

More to come...

_________