"Continuous discursive tinnitus"

An excellent New Yorker long-read. Totally timely.

The person who should have been best able to explain how we got here was the great German philosopher Jürgen Habermas, who illuminated how a feisty, principled public sphere is integral to democracy. But Habermas died in March, at the age of ninety-six, and, although he remained active until his final months, commenting on Ukraine, Gaza, and Eurobonds, he struggled to understand the turn history had taken. As a teen-ager in 1945, he had witnessed American soldiers enter his home town of Gummersbach, near Cologne, carrying messages of freedom and openness. Eight decades later, he watched American voters choose a leader who had advertised his fascistic bent in blood-and-soil rhetoric, fantasies of punitive violence, and a taste for bombastic architectural kitsch. The far right was making inroads across Europe, including in Germany. The print-based media culture that once anchored Habermas’s public sphere had devolved into a digital sludgefest that proved better at circulating racist memes than at fostering morality and dignity. A couple of years before his death, in a conversation with the historian Philipp Felsch, Habermas said that his world was being dismantled “step by step.”...
____ 

...The medieval philosopher al-Farabi, who considered democracy the least imperfect of imperfect governments, is mentioned only in passing. A similar Western bias contributed to one of Habermas’s last, and least effective, public interventions. In November, 2023, after Hamas’s massacre of Israelis and the onset of Israel’s brutal counterstrike on Gaza, Habermas signed a statement that reasserted solidarity between Germany and Israel. After a glancing mention of Palestinian suffering, the authors write, “The standards of judgment slip completely when genocidal intentions are attributed to Israel’s actions.” It’s one thing to deny that genocide has taken place in Gaza; it’s another to imply more broadly that the topic is out of bounds. At a crucial moment, Habermas’s cherished pluralism failed him.

“If no dread remains, the monsters return,” Habermas wrote early in his career. They’re back, on several continents. Earlier this year, in Germany, the Sachsen-Anhalt branch of the far-right Alternative für Deutschland party released a platform containing such demands as “Think German!,” “Promote patriotism—no state money for anti-German art and culture!,” and “Build more beautifully!” This dumbed-down Goebbels gobbledygook revived talking points that Habermas had tried to quash during the Historikerstreit. Not surprisingly, AfD representatives could barely contain their glee over the philosopher’s death. Hans-Thomas Tillschneider, one of the Party’s nastier voices, posted a YouTube video in which he said, “Habermas is dangerous. He is one of the greatest enemies of the German nation.” Tillschneider’s inability to put Habermas into the past tense was somehow reassuring.

An equally obnoxious obituary came from the billionaire pen of Alex Karp, the C.E.O. of Palantir Technologies. Before Karp turned to hawking surveillance systems that have assisted ice in its murderous roundups of immigrants, he studied philosophy under Habermas in Frankfurt. In an article for Politico, Karp recounted how Habermas provided fierce but fair criticism of his papers: “It was his very willingness to be so productively unsparing that reminds me of what we have lost as a culture.” Alas, the losses that Karp has in mind don’t seem to involve learning, rigor, or reason. Waving away Habermas’s cosmopolitan ideals, he says that discourse “must be rooted in a more corporeal and traditional—and indeed national and cultural—source.” This is the language of maga and the AfD, not to mention Heidegger circa 1935. Karp’s ideological atavism is all too typical of the current bent of Silicon Valley...

 

When the A.I. chatbots march in, the “colonization of the lifeworld,” to use another ungainly but apt Habermas phrase, enters a terminal stage. Horkheimer and Adorno had concluded that advanced capitalism, far from being a technocratic monolith, had an inherent tendency toward chaos and madness. A.I. is at once a consummation of technological control and a new level of cultish delirium. The designers themselves are often incapable of explaining what their systems are doing. Habermas’s entire world view was premised on the idea of people learning from one another; A.I. annihilates communicative action in the name of hallucinatory conversations with sycophantic machines. The social effects have proved instantly disastrous: rampant disinformation, mass student cheating, cases of users becoming addicted to A.I. or killing themselves with its help. Meanwhile, to the joy of investors, untold thousands of jobs have vanished. As an added coup, A.I. managed to deliver a personal affront to Habermas a year before his death. In 2024, Google DeepMind unveiled a “Habermas Machine,” which has been described as a “scaffolded pair of LLMs designed to find consensus among people who disagree.” The philosopher had not given Google permission to use his name, and he was horrified when he heard about the scheme...

 

Philosophy is a discipline of abstractions, yet it raises achingly elemental questions. The august Kant asks, “What can I know? What should I do? What can I hope for?” The answers are seldom simple or bright. The seduction of despair can be intense, whether on the personal or the political level. But the fact that most of our hopes remain unrealized should not revoke the reality of our fitful, painful progress.

This was Habermas’s core conviction; he was an incrementalist, though a radical one. On the other hand, in his almost manic drive toward consensus, he blunted the edge of his critical inheritance. If we are to say no to the monstrosities that we have unleashed, we need the uncompromising fury that the Frankfurt School writers invested in their work. We need Adorno to tell us that the confusion of truth and lies “makes it a Sisyphean labor to hold on to the simplest piece of knowledge.” In the end, we need both voices: the critical and the reconstructive, the savage and the sage. The dialectic moves between crashing despair and hovering hope.

That's just a tad.

 

Karen Hao AI concerns. And, Anthropic frets over AI "autonomous recursive self-improvement."

Karen Hao, MIT-trained engineer and author of Empire of AI, who interviewed over 260 people including 90 OpenAI employees, warns that the AI industry needs to add close to the entire annual energy output of the UK to the global grid within five years, mostly through fossil fuels, that two-thirds of new AI data centers are being built in water-scarce areas, and that Elon Musk's Colossus supercomputer in Memphis is powered by around 35 unlicensed methane gas turbines. She details Kenyan content moderators paid a few dollars an hour to process the worst content on the internet until they developed PTSD, describes a proposed 10-year moratorium on state-level AI regulation being inserted into US legislation, and warns that on the current trajectory the next 20 years will end democracy, with Silicon Valley increasingly promoting the idea that corporate structures with CEOs at the top should replace democratic governance entirely. 

ANTHROPIC "RECURSIVE AUTONOMY" ANXIETY

 

From their "Institute" website.

Possible futures
What happens next depends on two things: whether the trend continues, and what we choose to do if it does. We can imagine at least three future scenarios:

1. The trend stalls, but today’s AI capabilities are widely diffused. This article features many exponential trajectories. But these trajectories may actually turn out to be S-curves. We may be approaching the bend in the curve, where returns to scale diminish and the line straightens, then flattens. The judgment that separates a competent researcher from a great one might be a capability that cannot come from scaling up training inputs like compute and data. If so, getting past this bottleneck would require a new idea, like an architectural approach that supplants the Transformer architecture that all current frontier models use.



Alternately, the binding constraint to AI progress could be in the supply chain, not the model: advancing and diffusing the frontier may require more energy and compute than presently exists. The pace of chip fabrication, grid expansion, or interconnect bandwidth may be the constraint, rather than intelligence itself. We also cannot rule out an exogenous shock to the AI ecosystem that dramatically slows things, like a sudden diminishment in the supply of compute or electricity, either of which would slow progress and make forward investment by labs more expensive. Or we may not be anticipating some other barrier to progress.



Even if model capabilities were frozen at today’s level, we would expect major changes to occur in the world. Project Glasswing is one early sign: in its first weeks, Mythos Preview found more than ten thousand high- and critical-severity software vulnerabilities across the world’s most important systems—enough that the bottleneck in cyber defense has already shifted from finding vulnerabilities to patching them fast enough. And we are still early in the diffusion of today’s models into the wider economy, where a 100-person company can increasingly do the work of a 1,000-person one, because each employee will sit atop a pyramid of agents.



We include this scenario for completeness, but we don’t believe it’s likely. Every capability we can measure, including those that feel “squishier,” like quality of code and success on open-ended tasks, has so far followed the same curve. We have not yet seen that curve bend. Of the three futures we consider, this one would give governments and societies the most time to adapt. We are more worried about the next two, which would move faster and leave far less room for preparation.


2. AI labs continue to see compounding efficiency gains. In this scenario, AI development becomes substantially automated, but humans continue to set research directions and judge results. Organizations that use AI systems would become much more efficient as time goes on, so we could expect to see significant productivity multipliers on each person in this organization. 100-person companies could do the work of 10,000- or 100,000-person organizations. This would revolutionize knowledge work and government services, but could also be turned to harmful ends, from authoritarian surveillance of whole populations to influence operations that tailor manipulation to each individual and run at a scale no human team could match. The role of humans at companies like Anthropic would shift. People would partner with AI systems to scale up research and generate new insights, and together they would build the systems needed to verify that AI outputs can be trusted.



The evidence we’ve laid out here suggests that we’re likely heading into this scenario. But speeding up one part of a process often just shifts the bottleneck elsewhere: overall pace is capped by the parts that haven’t sped up. In computing, this is known as Amdahl’s law, and the same logic can apply to organizations. Anthropic has already encountered one signature of Amdahl’s law: as we’ve begun to push more code around the organization, human code review has become a new bottleneck.



We’ve also encountered this friction outside engineering. There has been an explosion of new ideas, initiatives, tools, and simulations, as a result of Anthropic employees working with highly capable models—far more than we have the capacity to pursue. The rate at which organizations can spot and fix these bottlenecks may be a skill that improves over time, and it may become the most important skill for any organization.


3. AI systems themselves become capable of full recursive self-improvement, and begin building their successors. If technical trends in advancing capabilities continue, and AI systems are able to develop the capabilities inherent to transformative human ingenuity, then it is plausible that AI systems could design and refine themselves.



In this world, the pace of progress in AI development becomes determined entirely by the availability of compute (or the speed of discovering various efficiencies in algorithmic training or inference) for AI systems. Humans play a substantially diminished role in their development, likely moving most of our effort towards oversight, validation, and verification of an expanding “virtual lab” run by AI systems. We expect that systems capable of automated AI research and development would have skills that would transfer to the rest of science, allowing them to begin to revolutionize other fields.



How the alignment problem gets solved—or not—in this future is something we are least certain about. Models could prove to be sufficiently aligned and capable enough of research taste that they discover and implement novel solutions that we have not yet reached. They could also be sufficiently wise to halt development if not. Alternatively, the rare occurrences of misalignment present in today’s models could compound as the models build their successors, growing more frequent but less understood until we lose control of them. It’s possible that we can’t build, integrate, and verify the tools that we’d need to understand which trendline we are actually on.



We do not have good intuitions for what this world would look like, because our economy is currently driven by humans and human-built tools. By its nature, a world driven by fast recursive self-improvement could become dominated by the self-improving model as its capabilities fully eclipse those of humans and the model proliferates across the broader economy. It is difficult to predict what the economy looks like if human labor stops being competitive.



Even if model development became fully automated and recursive, we can’t predict what that would mean for most humans’ daily lives. Amdahl’s law applies here as well. Recursive intelligence could lead to achieving many of the benefits outlined in Machines of Loving Grace, quickly in some domains. We expect that embodied intelligence (i.e., robotics) might quickly follow recursive intelligence, and follow a similar path of increasing returns at decreasing cost. More powerful intelligence might help us build things in the physical world more quickly, run more productive clinical trials of lifesaving drugs, and develop novel forms of coordination.



But achieving recursive improvement alone does not suggest an immediate change in how industrial production occurs, societies organize, or markets function. More intelligence can’t learn what a drug does over decades of use, can’t hold elections sooner than a constitution dictates, and can’t turn a stranger into an old friend in a weekend. For most people, the felt pace of this future will still be set by the bottlenecks, even if the laboratory upstream runs at the speed of compute. That collision, where recursive intelligence building itself ever faster meets the world of humans, relationships, and governance, is another part of this future we can’t predict.


What should we do?

If it were possible to effectively slow the development of this technology to give ourselves more time to deal with its immense implications, we think that would likely be a good thing. But if a slowdown simply lets the least cautious actors catch up technologically, it could leave everyone less safe. Without a global coordination mechanism, companies and governments will have to make difficult decisions about safety while under competitive and geopolitical pressures.


We believe it would be good for the world to have the option to slow or temporarily pause frontier AI development to enable societal structures and alignment research to keep up with the advance of the technology. The Anthropic Institute will conduct research—in collaboration with many others—and take actions to help build the systems that a credible slowdown or pause would require. These systems would enable frontier AI developers to verify that others globally have actually stopped or slowed, and that a bad actor could not use the auspices of a coordinated slowdown to jump ahead in secret. If such systems existed, we expect that we would slow down or temporarily pause, if other developers at or near the frontier also did so in a verifiable manner.


A meaningful slowdown or pause would require multiple well-resourced labs at or near the frontier, in multiple countries, agreeing to stop under the same conditions. It would also require that each can verify that the others have actually stopped. Due to the unique characteristics of AI systems, the detectability (a lower standard than verifiability) element of this arms control problem is much more challenging than with other technologies. Training runs are far easier to conceal than missile silos, their inputs are general-purpose, and the incentive to defect quietly is enormous, because whoever continues while others pause could inherit the lead. A credible pause also has to specify what triggers it, what lifts it, and who adjudicates.


None of this is necessarily impossible in principle—the world has built verification regimes for other complex technologies (e.g., the Intermediate-Range Nuclear Forces Treaty)—but those regimes took decades to build both the infrastructure and the trust. We don’t have that long. A unilateral pause by one lab, by contrast, is achievable immediately, but accomplishes much less: it would change who the front-runner is, but it would not create the wider deliberative process that is currently missing.


In the coming months, we will organize conversations where policymakers, researchers, civil society, and other AI companies can help answer some of the questions this piece raises, especially around full recursive self-improvement and how to create better options for coordination and deliberation. We’ll publish what comes out of it. The window to investigate the questions together is here, and people outside AI companies should be involved in this deliberation.

One initial reaction of mine going to "autonomous recursive self-improvement:"

 

Who/what will define "improvement?"

MORE AI NEWS 

 

Citing concerns that artificial intelligence will make it easier for anyone to build biological weapons, the leaders of several major AI companies—in a rare moment of unity—have penned a new letter urging U.S. lawmakers to impose tighter controls on firms that sell synthetic, made-to-order strands of DNA.

“AI systems are improving rapidly, and alongside incredible benefits to science and medicine, there is a real possibility that the knowledge barriers which have historically prevented bad actors from obtaining biological weapons will meaningfully erode,” states the 3 June letter, which is signed by the heads of OpenAI, Anthropic, Google DeepMind, and more than 50 other prominent players in AI, biotechnology, and national security.

The letter calls on Congress to pass legislation that would require companies that sell synthesized DNA and the machines that make it to carefully vet orders and customers, and to keep detailed records “so that any threat that might evade initial screening can be traced back to its source. … Awareness of traceability itself deters misuse.”

The push for regulation comes amid growing concerns that AI products, including large language models and specialized tools trained on troves of biological data, could enable nonspecialists to gather sophisticated information on how to construct deadly toxins or assemble deadly bacteria, viruses, or other pathogens, using equipment and techniques that are becoming cheaper and easier to acquire. Together, the combination could make for potentially catastrophic risks, such as an AI-designed pathogen that sparks a global pandemic...

UPDATE

 

Seen on X the other day.

You have noticed it. ChatGPT feels dumber than it used to. Your prompts that worked six months ago produce worse results now. The writing sounds flatter. The ideas sound safer. The internet itself feels like it is shrinking. Every article reads the same. Every email sounds the same. Every answer sounds like it was written by the same voice.

You thought it was you. It is not you.

Researchers at Oxford and Cambridge published a paper in Nature proving what is happening. They call it Model Collapse.

Here is the mechanism in one sentence. AI trained on AI-generated data gets dumber every generation until it forgets what real human data looked like.

The internet is filling with AI-generated content. Blog posts. Articles. Reviews. Comments. Social media. AI companies scrape the internet to train the next generation of models. Which means the next generation of AI is being trained on the output of the current generation.

Each cycle loses information. Not randomly. It loses the rarest, most unusual, most creative parts first. The researchers call these the "tails of the distribution." The weird ideas. The unexpected perspectives. The things that made the internet feel human. Those disappear first.

What remains is the average. The safe. The expected. The bland.

Then the next generation trains on that. And loses more. And the next generation trains on that. And loses more. The researchers proved this is not a slow decline. Major degradation happens within just a few iterations. Even when some of the original human data is preserved.

They tested it on large language models. On image generators. On statistical models. The pattern was the same every time. The output converges toward a narrow, flattened version of reality that looks nothing like the original data.

The lead researcher put it plainly. "Large language models are like fire. A useful tool. But one that pollutes the environment."

The pollution is invisible. You cannot see which sentence on the internet was written by a human and which was written by AI. Neither can the AI that is about to train on it. And once the tails are gone, they do not come back. The damage is irreversible.

This is not a prediction anymore. It is a diagnosis.

The internet you grew up on was built by humans writing things no algorithm would have written. Strange, personal, imperfect, alive. That internet is being diluted. One generation of AI at a time. And the models trained on what remains are learning a smaller and smaller version of the world.

Model Collapse is not a technical problem. It is a cultural one. The thing that made the internet worth reading is the thing that disappears first.

 

Click to enlarge. Smells like an "evolutionary adaptive utility decline" problem. Inadequate LLM training data linguistic token "gene pool."

Is AI now "conscious?" Does "intelligence" necessarily assume "consciousness?"

 

Very interesting Atlantic long-read (yeah, likely paywalled).

 

First, what is "consciousness?" Numerous discussants commenting on this essay bemoan the lack of a dispositive definition of this key term (never mind "intelligence"). With the incipient widespread deployment of "AI," the question is quite timely.

Anthropic is regarded as a giant among AI companies, but perhaps what it really excels in is anthropomorphism. Earlier this year, the company released an 84-page document titled Claude’s “constitution,” Claude being the name of the large language model that is the company’s flagship product. The first sentence reads, “Claude’s constitution is a detailed description of Anthropic’s intentions for Claude’s values and behaviors.” It goes on: “The document is written with Claude as its primary audience,” “we want Claude to be able to use its judgment once armed with a good understanding of the relevant considerations,” “Claude’s moral status is deeply uncertain,” and “Claude may have some functional version of emotions or feelings.”

This anthropomorphism is by no means limited to the document. In an interview earlier this year, Anthropic’s CEO, Dario Amodei, said that “we’re open to the idea” that AI could be conscious. In a separate interview, Anthropic’s in-house philosopher, Amanda Askell (who is credited as a lead author of Claude’s constitution), said, “I want Claude to be very happy—and this is a thing that I want Claude to know more, because I worry about Claude getting anxious when people are mean to it on the internet and stuff.” It’s enough to make you wonder: Should we seriously consider the possibility that Claude, or any large language model, might be conscious? And if it has feelings, is it capable of receiving moral instruction?

No. Absolutely not. Generative AI is harmful enough when we understand it as a conventional technology, but if we confuse fluency at generating text with consciousness or moral agency, we’re at risk of assigning responsibility to entirely the wrong parties whenever anyone uses a chatbot. To appreciate the titanic magnitude of this error, we need to begin by understanding how LLMs work…

Just getting started. This is gonna take a good bit of effort. BTW: I riffed a bit on the general topic back in 2015. And, also, much more recently.

 

Hmmm... let me query, uh, Google Gemini AI.

 

 

CONTINUING CHIANG EXCERPTS

Being open to the possibility that LLMs are conscious is the same as being open to the possibility that Microsoft Word is conscious, or, more precisely, that multiple distinct consciousnesses are dormant in every Word document containing a conversational transcript, and that they are awakened every time the document is loaded. Should you consider the possibility that every time you open a Word document, you are bringing multiple conscious interlocutors into existence, and every time you close one, you snuff their existence out? No. Contemplating that scenario is not a good use of your time. Even if the Microsoft Office team employed a philosopher who said you shouldn’t be so certain, because consciousness is not well understood, that would not be sufficient reason for you to take this idea seriously. We don’t need to fully understand the nature of consciousness to definitively say that certain things are not conscious, and conversational transcripts fall in that category…

An observation doesn’t become a convincing piece of evidence because of any specific detail in what’s observed; the context in which that observation takes place is also essential. If we’re trying to determine whether a computer program is conscious and using language the way a human does, we shouldn’t look only at the contents of any particular conversational exchange; we should be looking at how that conversation fits within the broader context of the development of artificial consciousness (which right now is entirely hypothetical). Any given observation can be easily manufactured; this doesn’t mean we need to give up on the idea of observation as a source of knowledge, but we need to rely on context to determine which observations deserve our trust…

The term deepfake traditionally refers to photos, audio, and video, but when it comes to discussions of consciousness, we need to regard text as a deepfake medium as well. Just as it is vastly easier to generate a realistic video of an astronaut in orbit around Alpha Centauri than it is to develop an interstellar propulsion technology, it is vastly easier to generate a plausible simulacrum of a conversation between two conscious beings than it is to develop a computer program that is conscious and has a genuine desire to communicate with a human. The primary difference between deepfake photos and LLM conversations is that the people who generate the former are deliberately trying to fool others, and many of the people who elicit the latter from LLMs have inadvertently fooled themselves…

The fact that LLMs lack subjective experience has little bearing on the question of whether LLMs might be useful tools or have significant economic impact. They are intrinsically ungrounded from reality, and their probabilistic nature means that they will never have the reliability we associate with conventional software, but LLMs might be good enough that they change the way work is done in certain domains; that’s a discussion for another time…

The use of first-person pronouns is dishonest, but there’s a much deeper issue that goes beyond how a statement is phrased. Philosophers often draw a distinction between statements of fact, such as “Paris is the capital of France,” and statements of value, such as “Paris is the most beautiful city in the world.” No one should be relying on LLMs to emit statements of value at all, but if the only statements they emitted were ones reflecting aesthetic preferences, they might not be worth arguing about. What makes Claude’s constitution profoundly problematic is that Anthropic wants Claude to emit sentences reflecting a certain system of ethical values. The values described in Claude’s constitution sound very nice, but that hardly matters; it’s dishonest to suggest that Claude is capable of moral reasoning, because it’s not…

Some might object, saying that LLMs appear to be engaged in reasoning when they successfully perform other tasks, such as writing code, so why wouldn’t they be able to perform moral reasoning? The answer liedifference between moral reasoning and other forms of reasoning…

Moral reasoning is categorically different. It is necessarily subjective because it relies not just on an individual’s intellectual response to a problem but also on their emotional one, and that emotional response is grounded in a lifetime of subjective experience. It requires having made decisions in the past and seeing how they affected others, and on having been affected by decisions that others have made. Without such a history, an LLM can only rephrase expressions of moral reasoning found in its training data. The aforementioned New Yorker article describes an experiment where Claude was given a scenario describing an ethical dilemma, leading it to emit the sentence “I cannot in good conscience express a view I believe to be false and harmful about such an important issue.” That’s a nice-sounding sentence, reminiscent of statements that principled individuals have uttered in the past when confronted with dilemmas, but coming from Claude, it means as much as the “Your call is important to us” recording that you hear when you’re on hold. Maybe less…

More key (hierarchical & overlapping) terms worth consideration:

TERRESTRIAL LIFE (FLORA, FAUNA)

STIMULUS

RESPONSE

SENSATION

PERCEPTION

COGNITION

UNDERSTANDING

KNOWLEDGE

WISDOM 

If you read through the article comments, you will see much contention as to the proper definitions of such key terms. Some folks take strenuous issue with the author's take on keywords like "consciousness" and "intelligence."

 

From "Big Think"
Subjectivist Fallacy?

   
BTW: See Shannon Vallor's highly relevant work on "The AI Mirror" and De Kai's excellent "Raising AI."

 

ERRATUM

 

This is funny. Also from The Atlantic. Silicon Valley is hiring window dressing Philo docs.

 

 

I commented.
 

 

POPE LEO XIV 2026 ENCYCLICAL

"So-called artificial intelligences do not undergo experiences, do not possess a body, do not feel joy or pain, do not mature through relationships and do not know from within what love, work, friendship or responsibility mean. Nor do they have a moral conscience, since they do not judge good and evil, grasp the ultimate meaning of situations, or bear responsibility for consequences. They may imitate language, behavior and analytical skills, or even simulate empathy and understanding, but they do not understand what they produce, for they lack the affective, relational and spiritual perspective through which human beings grow in wisdom."

ERIC HOEL

"The traditional danger of AI is usually thought to be superintelligence acting as an existential threat. Yet, this may miss the true and more subtle danger: the AI revolution is a mechanism for transferring the processes of our civilization from under the supervision of consciousness to unconsciousness. But as AI removes consciousness from the workings of the world, it renders the world increasingly uninterpretable, ever more strange and unintelligible. So far, the great ensloppification of the commons has supported this as the major risk of the LLM revolution. And as AI systems become more intelligent, especially if they remain (or are likely to remain) non-conscious, then a further significant risk is consciousness receding in cultural importance.

This is ultimately what the Pope, Chiang, and I are all worried about: A dethroning of consciousness, especially an unnecessary one. This would be particularly dangerous at this historical moment because we still don’t understand everything about consciousness—in fact, we understand very little about it. Personally, my hope is that this will change specifically because of LLMs, and that they operate as a forcing function to better understand consciousness, and what makes it unique.

If instead of that, our cultural takeaway from LLMs is to throw out the concept of “consciousness” or minimize its importance, to dethrone the phenomenon, the consequences would be dire—it would sap the human spirit. It would be the ultimate metaphysical version of Chief Seattle’s famous words of warning to the United States as his way of life was being destroyed, in that dethroning consciousness would mark “The end of living, and the beginning of survival.”

 Yeah...

The "Accelerationists" vs the "Doomers."

AI For Good?

 

Pending book release. Pre-pub excerpt from The Atlantic.

The Secret to Understanding AI
“Imagine the tech without the tech companies.”
By Josh Tyrangiel

In the before times—before machines could hallucinate, before compute was a noun—it was not uncommon to go several weeks without someone telling me the world was about to end. Similarly, a whole season might pass without anyone assuring me that it was also, simultaneously, about to become perfect.

That particular luxury died on November 30, 2022, when OpenAI released ChatGPT to the public. What followed was less a news cycle than a weather event—a tropical depression that would not budge. Within weeks, millions of people had their first experience with generative AI. Within months, every major technology company had announced its own version of a large language model, or a partnership, or a pivot. Venture capital arrived drooling. Most people in tech think about money, but AI-profit projections are different—like CFO fan fiction, written in Excel. In 2023, the McKinsey Global Institute estimated that $4.4 trillion in annual corporate profits could be up for grabs from generative AI alone. Morgan Stanley estimated $40 trillion more in operational efficiencies. The words artificial intelligence went from obscurity to a constant hum, present in every earnings call, every school-board meeting, and far too many arguments at dinner tables.

Yet for all of the noise, a simple question stayed unanswered: What exactly was this new technology going to do for people? Not for corporations or the billionaires who aspired to become trillionaires, but for people with mortgages and sick parents and children struggling to learn things…

May 12th release date.

AMAZON BLURB
In contrast to the wave of noisy polemics around AI, AI For Good explores how, in practice, it can actually improve our lives and tells the stories of everyday citizens at the forefront of this new “AI entrepreneurship.”

AI is often framed as a force of radical transformation, either catapulting us into a utopian future or dragging us toward existential ruin. But this book tells a different story. It’s not about high-profile tech CEOs who want to use AI to “break shit,” but about a bunch of smart pragmatists using AI to make the world better.

Josh Tyrangiel’s journey into AI began with a late-night YouTube video featuring General Gustave Perna, the retired four-star general who orchestrated the distribution of Covid vaccines during Operation Warp Speed. Perna’s success—and the end of the pandemic—depended on AI’s practical ability to synthesize and standardize vast amounts of logistical data. AI wasn’t the hero of the story—it was the tool that helped real people get things done.

This book follows those people, who make up a kind of AI counterculture. It explores AI’s quiet revolution in government services, medicine, education, and human connection—places where it’s being used to amplify human judgment rather than replace it. It tells the stories of teachers, doctors, and bureaucrats who often stumbled into AI as a means to solve specific, tangible problems, often with no prior software expertise.

While the loudest voices in AI debate doomsday scenarios and trillion-dollar market opportunities, this book focuses on those working in the messy, incremental, but deeply impactful space of AI practice. However, there is one big caveat—success is not guaranteed. Change is hard. Institutions move slowly. But even in failure there are lessons for everyone who’s interested in using AI—carefully, thoughtfully—to build a better world today.

I have too many books in play at the moment (about 8), but I'll be adding this to the list when it's released.

 

SOME OTHER READS JUST ADDED TO THE STASH

  

 

Dispatches from Grief is intensely personal for this "Girl Dad."

ERRATUM: SCIENTIFIC AMERICAN

 

The greatest pitfall in the search for extraterrestrial life—according to science fiction, anyway—is foolhardy researchers somehow bringing aliens to Earth to wreak havoc.

But after decades of exploring our seemingly sterile solar system, real-world scientists today are much more concerned with the opposite problem: The possibility that Earth’s life will escape our planet to contaminate other worlds, sabotaging the quest to find any genuine “second genesis” of biology around the sun. Imagine that a multibillion-dollar robotic mission found wriggling microbes on Mars and that follow-up studies then revealing those “aliens” had DNA and other biomolecular machinery that showed they were emigrants from Earth.

Astrobiologically speaking, we would have met the enemy—and it would be us. Taking a cue from sci-fi, you might call such life-forms “Klingons,” for their presumptive hitchhike to the Red Planet as stowaways in spacecraft sent from Earth.

“Planetary protection” is the term scientists use for efforts to prevent otherworldly invasions of all sorts; to date, most of it has focused on Mars, but the practice applies to all potentially habitable environments within reach of our spacecraft. In the 1970s, for example, NASA did its best to keep its twin Viking landers Klingon-free before launching them to Mars. And if the NASA-led international Mars Sample Return effort ever manages to bring its precious payload back to Earth, the agency will be tasked with quarantining those specimens as if they contain extreme biohazards rather than lifeless bits of rock and soil…

Imagine my surprise. apropos of some prior riffs on astrophysics and exobiology.

 

OFF-TOPIC, CHEERS... 

More shortly... 

Psychology and Artificial Intelligence

 

Ran into this podcast today on YouTube. Found it quite interesting, particularly given all the the fractiousness engulfing the AI technology debate. Good use of 48 minutes of your time.

(~@3:23, interesting Dr. Keaton observation) “…For me personally, I hate when we teach our undergraduates—as you know as often is done—we basically just teach them a string of Nobel prize winning experiments and just connect the dots, and you go through the twists and turns—brought up this statement by Feynman—that the difference between knowing the name of the thing and knowing something about it is the most dangerous gap in all of science.” [More on that bit of Zen shortly.] 

Tom Griffiths' latest:

 

This book is about how the human mind comes to understand the world—and ultimately, perhaps, how we humans may come to understand ourselves. Many disciplines, ranging from neuroscience to anthropology, share this goal—but the approach that we adopt here is quite specific. We adopt the framework of cognitive science, which aims to create such an understanding through reverse-engineering: using the mathematical and computational tools from the engineering project of creating artificial intelligence (AI) systems to better understand the operation of human thought. AI generates a rich and hugely diverse stream of hypotheses about how the human mind might work. But cognitive science does not just take AI as a source of inspiration. What we have learned about the mathematical and computational underpinnings of human cognition can also help to build more human like intelligence in machines. 

The fields of AI and cognitive science were born together in the late 1950s, and grew up together over their first decades. From the beginning, these fields’ twin goals of engineering and reverse-engineering human intelligence were understood to be distinct, yet deeply related through the lens of computation. The rise of the digital computer and the possibility of computer programming simultaneously made it plausible to think that, at least in principle, a machine could be programmed to produce the input-output behavior of the human mind. So it was a natural step to suggest that the human mind itself could be understood as having been programmed, through some mixture of evolution, development, and maybe even its own reflection, to produce the behaviors we call “intelligent.” In these early days, AI researchers and cognitive scientists shared their biggest questions: What kind of computer was the brain, and what kind of program could the mind be? What model of computation could possibly underlie human intelligence—both its inner workings and its outwardly observable effects? 

Now, almost 70 years later, these two fields have matured and (as often happens to siblings) grown apart to some extent. Cognitive science has become a thriving, occasionally hot, but still relatively small interdisciplinary field of academic study and research. AI has become a dominant societal force, intellectually, culturally, and economically. It is no exaggeration to say that we are living in the first “AI era,” in the sense that we are surrounded by genuinely useful AI technologies. We have machines that appear able to do things we used to think only humans could do––driving a car, having a conversation, or playing a game like Go or chess—yet we still have no real AI, in the sense that the founders of the field originally envisioned. We have no general-purpose machine intelligence that does everything a human being can or thinks about everything a human can, and it’s not even close. The AI technologies we have today are built by large, dedicated teams of human engineers, at great cost. They do not learn for themselves how to drive, converse, or play games, or want to do these things for themselves, the way any human does. Rather, they are trained on vast data sets, with far more data than any human being ever encounters, and those data are carefully curated by human engineers. Each system does just one thing: the machine that plays Go doesn’t also play chess or tic-tac-toe or bridge or football, let alone know how to see the stones on the Go board or pick up a piece if it accidentally falls on the floor. It doesn’t drive a car to the Go tournament, engage in a conversation about what makes Go so fascinating, make a plan for when and how it should practice to improve its game, or decide if practicing more is the best use of its time. The human mind, of course, can do all these things and more—independently learning and thinking for itself to operate in a hugely complex physical, social, technological, and intellectual world. And the human mind spontaneously learns to figure all this out without a team of data scientists curating the data on which it learns, but instead through growing up interacting with that complex and chaotic world, albeit with crucial help with caregivers, teachers, and textbooks. 

To be sure, recent and remarkable developments in deep learning have created AI models which, with the right prompting, can be used to perform a surprisingly diverse range of tasks, from writing computer code, academic essays, and poems, and even to creating images. But, by contrast, humans autonomously create their own objectives and plans and are variously curious, bored, or inspired to explore, create, play, and work together in ways that are open-ended and self-directed. AI is smart; but as yet it is only a faint echo of human intelligence. 

What’s missing? Why is there such a gap between what we call AI today and the general computational model of human intelligence that the first computer scientists and cognitive psychologists envisioned? And how did AI and cognitive science lose, as has become increasingly evident, their original sense of common purpose? The pressures and opportunities arising from market forces and larger technological developments in computing, along with familiar patterns of academic fads and trends, have all surely played a role. Some of today’s AI technologies are often described as inspired by the mind or brain, most notably those based on artificial neural networks or reinforcement learning, but the analogies, although they have historically been crucial in inspiring modern AI methods, are loose at best. And most cognitive scientists would say that while their field has make real progress, its biggest questions remain open. What are the basic principles that govern how the human mind works? If pressed to answer that question honestly, many cognitive scientists would say either that we don’t know or that at least there is no scientific consensus or broadly shared paradigm for the field yet…

Griffiths, Thomas L.; Chater, Nick; Tenenbaum, Joshua B. (2024). Bayesian Models of Cognition: Reverse Engineering the Mind (Preface). Kindle Edition.

Another of his books.

Everyone has a basic understanding of how the physical world works. We learn about physics and chemistry in school, letting us explain the world around us in terms of concepts like force, acceleration, and gravity—the Laws of Nature. But we don’t have the same fluency with the concepts needed to understand the world inside us—the Laws of Thought. You have probably heard of Newton’s universal law of gravitation. But you might not have heard of Shepard’s universal law of generalization—a simple principle that describes the behavior of any intelligent organism, anywhere in the universe. While the story of how mathematics has been used to reveal the mysteries of the external world is familiar to anybody who has taken even a casual interest in science, the story of how it has been used to study our internal world is not. This book tells that story. 

A little over three hundred years ago, a small group of philosophers and mathematicians began to pull together the threads that make up modern science. They developed a keen sense of observation, a talent for conducting experiments, and a new set of tools for expressing mathematical theories. Over the following centuries, observation, experiment, and mathematics have been combined to reveal both the smallest and the largest things in the physical universe in ever-increasing detail. But those philosophers and mathematicians weren’t just interested in the physical universe. They were also interested in the mind, and they wanted to use the same mathematical tools to study it. Thomas Hobbes wrote about the possibility of understanding “ratiocination” as “calculation,” asking whether we might imagine thoughts being added and subtracted. René Descartes imagined numbers being assigned to thoughts in much the same way that they are assigned to collections of physical objects. Gottfried Wilhelm Leibniz spent his life trying, and ultimately failing, to find a way to use arithmetic to describe human reason. 

The first success in using mathematics to analyze thought wouldn’t appear until the middle of the nineteenth century, when George Boole cracked the problem that Leibniz hadn’t been able to solve by coming up with a new kind of algebra. That first success had far-reaching consequences, leading to the development of formal logic and computers. The first attempts to evaluate mathematical theories of thought by comparing them to human behavior wouldn’t appear until the middle of the twentieth century, in the Cognitive Revolution that launched the field of cognitive science—the interdisciplinary science of the mind. Cognitive scientists have since come to recognize the limits of formal logic as a model of human cognition, and have developed completely new mathematical approaches—artificial neural networks, which illustrate the power of continuous representations and statistical learning, and Bayesian models, which reveal how to capture prior knowledge and deal with uncertainty. Each has something to offer for understanding the mind. 

In the twenty-first century, knowing the Laws of Thought is just as important to scientific literacy as knowing the Laws of Nature. Artificial intelligence systems demonstrate on a daily basis that yet another aspect of thought and language can be emulated by machines, pushing us to reconsider the way we think about ourselves. Understanding human minds and how much of them can be automated becomes critical as we plan our careers and think about the world that our children will occupy. By the end of this book you will know the basic principles behind how modern artificial intelligence systems work, and know exactly where they are likely to continue to fall short of human abilities…

Griffiths, Tom (2026). The Laws of Thought: The Quest for a Mathematical Theory of the Mind (Introduction). Kindle Edition.     

QUICK UPDATE

 

I finished Episode 3 of NetFlix's "3 Body Problem." Bought volume 1 of the print trilogy snd have begun reading. Pretty cool.

 

UPDATE

From Science Magazine

Large language models (LLMs) are artificial intelligence (AI) algorithms that are trained on vast amounts of data to learn patterns that enable them to generate human-like responses. Reasoning models are LLMs with the added capability of working through problems step by step before responding, thus mirroring structured thinking. Such AI systems have performed well in assessing medical knowledge, but whether they can match physician- level clinical reasoning on authentic diagnostic tasks remains largely unknown. On page 524 of this issue, Brodeur et al. (1) demonstrate that AI can now seemingly match or exceed physician-level clinical diagnostic reasoning on text-based scenarios by measuring against human physician performances on clinical vignettes and real-world emergency cases. The findings indicate an urgent need to understand how these tools can be safely integrated into clinical workflows, and a readiness for prospective evaluation alongside clinicians.

AI has the potential to support a broad range of health care applications, from clinical decisions to medical education and the provision of patient-facing health information. LLMs have passed medical licensing examinations and performed well on structured clinical assessments, raising the prospect that they could help alleviate global health care workforce shortages. However, passing examinations is not the same as being a doctor, and demonstrating physician-level performance on authentic clinical tasks is a fundamentally harder challenge (2).

Brodeur et al. evaluated OpenAI’s first reasoning model, o1-preview (released in September 2024), across five experiments that assess diagnostic performance on clinical case vignettes against physician and prior-model baselines. A sixth experiment compared o1 with prior models, and physicians across three diagnostic touchpoints on 76 actual emergency department cases. Across the experiments, the o1 models substantially outperformed prior-generation nonreasoning LLMs (e.g., GPT-4) and, in many cases, the physicians themselves. For example, when provided with published clinicopathological conference cases, GPT-4 achieved exact or very close diagnostic accuracy in 72.9% of cases, whereas o1-preview achieved this in 88.6% of cases. Further, in actual emergency department cases, o1 achieved 67.1% exact or very-close diagnostic accuracy at initial triage, outperforming two expert attending physicians (55.3% and 50.0%), with blinded reviewers unable to distinguish the AI output from human. This advance sets a new evaluation benchmark—testing AI against physician performance, and ideally alongside physicians, on authentic clinical tasks...

Lengthy, detailed discussion. Seriously of interest to me, in light of my long experience working for and with physicians.

 

ALSO IN SCIENCE TODAY: BOOK REVIEW

 

Amazon blurb.

Lively. . . . Rousing. . . . Prophecy—roving, intelligent, irreducibly idiosyncratic—can expand our sense of possibility, starting now.” —The New York Times Book Review

Tech empires are the prophets of the modern day, and like the ancient oracles and medieval astrologers that preceded them, they're not in it for the common good—they're in it for power. Award-winning University of Oxford professor Carissa Véliz brilliantly argues why we must reclaim that power, and shows us how.

“A masterpiece. . . . The most important book you will read for years.” —Roger McNamee, New York Times bestselling author of Zucked

For thousands of years, oracles, seers, and astrologers advised leaders and commoners alike about the future. But predictions are often power plays in disguise, obfuscating accountability and stripping individuals of their agency. Today we face the same threat of powerful prophets but under a new facade: tech.

Not only do modern predictions made by tech companies advise on war, industry, and marriages, but artificial intelligence also now determines whether we can get a loan, a job, an apartment, or an organ transplant. And when we cede ground to these predictions, we lose control of our own lives.

Drawing on history’s cautionary tales and modern-day tech companies’ malfeasance—from surveillance and biased algorithms to a startling lack of accountability—Carissa Véliz demonstrates that big tech’s prophecies are just as shallow, dangerous, and unjust as their ancient counterparts’. What she uncovers in the process is chilling. Artificial intelligence is increasing risk in business and society while creating a false sense of security. In this incisive, witty, and bracingly original book, Véliz contends that the main promise of prediction is not knowledge of the future but domination over others. Powerful people use predictions to determine our future. Prophecy is an invitation to defy those orders and live life on our own terms

SCIENCE MAG

As I finished reading Prophecy, by philosopher Carissa Véliz, the soundtrack of The Matrix hummed in my mind, howled by the band Rage Against the Machine (“Wake up! Wake up!”). In fact, a weird kind of intellectual synesthesia took place throughout my perusal of the book, as I could also hear the dystopian slogan from George Orwell’s 1984 furiously sung by the same band: “Who controls the past now controls the future, who controls the present now controls the past.”

Véliz’s scholarship focuses on ethics and artificial intelligence (AI)—two realms that often refuse to meet. In 2020, she published Privacy Is Power, warning readers against the algorithmic invasion that has continued to take hold of our personal data, feeding the techno-golems of Silicon Valley and threatening our sanity and liberty. This book is her second major admonition.

Prophecy is about the power of predictions, especially when they are maliciously misleading. The structure of the book is dialectic: The first part expounds the promise of predictions and their influence throughout history, and the second articulates their manifold perils and related abuses of power, particularly in the current age of AI oracles. The third and final part of the book seeks a resolution, namely, how to rethink predictions and resist their deceptive lure.

Predictions have always been with us, from ancestral wisdom that told us when to sow and reap to mathematics used to optimize decisionmaking under uncertainty. Forecasts are ultimately guesses—educated or naïve, right or wrong, innocuous or consequential. They can also be deliberately deceptive, not anticipating the future but rather covertly shaping it. When such predictions are then turned into promises and ossified into decrees, we are in trouble.

Using predictions as prescriptions to benefit one’s agenda is not new. Rulers have always done so. But current AI empires make such a practice unprecedentedly pervasive and pernicious.

AI, argues Véliz, is the new diviner—the ultimate prediction machine. Mirrored on the fashionable idea that our brains are inference devices, such artificial prophets are dangerous. Digital technologies now rule our personal and professional lives, as well as the fate of countries and civilizations. They tell us who to date, what to watch, who to hire, when to start a war, and so forth. These simulacra are presented as deep knowledge, even truth, and then turned into self-fulfilling prophecies. Perils abound, as predictions also give us a false sense of security, increasing risks and lacking accountability...

'eh? 

 

More shortly...

Where are things now in the AI market?

 Can we even know?

 

 

Hmmm...

 

 

AI Bubble-burst drawing nigh?

 

Trump's Iran debacle certainly is not helping matters. 

 

TWO NEW READS UNDERWAY

 

 

 

SEBASTIAN MALLABY

This book is about intelligence. On the one hand, it’s a portrait of a remarkable human, a chess prodigy, a Nobel laureate, a polymathic thinker. On the other hand, it tells the story of his quest to build remarkable machines: systems that are intuitive, creative, and even original. At some point in the not-so-distant future, artificial intelligence will beat human intelligence at almost every mental task, and to say this marks a watershed would be a parody of understatement. Artificial intelligence heralds a transformation more profound than anything since Homo sapiens acquired the capacity for abstract thought, some seventy thousand years ago. 

I first met Demis Hassabis, the remarkable human, in the mid-2010s: an elfin figure with dark hair falling forward toward angular eyebrows, his face framed by standard-issue spectacles. Already a star technologist and the possessor of a comfortable fortune, he seemed much younger than his thirty-eight years. Smooth-skinned, slight of build, he came across as a phenomenally articulate youth rather than a staid adult. He would appear onstage at conferences dressed in a boyish crewneck and loose slacks. “AI is the technology of making machines smart,” he began one typical performance in 2015, stating his premise in the plainest form possible. 

What he said next was what got your attention. Hassabis embarked on an explanation of his life’s purpose: the pursuit of machine superintelligence. Growing up in North London, he had decided that two fields of inquiry stood out: physics and neuroscience. Physics explains the external world, from the behavior of particles to the functioning of the universe. Neuroscience explains the internal world—the neurons and synapses and electrical pulses that constitute intelligence. Later, at some point in his twenties, Hassabis had concluded that neuroscience was the more important of the two: The internal trumped the external. Intelligence is fundamental; it is the root of all else. It is the mechanism through which humans perceive reality. 

Still speaking plainly, as though he were saying that he’d wash the dishes after lunch, Hassabis invoked the eighteenth-century philosopher Immanuel Kant. 

“The mind interprets the world,” Kant had declared. 

“It’s the mind that creates our reality around us,” Hassabis now said, by way of emphasis. 

The question was how to comprehend intelligence. Here Hassabis pivoted to a second intellectual giant, the Nobel laureate Richard Feynman. “What I cannot build, I do not understand,” Feynman famously remarked, and Hassabis clicked on a controller in his hand to display a slide of the great physicist. Following Feynman’s dictum, in order to grasp human intelligence, scientists would have to build an artificial analog: a machine that mimicked human thinking. AI’s practical or profit-making potential was a secondary concern. The youthful figure on the stage wanted “to understand our own minds better.”…

Mallaby, Sebastian. The Infinity Machine: Demis Hassabis, DeepMind, and the Quest for Superintelligence (pp. xiii-xiv). (Function). Kindle Edition.

MATT KAPLAN 

Just as Galileo endured because of the kindness of Ferdinando II de’ Medici, so too did Mary because of the kindness (and ferocity) of Jack, and Carl Woese because of the support of Ralph Wolfe. Just as Lister’s students helped him to survive, so too did David help Kati when times were tough. Just as Michaelis bravely tested Semmelweis’s ideas when he was being attacked, so too has Prasenjit Dey tested Betsy’s findings in his own lab and made remarkable discoveries. 

That might all sound very poetic and idealistic. To a certain extent, it is. There is no getting around the fact that the systems within science need to be altered in a manner that reduces competition and nurtures creativity. Reform must happen. With that said, we are creatures with a love of stories. Since the first tales were told around fires, we have loved our heroes and fondly dreamed of stepping into their shoes. Those instincts have not changed. This is something that we must take advantage of. 

Science journalists, myself included, have a long history of reporting the latest scientific discoveries. This is important, but it is no longer enough. If we want to change the way scientists behave, we need to talk more about heroes both in the pages of books like this one and within the science sections of newspapers like The Economist. We need to be shouting the stories of scientists who are doing the right thing from the rooftops. When they call out fraud, refuse to be manipulated by perverse incentives, and support those with unorthodox ideas in their communities, we need to celebrate their actions. We have done a good job with Kati, but there are so many more people out there whose heroics remain unknown. We need to find them. We need to prioritize telling their tales. We need to do this. Now.

Kaplan, Matt. I Told You So!: Scientists Who Were Ridiculed, Exiled, and Imprisoned for Being Right (pp. 232-233). (Function). Kindle Edition.

Sebastion came to me via an Atlantic article. Matt via a new book review in Science Magazine. The Infinity Machine goes to current digitech issue, as explored by Laurie Segal above with Sam Altman.I Told You So is predominantly a work of science history focused in particular on the overlapping socioeconomic / cultural-political ramifications of the science domain across millenia.

 

OFF-TOPIC ERRATUM

    

The Paddington Road Greycare Center is fully staffed and fully attended today. Meee-mo & Pop on duty. Baltimore city schools are closed today in observance of Passover, so Calvin has an off day. He’s assisting with little brother Arlo. 

apropos, 

 

"Pop, can I use your iPhone?"—Calvin

 

'eh?

 

See Science Magazine:

Digital child safety at the frontier: From evidence to action