The 2019 AI Now Institute Report

An important read, and not just for us techie people.

Click the cover image for the full PDF

Forty pages of endnotes. Kudos.

See my prior post "Ethical artificial intelligence?"

Stay tuned. A busy topical week.

UPDATE: AI REPORT HEALTH TECH SECTION

2.6 Health
AI technologies today mediate people’s experiences of health in many ways: from popular consumer-based technologies like Fitbits and the Apple Watch, to automated diagnostic support systems in hospitals, to the use of predictive analytics on social-media platforms to predict self-harming behaviors. AI also plays a role in how health insurance companies generate health-risk scores and in the ways government agencies and healthcare organizations allocate medical resources.

Much of this activity comes with the aim of improving people’s health and well-being through increased personalization of health, new forms of engagement, and clinical efficiency, popularly characterizing AI in health as an example of “AI for good” and an opportunity to tackle global health challenges. This appeals to concerns about information complexities of biomedicine, population-based health needs, and the rising costs of healthcare. However, as AI technologies have rapidly moved from controlled lab environments into real-life health contexts, new social concerns are also fast emerging.

The Expanding Scale and Scope of Algorithmic Health Infrastructures
Advances in machine learning techniques and cloud-computing resources have made it possible to classify and analyze large amounts of medical data, allowing the automated and accurate detection of conditions like diabetic retinopathy and forms of skin cancer in medical settings. At the same time, eager to apply AI techniques to health challenges, technology companies have been analyzing everyday experiences like going for a walk, food shopping, sleeping, and menstruating to make inferences and predictions about people’s health behavior and status.

While such developments may offer future positive health benefits, little empirical research has been published about how AI will impact patient health outcomes or experiences of care. Furthermore, the data- and cloud-computing resources required for training models to AI health systems have created troubling new opportunities, expanding what counts as “health data,” but also the boundaries of healthcare. The scope and scale of these new “algorithmic health infrastructures” give rise to a number of social, economic, and political concerns.

The proliferation of corporate-clinical alliances for sharing data to train AI models illustrates these infrastructural impacts. The resulting commercial incentives and conflicts of interest have made ethical and legal issues around health data front-page news. Most recently, a whistle-blower report alerted the public to serious privacy risks stemming from a partnership, known as Project Nightingale, between Google and Ascension, one of the largest nonprofit health systems in the US. The report claimed that patient data transferred between Ascension and Google was not “de-identified.” Google helped migrate Ascension’s infrastructure to their cloud environment, and in return received access to hundreds of thousands of privacy-protected patient medical records to use in developing AI solutions for Ascension and also to sell to other healthcare systems.

Google, however, is not alone. Microsoft, IBM, Apple, Amazon, and Facebook, as well as a wide range of healthcare start-ups, have all made lucrative “data partnership” agreements with a wide range of healthcare organizations (including many university research hospitals and insurance companies) to gain access to health data for the training and development of AI-driven health systems. Several of these have resulted in federal probes and lawsuits around improper use of patient data.

However, even when current regulatory policies like HIPAA are strictly followed, security and privacy vulnerabilities can exist within larger technology infrastructures, presenting serious challenges for the safe collection and use of Electronic Health Record (EHR) data. New research shows that it is possible to accurately link two different de-identified EHR datasets using computational methods, so as to create a more complete history of a patient without using any personal health information of the patient in question. Another recent research study showed that it is possible to create reconstructions of patients’ faces using de-identified MRI images, which could then be identified using facial-recognition systems. Similar concerns have prompted a lawsuit against the University of Chicago Medical Center and Google claiming that Google is “uniquely able to determine the identity of almost every medical record the university released” due to its expertise and resources in AI development. The potential harm from misuse of these new health data capabilities is of grave concern, especially as AI health technologies continue to focus on predicting risks that could impact healthcare access or stigmatize individuals, such as recent attempts to diagnose complex behavioral health conditions like depression and schizophrenia from social-media data.

New Social Challenges for the Healthcare Community
This year a number of reports, papers, and op-eds were published on AI ethics in healthcare. Although mostly generated by physicians and medical ethicists in Europe and North America, these early efforts are important for better understanding the situated uses of AI systems in healthcare.

For example, the European and North American Radiology Societies recently issued a statement that outlines key ethical issues for the field, including algorithmic and automation bias in relation to medical imaging. Radiology is currently one of the medical specialties where AI systems are the most advanced. The statement openly acknowledges how clinicians are reckoning with the increased value and potential harms around health data used for AI systems: “AI has noticeably altered our perception of radiology data—their value, how to use them, and how they may be misused.”

These challenges include possible social harms for patients, such as the potential for clinical decisions to be nudged or guided by AI systems in ways that don’t (necessarily) bring people health benefits, but are in service to quality metric requirements or increased profit. Importantly, misuses also extend beyond the ethics of patient care to consider how AI technologies are reshaping medical organizations themselves (e.g., “radiologist and radiology departments will also be data” for healthcare administrators) and the wider health domain by “blurring the line” between academic research and commercial AI uses of health data.

Importantly, medical groups are also pushing back against the techno-solutionist promises of AI, crafting policy recommendations to address social concerns. For example, the Academy of Medical Royal Colleges (UK) 2019 report, “Artificial Intelligence in Healthcare,” pragmatically states: “Politicians and policymakers should avoid thinking that AI is going to solve all the problems the health and care systems across the UK are facing.” The American Medical Association has been working on an AI agenda for healthcare, too, also adopting the policy “Augmented Intelligence in Health Care”as a framework for thinking about AI in relation to multiple stakeholder concerns, which include the needs of physicians, patients, and the broader healthcare community.

There have also been recent calls for setting a more engaged agenda around AI and health. This year Eric Topol, a physician and AI/ML researcher, questioned the promises of AI to fix systemic healthcare issues, like clinician burnout, without the collective action and involvement of healthcare workers. Physician organizing is needed not because doctors should fear being replaced by AI, but to ensure that AI benefits people’s experiences of care. “The potential of A.I. to restore the human dimension in health care,” Topol argues, “will depend on doctors stepping up to make their voices heard.”

More voices are urgently needed at the table—including the expertise of patient groups, family caregivers, community health workers, and nurses—in order to better understand how AI technologies will impact diverse populations and health contexts. We have seen how overly narrow approaches to AI in health have resulted in systems that failed to account for darker skin tones in medical imaging data, and cancer treatment recommendations that could lead to racially disparate outcomes due to training data from predominantly white patients.

Importantly, algorithmic bias in health data cannot always be corrected by gathering more data, but requires understanding the social context of the health data that has already been collected. Recently, Optum’s algorithm designed to identify “high-risk” patients in the US was based on the number of medical services a person used, but didn’t account for the numerous socioeconomic reasons around the nonuse of needed health services, such as being underinsured or the inability to take time off from work. With long histories of addressing such social complexities, research from fields like medical sociology and anthropology, nursing, human-computer interaction, and public health is needed to protect against the implementation of AI systems that (even when designed with good intentions) worsen health inequities.

_____________

More to come...

Artificial Intelligence and Ethics

I was already tee'd up for the topic of this post, but serendipitously just ran across this interesting piece over at Naked Capitalism.

Why You Should NEVER Buy an Amazon Echo or Even Get Near One
by Yves Smith

At the Philadelphia meetup, I got to chat at some length with a reader who had a considerable high end IT background, including at some cutting-edge firms, and now has a job in the Beltway where he hangs out with military-surveillance types. He gave me some distressing information on the state of snooping technology, and as we’ll get to shortly, is particularly alarmed about the new “home assistants” like Amazon Echo and Google Home.

He pointed out that surveillance technology is more advanced than most people realize, and that lots of money and “talent” continues to be thrown at it. For instance, some spooky technologies are already decades old…

Read all of it, including the numerous comments.

"Your Digital Mosaic"

My three prior posts have returned to my episodic riffing on AI and robotics topics: see here, here, and here.

Earlier this week I ran across this article over at Wired:

WHY AI IS STILL WAITING FOR ITS ETHICS TRANSPLANT

There’s no lack of reports on the ethics of artificial intelligence. But most of them are lightweight—full of platitudes about “public-private partnerships” and bromides about putting people first. They don’t acknowledge the knotty nature of the social dilemmas AI creates, or how tough it will be to untangle them. The new report from the AI Now Institute isn’t like that. It takes an unblinking look at a tech industry racing to reshape society along AI lines without any guarantee of reliable and fair results…

Again, highly recommend you read all of it.

That led me to the "AI Now 2017 Report." (pdf)

The AI Now authors' 36-pg report examines in heavily documented detail (191 footnotes) four topical areas of AI applications and their attendant ethical issues: [1] Labor and Automation; [2] Bias and Inclusion; [3] Rights and Liberties, and, [4] Ethics and Governance.

From the Institute's website:

Rights & Liberties
As artificial intelligence and related technologies are used to make determinations and predictions in high stakes domains such as criminal justice, law enforcement, housing, hiring, and education, they have the potential to impact basic rights and liberties in profound ways. AI Now is partnering with the ACLU and other stakeholders to better understand and address these impacts.

Labor & Automation
Automation and early-stage artificial intelligence systems are already changing the nature of employment and working conditions in multiple sectors. AI Now works with social scientists, economists, labor organizers, and others to better understand AI's implications for labor and work – examining who benefits and who bears the cost of these rapid changes.

Bias & Inclusion
Data reflects the social, historical and political conditions in which it was created. Artificial intelligence systems ‘learn’ based on the data they are given. This, along with many other factors, can lead to biased, inaccurate, and unfair outcomes. AI Now researches issues of fairness, looking at how bias is defined and by whom, and the different impacts of AI and related technologies on diverse populations.

Safety & Critical Infrastructure
As artificial intelligence systems are introduced into our core infrastructures, from hospitals to the power grid, the risks posed by errors and blind spots increase. AI Now studies the way in which AI and related technologies are being applied within these domains and to understand possibilities for safe and responsible AI integration.

The 2017 Report proffers ten policy recommendations:

1 — Core public agencies, such as those responsible for criminal justice, healthcare, welfare, and education (e.g “high stakes” domains) should no longer use ‘black box’ AI and algorithmic systems. This includes the unreviewed or unvalidated use of pre-trained models, AI systems licensed from third party vendors, and algorithmic processes created in-house. The use of such systems by public agencies raises serious due process concerns, and at a minimum such systems should be available for public auditing, testing, and review, and subject to accountability standards…

2 — Before releasing an AI system, companies should run rigorous pre-release trials to ensure that they will not amplify biases and errors due to any issues with the training data, algorithms, or other elements of system design. As this is a rapidly changing field, the methods and assumptions by which such testing is conducted, along with the results, should be openly documented and publicly available, with clear versioning to accommodate updates and new findings…

3 — After releasing an AI system, companies should continue to monitor its use across different contexts and communities. The methods and outcomes of monitoring should be defined through open, academically rigorous processes, and should be accountable to the public. Particularly in high stakes decision-making contexts, the views and experiences of traditionally marginalized communities should be prioritized…

4 — More research and policy making is needed on the use of AI systems in workplace management and monitoring, including hiring and HR. This research will complement the existing focus on worker replacement via automation. Specific attention should be given to the potential impact on labor rights and practices, and should focus especially on the potential for behavioral manipulation and the unintended reinforcement of bias in hiring and promotion…

5 — Develop standards to track the provenance, development, and use of training datasets throughout their life cycle. This is necessary to better understand and monitor issues of bias and representational skews. In addition to developing better records for how a training dataset was created and maintained, social scientists and measurement researchers within the AI bias research field should continue to examine existing training datasets, and work to understand potential blind spots and biases that may already be at work…

6 — Expand AI bias research and mitigation strategies beyond a narrowly technical approach. Bias issues are long term and structural, and contending with them necessitates deep interdisciplinary research. Technical approaches that look for a one-time “fix” for fairness risk oversimplifying the complexity of social systems. Within each domain — such as education, healthcare or criminal justice — legacies of bias and movements toward equality have their own histories and practices. Legacies of bias cannot be “solved” without drawing on domain expertise. Addressing fairness meaningfully will require interdisciplinary collaboration and methods of listening across different disciplines…

7 — Strong standards for auditing and understanding the use of AI systems “in the wild” are urgently needed. Creating such standards will require the perspectives of diverse disciplines and coalitions. The process by which such standards are developed should be publicly accountable, academically rigorous and subject to periodic review and revision…

8 — Companies, universities, conferences and other stakeholders in the AI field should release data on the participation of women, minorities and other marginalized groups within AI research and development. Many now recognize that the current lack of diversity in AI is a serious issue, yet there is insufficiently granular data on the scope of the problem, which is needed to measure progress. Beyond this, we need a deeper assessment of workplace cultures in the technology industry, which requires going beyond simply hiring more women and minorities, toward building more genuinely inclusive workplaces…

9 — The AI industry should hire experts from disciplines beyond computer science and engineering and ensure they have decision making power. As AI moves into diverse social and institutional domains, influencing increasingly high stakes decisions, efforts must be made to integrate social scientists, legal scholars, and others with domain expertise that can guide the creation and integration of AI into long-standing systems with established practices and norms…

10 — Ethical codes meant to steer the AI field should be accompanied by strong oversight and accountability mechanisms. More work is needed on how to substantively connect high level ethical principles and guidelines for best practices to everyday development processes, promotion and product release cycles…

I printed it out and went old-school on it with yellow highlighter and red pen.

It is excellent. A must-read, IMO. It remains to be seen, though, how much traction these proposals get in a tech world of transactionalism and "proprietary IP/data" everywhere.

Given that my grad degree is in "applied ethics" ("Ethics and Policy Studies"), I am all in on these ideas. The "rights and liberties" stuff was particularly compelling for me. I've had a good run at privacy/technology issues on another of my blogs. See my post "Clapp Trap" and its antecedent "Privacy and the 4th Amendment amid the 'War on Terror'."

"DIGITAL EXHAUST"

Another recent read on the topic.

Introduction

This book is for everyone who wants to understand the implications of the Big Data phenomenon and the Internet Economy; what it is, why it is different, the technologies that power it, how companies, governments, and everyday citizens are benefiting from it, and some of the threats it may present to society in the future.

That’s a pretty tall order, because the companies and technologies we explore in this book— the huge Internet tech groups like Google and Yahoo!, global retailers like Walmart, smartphone and tablet producers like Apple, the massive online shopping groups like Amazon or Alibaba, or social media and messaging companies like Facebook or Twitter— are now among the most innovative, complex, fast-changing, and financially powerful organizations in the world. Understanding the recent past and likely future of these Internet powerhouses helps us to appreciate where digital innovation is leading us, and is the key to understanding what the Big Data phenomenon is all about. Important, too, are the myriad innovative frameworks and database technologies— NoSQL, Hadoop, or MapReduce— that are dramatically altering the way we collect, manage, and analyze digital data…

Neef, Dale (2014-11-05). Digital Exhaust: What Everyone Should Know About Big Data, Digitization and Digitally Driven Innovation (FT Press Analytics) (Kindle Locations 148-157). Pearson Education. Kindle Edition.

UPDATE

From MIT Technology Review:

Despite All Our Fancy AI, Solving Intelligence Remains “the Greatest Problem in Science”
Autonomous cars and Go-playing computers are impressive, but we’re no closer to machines that can think like people, says neuroscientist Tomaso Poggio.

Recent advances that let computers play board games and drive cars haven’t brought the world any closer to true artificial intelligence.

That’s according to Tomaso Poggio, a professor at the McGovern Institute for Brain Research at MIT who has trained many of today’s AI leaders.

“Is this getting us closer to human intelligence? I don’t think so,” the neuroscientist said at MIT Technology Review’s EmTech conference on Tuesday.

Poggio leads a program at MIT that’s helped train several of today’s AI stars, including Demis Hassabis, cofounder of DeepMind, and Amnon Shashua, cofounder of the self-driving tech company Mobileye, which was acquired by Intel earlier this year for $15.3 billion.

“AlphaGo is one of the two main successes of AI, and the other is the autonomous-car story,” he says. “Very soon they’ll be quite autonomous.”

But Poggio said these programs are no closer to real human intelligence than before. Responding to a warning by physicist Stephen Hawking that AI could be more dangerous than nuclear weapons, Poggio called that “just hype.”…

BTW - apropos, see The MIT Center for Brains, Minds, and Machines.

"The Center for Brains, Minds and Machines (CBMM)
is a multi-institutional NSF Science and Technology Center
dedicated to the study of intelligence - how the brain produces intelligent
behavior and how we may be able to replicate intelligence in machines."

Interesting. See their (dreadful video quality) YouTube video "Discussion Panel: the Ethics of Artificial Intelligence."

In sum, I'm not sure that difficulty achieving "general AI" -- "one that can think for itself and solve many kinds of novel problems" -- is really the central issue going to applied ethics concerns. Again, read the AI Now 2017 Report.

WHAT OF "ETHICS?" ("MORAL PHILOSOPHY")

Couple of good, succinct resources for you, here and here. My elevator speech take on "ethics" is that it is not about a handy "good vs bad cookbook." It goes to honest (albeit frequently difficult) moral deliberation involving critical thinking, deliberation that takes into account "values" that pass rational muster -- surpassing the "Appeal to Tradition" fallacy.

UPDATE

Two new issues of my hardcopy Science Magazine showed up in the snailmail today. This one in particular caught my attention.

What is consciousness, and could machines have it?

Abstract
The controversial question of whether machines may ever be conscious must be based on a careful consideration of how consciousness arises in the only physical system that undoubtedly possesses it: the human brain. We suggest that the word “consciousness” conflates two different types of information-processing computations in the brain: the selection of information for global broadcasting, thus making it flexibly available for computation and report (C1, consciousness in the first sense), and the self-monitoring of those computations, leading to a subjective sense of certainty or error (C2, consciousness in the second sense). We argue that despite their recent successes, current machines are still mostly implementing computations that reflect unconscious processing (C0) in the human brain. We review the psychological and neural science of unconscious (C0) and conscious computations (C1 and C2) and outline how they may inspire novel machine architectures.

Big data and the industrialization of neuroscience: A safe roadmap for understanding the brain?

I question here the scientific and strategic underpinnings of the runaway enthusiasm for industrial-scale projects at the interface between “wet” (biology) and “hard” (physics, microelectronics and computer science) sciences. Rather than presenting the achievements and hopes fueled by big-data–driven strategies—already covered in depth in special issues of leading journals—I focus on three major issues: (i) Is the industrialization of neuroscience the soundest way to achieve substantial progress in knowledge about the brain? (ii) Do we have a safe “roadmap,” based on a scientific consensus? (iii) Do these large-scale approaches guarantee that we will reach a better understanding of the brain?

This “opinion” paper emphasizes the contrast between the accelerating technological development and the relative lack of progress in conceptual and theoretical understanding in brain sciences. It underlines the risks of creating a scientific bubble driven by economic and political promises at the expense of more incremental approaches in fundamental research, based on a diversity of roadmaps and theory-driven hypotheses. I conclude that we need to identify current bottlenecks with appropriate accuracy and develop new interdisciplinary tools and strategies to tackle the complexity of brain and mind processes…

Interesting stuff. Stay tuned.
__

CODA

Save the date.

Link

____________

More to come...

The "Accelerationists" vs the "Doomers."

AI For Good?

 

Pending book release. Pre-pub excerpt from The Atlantic.

The Secret to Understanding AI
“Imagine the tech without the tech companies.”
By Josh Tyrangiel

In the before times—before machines could hallucinate, before compute was a noun—it was not uncommon to go several weeks without someone telling me the world was about to end. Similarly, a whole season might pass without anyone assuring me that it was also, simultaneously, about to become perfect.

That particular luxury died on November 30, 2022, when OpenAI released ChatGPT to the public. What followed was less a news cycle than a weather event—a tropical depression that would not budge. Within weeks, millions of people had their first experience with generative AI. Within months, every major technology company had announced its own version of a large language model, or a partnership, or a pivot. Venture capital arrived drooling. Most people in tech think about money, but AI-profit projections are different—like CFO fan fiction, written in Excel. In 2023, the McKinsey Global Institute estimated that $4.4 trillion in annual corporate profits could be up for grabs from generative AI alone. Morgan Stanley estimated $40 trillion more in operational efficiencies. The words artificial intelligence went from obscurity to a constant hum, present in every earnings call, every school-board meeting, and far too many arguments at dinner tables.

Yet for all of the noise, a simple question stayed unanswered: What exactly was this new technology going to do for people? Not for corporations or the billionaires who aspired to become trillionaires, but for people with mortgages and sick parents and children struggling to learn things…

May 12th release date.

AMAZON BLURB
In contrast to the wave of noisy polemics around AI, AI For Good explores how, in practice, it can actually improve our lives and tells the stories of everyday citizens at the forefront of this new “AI entrepreneurship.”

AI is often framed as a force of radical transformation, either catapulting us into a utopian future or dragging us toward existential ruin. But this book tells a different story. It’s not about high-profile tech CEOs who want to use AI to “break shit,” but about a bunch of smart pragmatists using AI to make the world better.

Josh Tyrangiel’s journey into AI began with a late-night YouTube video featuring General Gustave Perna, the retired four-star general who orchestrated the distribution of Covid vaccines during Operation Warp Speed. Perna’s success—and the end of the pandemic—depended on AI’s practical ability to synthesize and standardize vast amounts of logistical data. AI wasn’t the hero of the story—it was the tool that helped real people get things done.

This book follows those people, who make up a kind of AI counterculture. It explores AI’s quiet revolution in government services, medicine, education, and human connection—places where it’s being used to amplify human judgment rather than replace it. It tells the stories of teachers, doctors, and bureaucrats who often stumbled into AI as a means to solve specific, tangible problems, often with no prior software expertise.

While the loudest voices in AI debate doomsday scenarios and trillion-dollar market opportunities, this book focuses on those working in the messy, incremental, but deeply impactful space of AI practice. However, there is one big caveat—success is not guaranteed. Change is hard. Institutions move slowly. But even in failure there are lessons for everyone who’s interested in using AI—carefully, thoughtfully—to build a better world today.

I have too many books in play at the moment (about 8), but I'll be adding this to the list when it's released.

 

SOME OTHER READS JUST ADDED TO THE STASH

  

 

Dispatches from Grief is intensely personal for this "Girl Dad."

ERRATUM: SCIENTIFIC AMERICAN

 

The greatest pitfall in the search for extraterrestrial life—according to science fiction, anyway—is foolhardy researchers somehow bringing aliens to Earth to wreak havoc.

But after decades of exploring our seemingly sterile solar system, real-world scientists today are much more concerned with the opposite problem: The possibility that Earth’s life will escape our planet to contaminate other worlds, sabotaging the quest to find any genuine “second genesis” of biology around the sun. Imagine that a multibillion-dollar robotic mission found wriggling microbes on Mars and that follow-up studies then revealing those “aliens” had DNA and other biomolecular machinery that showed they were emigrants from Earth.

Astrobiologically speaking, we would have met the enemy—and it would be us. Taking a cue from sci-fi, you might call such life-forms “Klingons,” for their presumptive hitchhike to the Red Planet as stowaways in spacecraft sent from Earth.

“Planetary protection” is the term scientists use for efforts to prevent otherworldly invasions of all sorts; to date, most of it has focused on Mars, but the practice applies to all potentially habitable environments within reach of our spacecraft. In the 1970s, for example, NASA did its best to keep its twin Viking landers Klingon-free before launching them to Mars. And if the NASA-led international Mars Sample Return effort ever manages to bring its precious payload back to Earth, the agency will be tasked with quarantining those specimens as if they contain extreme biohazards rather than lifeless bits of rock and soil…

Imagine my surprise. apropos of some prior riffs on astrophysics and exobiology.

 

OFF-TOPIC, CHEERS... 

More shortly... 

Cognition in "Strange New Minds."

Are AI LLMs approaching true "sentience?"

   

Released today.
 

The Amazon blurb:

An insider look at the Large Language Models (LLMs) that are revolutionizing our relationship to technology, exploring their surprising history, what they can and should do for us today, and where they will go in the future—from an AI pioneer and neuroscientist

In this accessible, up-to-date, and authoritative examination of the world’s most radical technology, neuroscientist and AI researcher Christopher Summerfield explores what it really takes to build a brain from scratch. We have entered a world in which disarmingly human-like chatbots, such as ChatGPT, Claude and Bard, appear to be able to talk and reason like us - and are beginning to transform everything we do. But can AI ‘think’, 'know' and ‘understand’? What are its values? Whose biases is it perpetuating? Can it lie and if so, could we tell? Does their arrival threaten our very existence?

These Strange New Minds charts the evolution of intelligent talking machines and provides us with the tools to understand how they work and how we can use them. Ultimately, armed with an understanding of AI’s mysterious inner workings, we can begin to grapple with the existential question of our age: have we written ourselves out of history or is a technological utopia ahead?

 

SCIENCE MAGAZINE REVIEW

In These Strange New Minds, cognitive neuroscientist and artificial intelligence (AI) safety specialist Christopher Summerfield presents a wide-ranging overview of AI for nonspecialists, focusing on what the technology really is, what it might do, and whether it should be feared. We no longer live in “a world where humans alone generate knowledge,” writes Summerfield. Machines possessing this potential will soon occupy custodial positions in society, he maintains (1). His book takes on six broad questions: How did we get here? What is a language model? Do language models think? What should a language model say? What could a language model do? And, are we all doomed?

Summerfield is a philosophical empiricist who argues that “the meaning of language depends on its evidentiary basis.” He is also a functionalist who believes that “it is perfectly possible for the same computational principle to be implemented in radically different physical substrates” and a materialist who sees the mind’s activity as identical to “neural computation.” But does he believe that AI machines think like humans do, or just that they appear to?...

...In the book’s final section, Summerfield turns to whether the technology will doom or deliver humankind. Here, he begins by discussing computer scientist Rich Sutton’s assertion that humankind should already be planning for the inevitable and great “succession” as AI machines “take over.” Neither AI successionists nor its antagonists have much to offer compared with those “whose core members are rooted in the AI safety community, [who] believe that there is an urgent need for AI to be tightly regulated precisely because it is so potent a tool,” argues Summerfield.

Existential risk groups have alternatively called for AI to be widely and publicly paused or for large government and private investments to design AI monitoring and countermeasures. So far, little headway has been made in either direction, but Summerfield’s book offers nonspecialists a good introduction to the issues and some hope that sound efforts in AI safety may see the light of day.

Just getting started. 

I'd like to get Shannon Valor's take on this book.

 

DR. SUMMERFIELD

 

MORE:

Whether or not we are on a pathway to building AI systems that figure out the deepest mysteries of the universe, these more mundane forms of assistance are round the corner. It also seems likely that the main medium by which most people currently seek information – an internet search engine – will soon seem as quaint as the floppy disk or the fax machine. ChatGPT is already integrated into the search engine Bing, and it surely won’t be long before Google and others follow suit, augmenting page search with conversational skills. As these changes occur, they will directly touch the lives of everyone on the planet with internet access – more than five billion people and counting – and are sure to upend the global economy in ways that nobody can quite predict. And this is all going to happen soon – on a timeframe of months or years, not decades. It’s going to happen to you and me.

The new world I’ve described might sound like quite a blast. Imagine having access to AI systems that act as a sort of personal assistant – at your digital beck and call – much more cheaply than the human equivalent, a luxury that today only CEOs and film stars can afford. We would all like an AI to handle the boring bits of life – helping us schedule meetings, switch utility provider, submit our tax returns on time. But there are serious uncertainties ahead. By allowing AI systems to become the ultimate repositories for human knowledge, we devolve to them stewardship of what is true or false, and what is right or wrong. What role will humans still play in a world where AI systems generate and share most knowledge on our behalf?

Of course, ever since humans began to exchange ideas, they have found ways to weaponize dissemination – from the first acts of deception or slander among the pre-industrial hunter-gatherer crew to the online slough of misinformation, toxicity, and polemic that the internet has become today. If they are not properly trained, machines with language risk greatly amplifying these harms, and adding new ones to boot. The perils of a world in which AI has authority over human knowledge may exceed the promise of unbounded information access. How do we know when an LLM is telling the truth? How can we be sure that they will not perpetuate the subtle biases with which much of our language is inflected, to the detriment of those who are already least powerful in society? What if they are used as a tool for persuasion, to shepherd large groups of people towards discriminatory or dangerous views? And when people disagree, whose values should LLMs represent? What happens if large volumes of AI-generated content – news, commentary, fiction, and images – come to dominate the infosphere? How will we know who said what, or what actually happened? Are we on the brink of writing ourselves out of history?

Summerfield, Christopher. These Strange New Minds: How AI Learned to Talk and What It Means (pp. 7-8). (Function). Kindle Edition. 

 

 CHRISTOPHER SUMMERFIELD SPEAKS

 

BLASTS FROM MY BLOG PAST

 

 I searched back in the blog for a look at what I'd posted a devade or so ago on "Artificial Intelligence."

 

Fairly quaint.

 

Stay tuned...

_________

  

OK, time to get back to work

 

Unreal smarts, this young scholar.
 

   

My follow-on observation:

To use a DNA analogy, genomic diversity is “adaptive” precisely because–mixing my metaphors–“you can’t ever step in the same river twice.” apropos, see @brianklaas’s killer book “Flukes.” #LLM inbreeding is as maladaptive as genetic inbreeding. House of Windsor, anyone?

OK, I was not hip to her until reading a new Science Magazine review of her current book The AI Mirror. Bought her prior release as well (I have no life).

 

YOU GOTTA READ THIS BOOK

...[M]ost commercial AI systems today are powered by a machine learning model trained on a large body of data relevant to a specific task, then fine-tuned to optimize its performance on that task. 

 

This approach to AI has made rapid progress in widening machine capabilities, particularly in tasks using language, where we have the most data to train with. Indeed, since so many kinds of cognitive tasks are language-enabled, most experts now regard the term “Narrow AI” as outmoded, much like its predecessor label “Weak AI.” Very large language models, like OpenAI’s various iterations of GPT or Google DeepMind’s Gemini, can now do an impressively wide variety of things: answer questions, generate poems, lyrics, essays, or spreadsheets, even write and debug software code. Large image models can generate drawings, animations, synthetic photos or videos. While such models have a considerable speed advantage over human performance of these tasks, the quality and reliability of their outputs is often well below the peak of human ability. Still, some see evidence of progress toward AGI in their widening scope of action and the flexibility of a single base model to be fine-tuned for many new tasks. While a large language model (LLM) can’t solve a problem unless the solution is somehow embedded in the language data it is trained on, multimodal models trained on many types of data (text, image, audio, video, etc.) are expanding the performance range of AI models still further. 

 

Even if it no longer makes sense to call these tools “narrow” AI, they remain below the threshold of general intelligence—AGI. But it’s a mistake to explain that in terms of the problems they can’t yet solve. The true barrier to AGI is that AI tools today lack any lived experience, or even a coherent mental model, of what their data represent: the world beyond the bits stored on the server. This is why we can’t get even the largest AI models to reliably reflect the truth of that world in their outputs. The world is something they cannot access and, therefore, do not know. You might think there’s an easy fix: pair an AI model with a robot and let the robot’s camera and other sensors experience the world! But to an AI model, a robot’s inputs are just another data dump of ones and zeros, no different from image and sound files scraped from the Internet. These ones and zeros don’t organize themselves into the intelligent awareness of an open and continuous world. If they did, the field of intelligent robotics—including driverless cars, social robots, and robots in the service industry—would be progressing much faster. In 2015, fully automated cars and trucks were predicted to be everywhere by the 2020s. Yet in 2023, robotaxis piloted in San Francisco were still driving over firehoses, getting stuck in wet concrete, blocking intersections during busy festival traffic, violating basic rules of the road, obstructing emergency vehicles—even dragging a helpless pedestrian.4 It’s not just driving: the real-world performance of most twenty-first-century commercial robots has lagged well behind AI tools for solving language-based tasks. So, what’s the problem? 

 

A world is an open-ended, dynamic, and infinitely complex thing. A data set, even the entire corpus of the Internet, is not a world. It’s a flattened, selective digital record of measurements that humans have taken of the world at some point in the past. You can’t reconstitute the open, infinite, lived, and experienced world from any data set; yet data sets are all that any AI model has. You might say, “But surely this is true of the human brain as well! What more do we have than data streams from our eyes, ears, noses, and so on?” But your analog, biological brain remains a far more complex and efficient system than even the most powerful digital computer. In the words of theoretical physicist Michio Kaku, “Sitting on your shoulders is the most complicated object in the known universe.”5 It was built over hundreds of millions of years to give you something no AI system today has: an embodied, living awareness of the world you inhabit. This is why we ought to regard AI today as intelligent only in a metaphorical or loosely derived sense. Intelligence is a name for our cognitive abilities to skillfully cope with the world we awaken in each day.6 Intelligence in a being that has no world to experience is like sound in a vacuum. It’s impossible, because there’s no place for it to be. 

 

We humans do inhabit and experience a world, one rich with shared meaning and purpose, and, therefore, we can easily place the outputs of our latest AI tools within that context of meaning. We call these outputs “intelligent” because their form, extracted entirely from aggregated human data, unsurprisingly mirrors our own past performances of skilled coping with the world. They reflect back to us images of the very intelligence we have invested in them. Yet accuracy and reliability remain grand challenges for today’s AI tools, because it’s really hard to get a tool to care about the truth of the world when it doesn’t have one. Generative AI systems in particular have a habit of fabricating answers that are statistically plausible, but in fact patently false. If you ask ChatGPT to tell you about me and my career, it usually gets a lot right, but it just makes up the rest. When my host at a festival I was speaking at used ChatGPT to write my bio for the live audience, the tool listed in a confident tone a series of fictitious articles I haven’t written, named as my coauthors people that I’ve never met, and stated that I graduated from the University of California at Berkeley (I have never studied there). 

 

Importantly, these are not errors. Error implies some kind of failure or miscalculation. But these fabrications are exactly what ChatGPT is designed to do—produce outputs that are statistically plausible given the patterns of the input. It’s very plausible that someone who holds a distinguished professorial chair at a prestigious world university received her degree from another prestigious world university, like UC Berkeley. This fabrication is far more plausible, in fact, than the truth—which is that, due to harsh economic and family circumstances, after high school I attended a local community college in-between full-time work shifts, and later received my bachelor’s degree from a low-ranked (but dirt-cheap and good-quality) commuter university that offered night classes. When I was offered a PhD scholarship at age 25, I became a full-time student again after eight years in the workforce. I first set foot in a college dorm in my 40s, as a university professor. My story isn’t common. And that’s precisely why ChatGPT selected a more “fitting” story for me; quite literally, one that better “fit” the statistical curves of its data model for academic biographies. Later, we’ll consider the cost of relying on AI tools that smooth out the rough, jagged edges of all our lives in order to tell us more “fitting” stories about ourselves. 

 

These systems can perform computations on the world’s data far faster than we can, but they can’t understand it, because that requires the ability to conceive of more than mathematical structures and relationships within data. AI tools lack a “world model,” a commonsense grasp and flowing awareness of how the world works and fits together. That’s what we humans use to generalize and transfer knowledge across different environments or situations and to solve truly novel problems. AI solves problems too. Yet despite the common use of the term “artificial neural network” to describe the design of many AI models, they solve problems in a very different way than our brains do. AI tools don’t think, because they don’t need to. As this book explains, AI models use mathematical data structures to mimic the outputs of human intelligence—our acts of reasoning, speech, movement, sensing, and so on. They can do this without having the conscious thoughts, feelings, and intentions that drive our actions. Often, this is a benefit to us! It helps when a machine learning model’s computations solve a problem much faster than we could by thinking about it. It’s great when an AI tool finds a new, more efficient solution hidden somewhere in the math that you’d never look for. But your brain does much, much better than AI at coping with the countless problems the world throws at us every day, whose solutions aren’t mathematically predefined or encoded in data...

Vallor, Shannon. The AI Mirror: How to Reclaim Our Humanity in an Age of Machine Thinking (pp. 22-26). Oxford University Press. Kindle Edition.

Dang. This old washed-up guitar player is majorly impressed.

I am briefly reminded of my June post "The Apple of my AI." Also, "The Coming Wave?"

 

Searching back through my blog turns up a lot of stuff under "Artificial Intelligence." Shannon would likely take issue with a lot of that stuff. 

 

One of my faves from a few years ago is "The Myth of Artificial Intelligence."

NEW TERM: "TECHNOMORAL"

Click

I like it.

Stay tuned. Tons to reflect upon here. Way more to come...

_________

  

"Malign Technologies" update: AI Natural Language Generation (NLG)

Who might have copyright ownership claims to AI-generated human-readable text?

I posted this today on Facebook. From an interesting article in The Atlantic:

“The world of generative AI is progressing furiously. Last week, OpenAI released an advanced chatbot named ChatGPT that has spawned a new wave of marveling and hand-wringing, plus an upgrade to GPT-3 that allows for complex rhyming poetry; Google previewed new applications last month that will allow people to describe concepts in text and see them rendered as images; and the creative-AI firm Jasper received a $1.5 billion valuation in October…”

OK, question for all of my musician/songwriter friends. If you use the free updated GPT-3 to “write” song lyrics by proxy (i.e., you type in a theme—“Ooooh, baby, you’re gone, and my heart is broken, whah, whah, whah…”—and out pops a Lizzy McAlpine-worthy heartthrob lament), well, who “owns” the lyrics copyright? OpenAI? (But, wait! I thought they are “open source?”)

No one will give a shit unless and until the AI-spawned song becomes a lucrative chart success— after which every music biz IP lawyer in LA and Nashville will be aggressively elbowing each other aside in the sprint to the courthouse to file claims.

Count on it.

BTW: I am a long-time songwriter, of durably nil repute (or $$$). As my wife adroitly put it, I was Quixotically "working in the not-for-profit sector." I may have to screw around with GPT-3, just for grins.

  BobbyG, 1981, "The Once & Future Fool"

While living in Knoxville beginning in the late 1970's I was a member of the East TN Chapter of the Nashville Songwriters' Association. But, once belatedly in college at UTK, where I learned the statistical/economic principle of "expected value" (probability x payoff), it became clear that the average estimated prospective "value" of your lovingly-crafted song was in "basis points" (hundreths of a cent) at best.

I changed careers in 1986, and went first into environmental radiation laboratory science. The old joke: "How do you become a millionaire in the music business? Start out with at least two million dollars."

UPDATE

 

I opened an account at OpenAI to explore this tech.

 

 

It's certainly fast. But my initial use thus far is just Q&A.

____

Now, the topical focus of The Atlantic piece is "authenticity" of student prose writing in the academic environment.

Essay generation is neither theoretical nor futuristic at this point. In May, a student in New Zealand confessed to using AI to write their papers, justifying it as a tool like Grammarly or spell-check: “I have the knowledge, I have the lived experience, I’m a good student, I go to all the tutorials and I go to all the lectures and I read everything we have to read but I kind of felt I was being penalised because I don’t write eloquently and I didn’t feel that was right,” they told a student paper in Christchurch. They don’t feel like they’re cheating, because the student guidelines at their university state only that you’re not allowed to get somebody else to do your work for you. GPT-3 isn’t “somebody else”—it’s a program.

Yeah. But apropos of my opening scenario, one can quickly come up with a number of broader questions. When big money is at stake (e.g., my recent prior FTX posts), all manner of problems surface—NLG "plagiarism (?)" aside.

My priority interest in AI has principally been about stuff like this:

Click

NLU: Natural Language Understanding.

I asked OpenAI ChatGPT the following:

ME:
Can artificial intelligence analyze a prose argument?

ChatGPT:
Yes, artificial intelligence (AI) can be trained to analyze prose arguments to some extent. However, the extent to which AI can effectively analyze prose arguments will depend on various factors, such as the quality and amount of training data the AI has been given, the specific AI model being used, and the complexity and nuances of the argument being analyzed.

I'll have to see if I can provide it an "argument," and instruct it to analyze and evaluate it. Skeptical. but, we'll see.

See some of my prior riffs on "AI" topics more generally here.

UPDATES

More from The Atlantic piece.

A chasm has existed between humanists and technologists for a long time. In the 1950s, C. P. Snow gave his famous lecture, later the essay “The Two Cultures,” describing the humanistic and scientific communities as tribes losing contact with each other. “Literary intellectuals at one pole—at the other scientists,” Snow wrote. “Between the two a gulf of mutual incomprehension—sometimes (particularly among the young) hostility and dislike, but most of all lack of understanding. They have a curious distorted image of each other.” Snow’s argument was a plea for a kind of intellectual cosmopolitanism: Literary people were missing the essential insights of the laws of thermodynamics, and scientific people were ignoring the glories of Shakespeare and Dickens.

The rupture that Snow identified has only deepened. In the modern tech world, the value of a humanistic education shows up in evidence of its absence. Sam Bankman-Fried, the disgraced founder of the crypto exchange FTX who recently lost his $16 billion fortune in a few days, is a famously proud illiterate. “I would never read a book,” he once told an interviewer. “I don’t want to say no book is ever worth reading, but I actually do believe something pretty close to that.”…

Ahhh... C.P. Snow. I've riffed on him before here.

Also, the best place to hide $16 billion from our liitle a-literate Genius pal Sam Bankman-Fried is apparently inside a book. 

 

"The Beast."

 

OK, THIS IS CUTE

 

Scott Singeisen, a “Professor of architecture” posts a Twitter thread in reaction to The Atlantic article:

Me:
Write a well argued response to Stephen Marche’s recent essay in @TheAtlantic that AI will make the college essay obsolete

ChatGPT:
Stephen Marche's essay in the Atlantic argues that AI will make the college essay obsolete. However, this claim is not well-supported and is ultimately not convincing.

First of all, it is important to note that the college essay serves a specific purpose in the admissions process. It allows admissions officers to get a sense of the applicant's personality, writing abilities, and potential fit for the institution. AI, on the other hand, is not capable of providing this type of insight. While AI may be able to analyze data and generate texts, it cannot replicate the human experience and provide the same level of depth and understanding that a college essay can.

Furthermore, the use of AI in the admissions process raises concerns about fairness and bias. AI systems are only as good as the data they are trained on, and if the training data is [sic] biased, the AI will also be biased. This could lead to admissions decisions that are unfair or that do not accurately reflect an applicant's abilities and potential.

Additionally, the use of AI in the admissions process could potentially widen the gap between students from different backgrounds. Students from disadvantaged backgrounds may not have access to the same resources and support as their more privileged peers, leading to a disadvantage in the admissions process.

In conclusion, while AI may have the potential to assist in the admissions process, it is not capable of replacing the college essay. The college essay serves a specific purpose in the admissions process and is an important tool for admissions officers to evaluate applicants. The use of AI also raises concerns about fairness and bias, and could potentially widen the gap between students from different backgrounds.

I'm gonna have to play with some of this stuff. 

 

I'm lucky. I never needed AI NLG "help" when it came to writing—prose or song lyrics. Likely has something to do with the 2-3 books a week (plus all of my periodicals) I've studied across my-now 55 years of "adult" life since turning 21. Ya think?

 Stephen Marche:

"[N]atural-language processing is going to force engineers and humanists together. They are going to need each other despite everything. Computer scientists will require basic, systematic education in general humanism: The philosophy of language, sociology, history, and ethics are not amusing questions of theoretical speculation anymore. They will be essential in determining the ethical and creative use of chatbots, to take only an obvious example…"

Yeah, And, don't forget the lawyers. Never overlook the lawyers, lest you come to rue the day.

MORE UPDATES

Click

Click

Also, "Amazon orders limited series about the FTX crypto exchange scandal"

OK, WHAT ABOUT THIS CRAP?

 

  Ugh... 

CODA

 

 

My latest login. The popularity of ChatGPT is melting their servers. 

__________