Finished this one. A great, timely,sobering read. I'll just leave you with this excerpt:
...The way Kurzweil and his fellow singularitarians talk about the technology to come makes it seem like they’re playing a video game like Civilization, where there is a technology tree laid out in front of them clearly, and humanity (or indeed any intelligent species) is just working its way through that preexisting tree. But technology isn’t on rails, barreling down a set course beyond anyone’s control. Moore’s law isn’t a law of nature; it was a decision. That decision was made by the executives running the computer industry, and holding to that decision required (and still requires) careful planning, along with enormous amounts of money, materials, and human attention. The laws of physics and other sciences set boundaries on the possible range of technology, but the actual technologies that humanity develops within those boundaries are determined by choices and social pressures. Yet Kurzweil, and others of his ilk, frequently discuss technology as if it’s an implacable, inhuman force with its own desires, running down a path that has absolutely nothing to do with the collective choices of humanity. This is present throughout Kurzweil’s claims about the Singularity: we will have nanotechnology, we will saturate the universe with our intelligence, rather than we may or we could choose to.
This rhetoric of inevitability serves several convenient purposes. For the people developing their ideas into technology, such rhetoric offers absolution for any unpleasant and unforeseen consequences of their inventions, because they were only uncovering the already extant course of technology, rather than steering it themselves. When there’s no room for human agency, there’s no room for moral responsibility either. And if the future of technology really is on rails, then much can be revealed simply by careful examination of the purportedly inevitable route—a route that leads straight to the vision of the Singularity. This offers the promise of not only accurately predicting the future (and the sense of control that comes with), but also (and more importantly) endless life in a future filled with the transcendence and control that characterize the ideology of technological salvation. With the end of nature and the advent of a universe that is simply one enormous, artificial computer—where we live in still-more-artificial worlds generated by those computers—the promise of control is total, especially for those who know how to control computers. This is a fantasy of a world where the single most important thing, the thing that literally determines all aspects of reality, is computer programming. All of humanity, running on a computer, until the end of time.
Let’s take a moment to consider what this would be like. Presumably, there would be a simulated world, something like the real world—that is to say, something like our own. Human minds would want at least a simulacrum of nature, and providing one to the uploaded minds running on the machine would be easy enough with all that memory and processing power. There would be none of the inhuman originality of nature, but there would be an attempt at re-creating it, to create a pleasant environment for the uploaded people to live in. All of this is fantasy, of course, but when asked to describe what life would be like for uploaded people after the Singularity, this is the kind of answer that Kurzweil and others give. They describe what sounds for all the world like a high-definition metaverse that you can’t actually leave, a permanent virtual reality that shares some of the basic features of our own real world, or a real world destroyed and reshaped to more closely resemble an immersive computer game. To the singularitarians, the largest and most crucial difference, the one they keep coming back to in their descriptions, is the end of death. Kurzweil’s work is just the latest entry in the annals of the oldest fantasy of humanity. Even if personal immortality were possible through technological means, ultimately cosmology would place limits on the duration of physical structures, but Kurzweil doesn’t accept those limits either. He called overcoming cosmological forces the “goal of the Singularity,” and this is indeed the point of the entire enterprise. The destruction of nature is secondary. The objective is to tame the universe, to make it into a padded playground. Paving over every paradise is just the side effect of building the universal parking lot, where nothing bad can ever happen again. Nobody would age, nobody would get sick, and—perhaps above all else—nobody’s dad would die.
“We collected everything my father had written—now, he died when I was 22, so he’s been dead for more than 50 years—and we fed that into a large language model,” Kurzweil said in 2024. “And then you could talk to him, you’d say something, you then go through everything he ever had written, and find the best answer that he actually wrote to that question. And it actually was a lot like talking to him. You could ask him what he liked about music—he was a musician. He actually liked Brahms the best. And it was very much like talking to him.” In a 2023 Rolling Stone profile—following up on the one fourteen years earlier—Kurzweil revealed some of the content of that conversation: “[Kurzweil] asked his ‘Dad Bot,’ as he puts it, what he loves most about music (‘The connection to human feelings’), his gardening (‘It’s the kind of work that never ends’), and his anxieties (‘Often nightmarish’). ‘What’s the meaning of life?’ he finally typed to his Dad Bot. ‘Love,’ his Dad Bot replied.” These responses look like the kind of thing one would expect from a generic version of ChatGPT, rather than a conversation between a professional musician and his adult son. But Kurzweil is sanguine that his “Dad Bot” will improve too. “I think as we get further on, we can actually do that more and more responsively, and more and more that really would match that person and actually emulate the way he would move and so on, [and his] tone of voice,” he says. “Computers are going to make things even better. I mean, just the kind of things you can do now with a large language model didn’t exist two years ago.”
Kurzweil’s attempt to resurrect his father feels poignantly deluded. It’s also a striking illustration of how Kurzweil’s confidence in the increasing power of technology—according to his strict schedule—is accompanied by his Panglossian insistence that more technology, especially computer technology, is always an improvement. He certainly thinks that technological dangers exist—nuclear weapons, for one, as well as certain applications of nanotechnology—but he thinks the solution is to build more technology that detects and counters those dangers. “Yes, there are dangers, but the computers will also be more intelligent to avoid kinds of dangers,” he says. And as for the superintelligent post-Singularity computers themselves, he’s confident that they’ll treat us well, whether or not we merge with them. “I would expect the intelligence that arises from the Singularity to have great respect for their biological heritage,” he writes.
This mirrors Kurzweil’s obvious respect and love for his late father. But not all children feel that way about their parents. Despite Kurzweil’s many questionable claims, this one—the idea that a superintelligent AGI will treat humans with respect—is easily the most contentious among his intellectual successors today. Eliezer Yudkowsky and his followers agree with Kurzweil that, surely, something is at hand. But rather than the Singularity, they fear a rougher beast is slouching toward us, with exponential speed.
Becker, Adam. More Everything Forever: AI Overlords, Space Empires, and Silicon Valley's Crusade to Control the Fate of Humanity (pp. 87-90). (Function). Kindle Edition.
The Science Magazine review that alerted me to the book.
'Powerful, intelligent, rich people with poor ideas can be terribly dangerous indeed. That is why this is a serious issue rather than a “quirky tour of wacky ideas.” We must rightfully protest: Who has appointed these deranged individuals to be the stewards of mankind?'Elon Musk, anyone?
Riveting book, this one. I've moved on to his prior book.
apropos of Elon. On deck.
 |
| Click |
 |
| Nothin' to see here, folks. |
More titles of topical interest.
Stay tuned,,,
No comments:
Post a Comment