11 And, as David Forrest suggests in his paper "Regulating Nanotechnology Development," available at www.foresight.org/NanoRev/Forrest1989.html, "If we used strict liability as an alternative to regulation it would be impossible for any developer to internalize the cost of the risk (destruction of the biosphere), so theoretically the activity of developing nanotechnology should never be undertaken." (See "Test of Time,"Wired 8.03, page 78.). It is this further danger that we now fully face—the consequences of our truth-seeking. In 1949, the Soviets exploded an atom bomb. Joy responded to this, stating that he liked that people were starting to respond to his article because it gave them an input on the subject.[13]. That’s why, I can’t say that I’m a hundred percent ok with us not being needed in the future. As a technologist, it gave me a sense of calm—that is, nanotechnology showed us that incredible progress was possible, and indeed perhaps inevitable. Churchill remarked, in a famous left-handed compliment, that the American people and their leaders "invariably do the right thing, after they have examined every other alternative." It is the essential source of information and ideas that make sense of a world in constant transformation. But if we are downloaded into our technology, what are the chances that we will thereafter be ourselves or even human? If they are humane they may use propaganda or other psychological or biological techniques to reduce the birth rate until the mass of humanity becomes extinct, leaving the world to the elite. I have spent the last 20 years still trying to figure out how to make computers as reliable as I want them to be (they are not nearly there yet) and how to make them simple to use (a goal that has met with even less relative success). "The Ethics of Nanotechnology: Vision and Values for a New Generation of Science and Engineering", Learn how and when to remove this template message, "Critique of Bill Joy's "Why the future doesn't need us, "A Response to Bill Joy and the Doom-and-Gloom Technofuturists", "Technological Utopias or Dystopias: Is There a Third Way? We don't know how widespread this ability is, but Kauffman notes that it may hint at "a route to self-reproducing molecular systems on a basis far wider than Watson-Crick base-pairing."7. Thus we have the possibility not just of weapons of mass destruction but of knowledge-enabled mass destruction (KMD), this destructiveness hugely amplified by the power of self-replication. Certainly not. Write a three- to five-page (600-900 word) response to thefollowing question In Why the Future Doesnt Need Us (pp. Reprinted as "Science and Society" in Greetings, Carbon-Based Bipeds! I had missed Ray's talk and the subsequent panel that Ray and John had been on, and they now picked right up where they'd left off, with Ray saying that the rate of improvement of technology was going to accelerate and that we were going to become robots or fuse with robots or something like that, and John countering that this couldn't happen, because the robots couldn't be conscious. I frankly believe that the situation in 1945 was simpler than the one we now face: The nuclear technologies were reasonably separable into commercial and military uses, and monitoring was aided by the nature of atomic tests and the ease with which radioactivity could be measured. Molecular electronics—the new subfield of nanotechnology where individual molecules are circuit elements—should mature quickly and become enormously lucrative within this decade, causing a large incremental investment in all nanotechnologies. Others, not so lucky or so prudent, perish. South America, like Australia today, was populated by marsupial mammals, including pouched equivalents of rats, deers, and tigers. Joy co-founded Sun Microsystems in 1982. Verification of AGI-limitation agreements would be difficult due to AGI's dual-use nature and ease of being hidden. [11] Goldsmith states his belief that scientists don't think of a lot of things that can go wrong when they start making inventions, because that will lead to less funding. "14 In the 21st century, this requires vigilance and personal responsibility by those who would work on both NBC and GNR technologies to avoid implementing weapons of mass destruction and knowledge-enabled mass destruction. Drexler's vision also led to a lot of good fun. We only point out that the fate of the human race would be at the mercy of the machines. Oppenheimer, though, was sufficiently concerned about the result of Trinity that he arranged for a possible evacuation of the southwest part of the state of New Mexico. Explain. But despite the strong historical precedents, if open access to and unlimited development of knowledge henceforth puts us all in clear danger of extinction, then common sense demands that we reexamine even these basic, long-held beliefs. What we do suggest is that the human race might easily permit itself to drift into a position of such dependence on the machines that it would have no practical choice but to accept all of the machines' decisions. I believe that we all wish our course could be determined by our collective values, ethics, and morals. Nature, 402, December 9, 1999: 583. My immediate hope is to participate in a much larger discussion of the issues raised here, with people from many different backgrounds, in settings not predisposed to fear or favor technology for its own sake. We will undoubtedly encounter strong resistance to this loss of privacy and freedom of action. Stories of run-amok robots like the Borg, replicating or mutating to escape from the ethical constraints imposed on them by their creators, are well established in our science fiction books and movies. We have our first pet robots, as well as commercially available genetic engineering techniques, and our nanoscale techniques are advancing rapidly. Ideas can't be put back in a box; unlike uranium or plutonium, they don't need to be mined and refined, and they can be freely copied. One would think we might be driven to such a dialogue by our instinct for self-preservation. And if our own extinction is a likely, or even possible, outcome of our technological development, shouldn't we proceed with great caution? As Thoreau said, "We do not ride on the railroad; it rides upon us"; and this is what we must fight, in our time. "Self-replication: Even Peptides Do It." It's unfortunate that the Pugwash meetings started only well after the nuclear genie was out of the bottle—roughly 15 years too late. Nearly 20 years ago, in the documentary The Day After Trinity, Freeman Dyson summarized the scientific attitudes that brought us to the nuclear precipice: "I have felt it myself. Perhaps it is always hard to see the bigger impact while you are in the vortex of a change. So I'm still searching; there are many more things to learn. By Bill Joy From the moment I became involved in the creation of new technologies, their ethical dimensions have concerned me, but it was only in the autumn of 1998 that I became anxiously … Continue reading "Why The Future Doesn’t Need … But I guess I wasn't totally surprised. I think it is no exaggeration to say we are on the cusp of the further perfection of extreme evil, an evil whose possibility spreads well beyond that which weapons of mass destruction bequeathed to the nation-states, on to a surprising and terrible empowerment of extreme individuals. I have long realized that the big advances in information technology come not from the work of computer scientists, computer architects, or electrical engineers, but from that of physical scientists. Aristotle opened his Metaphysics with the simple statement: "All men by nature desire to know." At around the same time, I found Hans Moravec's book Robot: Mere Machine to Transcendent Mind. © 2021 Condé Nast. It is irresistible if you come to them as a scientist. And so the nuclear arms race began. As society and the problems that face it become more and more complex and machines become more and more intelligent, people will let machines make more of their decisions for them, simply because machine-made decisions will bring better results than man-made ones. I excelled in mathematics in high school, and when I went to the University of Michigan as an undergraduate engineering student I took the advanced curriculum of the mathematics majors. Will we survive our technologies? The experiences of the atomic scientists clearly show the need to take personal responsibility, the danger that things will move too fast, and the way in which a process can take on a life of its own. As we have seen, Moravec agrees, believing we may well not survive the encounter with the superior robot species. WhatsApp. For more than 50 years, it has shown an estimate of the relative nuclear danger we have faced, reflecting the changing international conditions. Why doesn’t the future need us? It might be argued that the human race would never be foolish enough to hand over all the power to the machines. Nanotechnology has clear military and terrorist uses, and you need not be suicidal to release a massively destructive nanotechnological device—such devices can be built to be selectively destructive, affecting, for example, only a certain geographical area or a group of people who are genetically distinct. To seek revenge he constructs and disseminates a new and highly contagious plague that kills widely but selectively. 8 Else, Jon. With all the present talk of weapons of mass destruction, Joy is more concerned about weapons of knowledge-enabled mass destruction. I recently had the good fortune to meet the distinguished author and scholar Jacques Attali, whose book Lignes d'horizons ( Millennium, in the English translation) helped inspire the Java and Jini approach to the coming age of pervasive computing, as previously described in this magazine. He goes on to describe a fourth utopia, Fraternity, whose foundation is altruism. But does this mean it has reached people? His eyes and hands knew where every line, curve, mass must emerge, and at what depth in the heart of the stone to create the low relief. [3] Joy mentioned Hans Moravec's book ''Robot: Mere Machine to Transcendent Mind'' where he believed there will be a shift in the future where robots will take over normal human activities, but with time humans will become okay with living that way. Writing the code that argued so strongly to be written. 7 Kauffman, Stuart. Its argument was that "our most powerful 21st century technologies--robotics, genetic engineering, and nanotech--are threatening to make humans an endangered species." MIT Press, 1992: 269. (Actually, by getting rid of all but 100 nuclear weapons worldwide—roughly the total destructive power of World War II and a considerably easier task—we could eliminate this extinction threat.13. Bill Joy’s “Why the Future Doesn’t Need Us” caused a stir among the information technology community because of its dystopian vision of the future and also the fact that Joy is a well-respected leader of the technology industry. Yet, with each of these technologies, a sequence of small, individually sensible advances leads to an accumulation of great power and, concomitantly, great danger. [4], In The Singularity Is Near, Ray Kurzweil questioned the regulation of potentially dangerous technology, asking "Should we tell the millions of people afflicted with cancer and other devastating conditions that we are canceling the development of all bioengineered treatments because there is a risk that these same technologies may someday be used for malevolent purposes?". Kaczynski's actions were murderous and, in my view, criminally insane. Solving problems. (Actually, this is Finagle's law, which in itself shows that Finagle was right.) He is clearly a Luddite, but simply saying this does not dismiss his argument; as difficult as it is for me to acknowledge, I saw some merit in the reasoning in this single passage. Otherwise, we can easily imagine an arms race developing over GNR technologies, as it did with the NBC technologies in the 20th century. The WIRED conversation illuminates how technology is changing every aspect of our lives—from culture to business, science to design. It seems to me far more likely that a robotic existence would not be like a human one in any sense that we understand, that the robots would in no sense be our children, that on this path our humanity may well be lost. In our time, how much danger do we face, not just from nuclear weapons, but from all of these technologies?