So, we come to the most important of all our questions: What sort of control over machines that think do we need in human society?
MACHINE THAT BOTH THINKS AND ACTS
From a narrow point of view, a machine that only thinks produces only information. It takes in information in one state, and it puts out information in another state. From this viewpoint, information in itself is harmless; it is just an arrangement of marks; and accordingly, a machine that thinks is harmless, and no control is necessary.
Although it is true that the information produced only becomes good or evil after other machinery or human beings act on the information, in reality a machine with the power to produce information is constructed only for the reason of its use. We want to know what such machines can tell us only because we can then proceed to act much more efficiently than before. For example, a guided missile needs a mechanical brain only because then it can reach its target. In all cases mechanical brains are inseparable from their uses.
For the purposes of this chapter, the narrow view will be rejected because it dodges the issue. We shall be much concerned with the combination of a machine that thinks with another machine that acts; and we shall often call this combination the robot machine.
READING THIS CHAPTER
Now, before launching further into the discussion, we need to say that the conclusions suggested in this chapter are not final. Even if they are expressed a little positively in places, they are nevertheless subject to change as more information is discovered and as the appraisal of information changes with time. Also, almost any conclusions about social control—including, certainly, the conclusions in this chapter—are subject to controversy. But controversy is good: it leads to thought. The more minds that go to work on solving the problem of social control over robot machines and other products of the new technology—which is rushing upon us from the discoveries of the scientists—the better off we all will be. If, while stimulating disagreement, the ideas expressed in this chapter should succeed in stimulating thought and deliberation, the purpose of this chapter will be well fulfilled.
Up to this point in this book, the emphasis has been on possibilities of benefits to humanity that may arise from machines that think. In this chapter, devoted as it is to the subject of control, the emphasis is on possibilities for harm. Both possibilities are valid, and the happening of either depends upon the actions of men. In much the same way, atomic energy is a great possibility for benefit and for harm. It is the nature of control to put a fence around danger; and so it is natural in this chapter that the weight of attention should shift to the dangerous aspects of machines that think.
Perhaps a reader may feel that a chapter of this kind is rather out of place in a book, such as this one, that seeks to be scientific. If so, he is reminded that, in accordance with the general suggestions for reading this book stated in the preface, he should omit this chapter.