On-Line
Data-Acquisition Systems
in
Nuclear Physics, 1969

Ad Hoc Panel on On-Line Computers in Nuclear Research
Committee on Nuclear Science
National Research Council

NATIONAL ACADEMY OF SCIENCES
Washington, D.C.1970

This is a report of work under Contract NSF-C310, T.O. 47 between the National Science Foundation and the National Academy of Sciences and under Contract AT(49-1)3236 between the U.S. Atomic Energy Commission and the National Academy of Sciences.

Available from
Committee on Nuclear Science
2101 Constitution Avenue
Washington, D.C. 20418

[PREFACE]

The first digital electronic device employed to collect nuclear data was the binary electronic counter (scaler) of the 1930's. In the next decade single and multichannel pulse-height analyzers appeared, still using vacuum tubes. In the 1950's the development of multichannel analyzers continued vigorously, with vast improvement of the analog-to-digital converter sections and with the introduction of computer-type memories, based first on acoustic delay lines and a short time later on ferrite cores. The replacement of vacuum tubes by transistors beginning in the latter half of the 1950's accelerated the pace of development and application of all types of electronic laboratory instruments.

The 1960's was the decade of the computer. Before the 1960's almost no on-line computers were used in nuclear research, but since about 1962 the computer has moved into the nuclear laboratory. It provides the research worker with an immensely flexible, powerful, and accurate tool capable of raising the research output of a laboratory while eliminating the most tedious part of the experimental work.

The phenomenal speed of development of computer hardware, software, and methodology contributes to the difficulty experienced by everybody involved in decision-making processes regarding data-acquisition systems. Since the cost of a computer system is often a sizable fraction of the total cost of a new laboratory, there is urgent need for a set of guiding rules or principles for use by a laboratory director planning a system, a reviewer going over a proposal for support, or a potential funding agency considering proposals and reviews. The purpose of this report is to assist in filling this need. The material presented is current through 1969. Although we deal with a field that is developing rapidly, we hope that a substantial portion of the material covered will have long-lasting value.

The report was prepared by the Ad Hoc Panel on On-Line Computers in Nuclear Research of the Committee on Nuclear Science, National Research Council. Appointed in March 1968, the Panel first met in Washington, D.C., on April 22, 1968.

The original members of the Panel were H. W. Fulbright, H. L. Gelernter, L. J. Lidofsky, D. Ophir (through late 1968), L. B. Robinson, and M. W. Sachs. In June 1968, this group prepared an interim report. L. J. Lidofsky was on sabbatical leave in Europe and therefore could not participate during the academic year 1968-1969. Early in 1969 J. F. Mollenauer and J. Hahn joined the Panel.

The Panel has reviewed the present state of the field and has attempted to anticipate future needs. We have agreed on many important matters, including especially useful design features for computers employed in data acquisition, as well as types of organization of data-acquisition systems suitable for various purposes, types of software that manufacturers should supply, and approximate costs of systems, and we present a number of recommendations in these areas. However, the Panel makes no recommendation on standards for computer hardware, such as logic levels and polarities, because of a conviction that these are now rapidly being established as a result of sound engineering progress and the pressure of economic competition in the fast-moving computer business.

Throughout this report we have expressed opinions based on our own experience and on the best information at our disposal. The nature of the report seemed to demand some discussion of properties of specific computers by name. We have tried to be neither misleading nor unjust in our evaluations.

We wish to thank everyone who has aided us, especially P. W. McDaniel, C. V. Smith, and G. Rogosa of the U.S. Atomic Energy Commission and the many scientists in AEC-and NSF-sponsored laboratories who supplied the basic data on which the economic survey chapter is based. We are indebted to several members of the staff of the Department of Physics and Astronomy of the University of Rochester for assistance in the preparation of the manuscript, especially Mrs. Brignall and Mrs. Hughes. We also received initial directions and many helpful suggestions from D. A. Bromley, Chairman of the Committee on Nuclear Science, F. S. Goulding, Chairman of the Subcommittee on Instrumentation and Methods, W. S. Rodney and P. Donovan of the National Science Foundation, and Charles K. Reed, Executive Secretary of the Committee on Nuclear Science.

CONTENTS

[Chapter 1]
THE TASKS AND THE COMPUTERS

A. INTRODUCTION

On-line data-acquisition computer systems are made in a wide range of types and sizes. In all cases at least one electronic computer is involved—a stored-program machine—because wired-program devices such as pulse-height analyzers are not considered to be computers. The rest of the system typically consists of input/output (I/O) devices such as analog-to-digital converters (ADC's), printers, cathode-ray oscilloscopes, plotters, and control devices, which may include, in addition to the console typewriter, switch boxes to simplify the control of special types of operations and perhaps a set of logic circuits associated with the input system, used to provide preliminary selection of incoming data. In a small but increasing number of cases a computer is seen dedicated entirely to a "process-control" application such as the automatic adjustment of the shim coils of a variable-energy cyclotron or the control of data acquisition in a nuclear-scattering experiment, adjustments such as changing the angle of observation being made essentially under direct automatic control of the computer. The smallest on-line systems use the smallest commercially available computers; the largest use computers bigger than those which until recently served most computing centers. Large systems sometimes include one or more satellite computers. The cost of individual systems ranges from $25,000 to $1,000,000, approximately. The total cost of computer systems in low-energy nuclear laboratories is estimated by now to have reached about $20,000,000. (There has been a larger expenditure in the high-energy nuclear field, where computer systems have been employed extensively for some years longer and where experiments are so expensive that the economic advantages of computer use were quickly recognized.)

B. THE TASKS

We first list the main uses to which on-line computer systems have been put. We start with the simple operations, which we call Class 1.

Class 1 operations:

a. Accepting digital data from external devices and storing it in computer memory.

b. Preliminary processing of incoming data, on-line, before storage. This usually involves only operations of logic and simple arithmetic.

c. Controlling the presentation of data via cathode-ray oscilloscope or typewriter, often for the purpose of monitoring the progress of an experiment.

d. Controlling the recording of digital data on magnetic tape, paper tape, or other storage medium.

e. Controlling an incremental plotter.

f. Controlling the output of large quantities of data via a line printer.

g. Transmission of quantities of data between two computers or between a computer and a pulse-height analyzer or other device having a magnetic core memory.

Several operations of intermediate complexity we will label Class 2.

Class 2 operations:

a. Processing of data already accumulated and stored either in memory or on tape or other medium (off-line processing). This data reduction is often more complicated and lengthy than the preliminary on-line processing referred to in (Class 1b).

b. Calculation of information required by the experimenter during the experiment, for example, kinematics tables and particle energies corresponding to field strengths in analyzer magnets.

c. Process-control operations, in which the computer directs or regulates a sequence of events in an experiment. Under program control the computer monitors the course of the experiment and supplies signals that cause automatic changes in experimental conditions, such as starting and stopping times of event counting, angles of observation of scattered particles, and accelerator energies. Such applications are designed to relieve the experimenter of unnecessary labor and to reduce the probability of error in routine operations.

Our final class involves even more complex calculations.

Class 3 operations:

a. Complicated treatment of reduced data, including least squares and curve fitting.

b. Large-scale calculations such as those required for the evaluation of theoretical nuclear scattering and reaction cross sections, e.g., DWBA calculations, which may each require running times of the order of minutes, even at a modern computing center.

Apparently Class 3 operations do not always have to be done during the course of the experiment; in fact, they can in most cases be carried out later, leisurely, at the local computing center. Nonetheless, calculations of the first type, and to a lesser extent the second, are currently being done at laboratories having large, powerful computers in their on-line data-acquisition systems.

C. THE COMPUTERS

1. Introduction

Because computers have proved useful in so many fields, many varieties are now on the market, quite a few of them having properties highly suitable for nuclear-data acquisition. The properties particularly useful are, first, the ease with which a great variety of external input and output devices can be attached (interfaced to the computer); second, provisions for rapid, efficient response to interrupt signals from external devices; and third, usually a means of transferring data from external devices directly into blocks of memory without use of the central processor, the transfer possibly requiring only a single memory cycle per word. (This is referred to as direct memory access through a direct data channel.)

Several types of small computers have appeared on the market during the past year, some having 8-bit words, but they are too small for general data-acquisition use, although valuables for special applications. For present purposes, the smallest useful machines have a minimum memory size of 4096 (4k) 12-bit words, which can usually be enlarged to 32k words by the addition of memory modules, while the larger machines have minimum memories of at least 8k, with provision for expansion to several hundred k. Regardless of their size, the machines of the present generation all have memory cycle times around 1 or 2 µsec.

2. Rough Classification of Computers

Before proceeding with the discussion it is convenient to find a simple scheme for classifying computers. The scheme adopted here is to divide them into three loosely defined classes—small, medium, and large—essentially on the basis of the properties of the basic central processors:

Small
Word length 12 to 18 bits
Useful memory size 4k
Number of bits in instruction 3 or 4
Floating-point hardware orally offered
Approximate cost range $8500 to $40,000

Medium
Word length 16 to 24 bits
Useful memory size 8 to 16k
Number of bits in instruction 4 to 6
Floating-point hardware option sometimes offered
Approximate cost range $30,000 to $120,000

Large
Word length 32 to 48 bits
Useful memory size at least 16k
Number of bits in instruction 7 or more
Floating-point hardware
Approximate cost range $150,000 or more

Computers do not fall neatly into these three classifications, especially since manufacturers offer many optional features; therefore, some argument about the assignment of a particular machine to one or the other class is possible. This is especially true with respect to the small and medium types. The properties of a large number of small and medium-sized computers are given in Appendix A. Information on larger machines can be found in the Adams Associates Computer Characteristics Quarterly.

D. MATCHING COMPUTERS TO TASKS

Having classified both the computers and the jobs that they may be called on to do, we now ask this question: How suitable is each of the three types of computers for each of the three classes of jobs, given that in every case the acquisition system consists of a single computer coupled to all necessary input and output equipment?

1. Large Computers

We start with the large computer system. All classes of jobs can be handled by this powerful system. However, we should question the wisdom of assembling a system based on a large machine unless a substantial amount of numerical calculating is anticipated, because the essential advantage of the large computer—the advantage that costs so much—is its capacity for rapidly executing highly accurate floating-point arithmetical operations.

2. Small Computers

The small computer system can handle the jobs of data acceptance, data manipulation, and output characteristic of the simple Class 1 operations, but they are suitable for very few jobs involving floating-point arithmetic. In fact, we must usually be skeptical about the use of small machines for any of the Class 2 operations except those of the process-control type, which in many cases would involve little if any arithmetic. (Process-control applications have been rather few to date, but a rapid increase can be expected in this field, especially because of the convenience and low cost of small modern computers.) It is apparent that these machines have been designed as economical instruments specifically intended to handle Class 1 jobs. The smallest word length of a machine in this group, 12 bits, is sufficient for storing in one word the output of a 4096-channel ADC unit, but it is not quite so convenient for handling the output of a typical scaler, which would likely require the use of two words. The capability of even a small computer system to convert experimental information into digital form, to transfer it into memory, to manipulate it, and to present it for inspection in a digested, convenient form, all at a high rate and essentially without error, is of immense value to an experimenter who has to cope with the abundant outflow of data from a modern nuclear experiment.

3. Medium-Sized Computers

The capabilities of medium-sized computers are less clear. These machines are superior to the small ones mainly in two respects: they have a more flexible command structure (i.e., they have a larger set of wired-in operations), and, usually, they have a longer word length. These features make them easier to program and give them a limited, but important, capability to execute floating-point operations sufficiently quickly and accurately for many purposes, even though these operations must in most cases be programmed, in the absence of floating-point hardware. We can reasonably conclude that the medium-sized machines will serve for any use listed in Classes 1 and 2. Certain simpler calculations of Class 3a are also expected to prove feasible, but few, if any, of those of Class 3b.

E. ON CHARACTERISTIC FEATURES OF COMPUTERS AND RELATED EQUIPMENT

The value of any feature depends on its need in the application involved; therefore detailed, absolute statements regarding each characteristic usually cannot be made. However, the Panel has discussed various features at some length, and we present here some general comments on the pros and cons of these features. Among the items discussed are some, such as word length and cycle time, that represent basic, inherent properties of the computer; while a great many others, such as priority interrupts, are customarily offered as options.

1. Word Length

The shorter the word length the cheaper the hardware, generally speaking, but the less the accuracy in calculations unless multiple precision is used. For example, although the 12-bit words of the PDP-8 match the accuracy of data from most ADC's, they are too small not to match the output data from most counters; furthermore, indirect addressing is often required because a single word is too short to include both the operation code and the absolute address of a memory location. Apart from addressing considerations, a 12-bit word is too small for many uses, e.g., in general-purpose pulse-height analyzer applications where 16 bits or, better, 18 bits should be considered a minimum. Fortran programs for numerical calculations are in general best run on machines having at least 32-bit words, although 24-bit words are usually acceptable here when double precision can be used.

2. Number of Memory Words

In general the more words that a system can retain the better; but the greater the memory, the greater the expense. The cost must be weighed against the need. For simple handling of data, a 4k memory may be adequate, but in a large shared-time general-purpose machine a 16k or greater memory is essential. In the latter case, the resident shared-time monitor will probably occupy at least 6k of the memory, so with a 16k memory only 10k would be left accessible to users, and experience has shown that this much can be taken up completely by one user compiling a Fortran IV program. A 4k memory is adequate for many process-control applications, but it is too small for many other applications such as general-purpose pulse-height analyzer use, where an 8k memory is highly desirable. Adding a supplemental rotating memory device (disk or drum), at a cost per word about 1 percent that of core storage, is often preferable to adding core memory. [See 6 below.]

3. Cycle Time

For most purposes the typical memory cycle time of 1 to 2 µsec is quite adequate. Some of the modern computers have cycle times under 1 µsec.

4. Direct Data Channels

These allow sequential depositing of digital data from external devices directly into blocks of computer memory without intervention of the central processor (direct memory access, DMA). Such input may require only one computer cycle per word, that being the next cycle after the one during which the interrupt signal arrives. This is the fastest means of getting data into memory, but it requires more external hardware and more complex interfacing than input through an accumulator of the central processor. Most data-acquisition machines provide both possibilities. Direct data channels can be valuable for interfacing to magnetic disks, drums, and tapes.

5. Priority Interrupts (Nested)

These can be very useful. They may cost as little as $125 each, depending on the machine, and can be used to reduce greatly the overhead running time losses of the computer. In complicated data-taking applications many interrupt lines are desirable; 8 to 16 priority levels are generally adequate. The usual Fortran compiler cannot compile programs that respond properly to interrupts, although a relocatable object code generated by the compiler can always be assembled with a machine-language subroutine designed to handle interrupts. Enlargement of Fortran compilers for data-acquisition use to include statements designed to handle interrupts is desirable. (See, for example, the discussion of the Yale-IBM system, [Chapter 2, Section E.])

6. Mass Storage

Magnetic media—drums, disks, and standard magnetic tapes—are employed here. DEC tapes are useful and reliable, but they have only a small capacity. The use of such microtapes is also limited by their incompatibility with typical computer-center equipment. Reliable, inexpensive incremental magnetic tape units are now available which can be operated asynchronously at about 300 Hz, too slow for many purposes. Some of them can also be run much faster in a synchronous mode. Drums and disks are highly desirable because they provide program-controlled rapid access to great volumes of data. Typically, access times are of the order of 17 µsec. In the past few years, good and inexpensive disks have been developed which are now on the market. Some suppliers are IBM, CDC, Datadisk, Burroughs, DEC, and SDS. Disk storage is cheaper per word than core storage by two orders of magnitude; therefore, it is preferable for applications where data can be organized serially and where access and transfer time requirements can be relaxed somewhat. For example, a small DEC disk system for the PDP-8 holds up to 128k 12-bit words and has an average access time of 17 µsec and a transfer rate of 16,000 12-bit words per sec. It costs $6000 for the first 32k of capacity, plus $3000 for each additional 32k, including interfacing through the direct data channel. Larger and faster versions are available. Disks (or drums) should be important in future systems. Magnetic tapes of the IBM-compatible type are valuable, especially for communication with machines at computing centers, but tape drives and interfacing are usually expensive. It often costs $25,000 or more to get a single tape drive in service, although the next few are usually less expensive. The cheapest tape drives available cost about $5000. The cost of interfacing depends greatly on the particular computer. It may be as little as $5000, but it is often in the neighborhood of $15,000 or $20,000.

7. Program Input Method

Because they provide immediate access, the most satisfactory program storage media are magnetic disks and drums, followed by the IBM tape. The most satisfactory cheap device for input of programs is the high-speed, punched-tape reader, but the advantages of using small "cartridge-type" magnetic tapes have recently been emphasized. Recently, card readers have appeared which are much cheaper than the older IBM models. They can read 200-300 cards per minute. They cost about $2000 plus interfacing. Examples: Soroban, General Devices, Uptime.

A simple means of restoring the basic loader program (other than toggling!) is desirable. Many computers have this feature, e.g., the IBM 360 series; the SDS Sigma 2, Sigma 5, and 910 PDP-9.

8. Memory Protection

Hardware memory protection is necessary in multiprogram systems. It is very helpful in any machine with a batch-processing resident monitor and in other special situations.

9. Parity Check

This feature is useful for purposes as detecting memory failures, but it is usually not worth its cost in computer speed and in capital investment in the case of a small system.

10. Ease and Cost of Interfacing

This is a big subject, partly because the organization of computers for input and output of data varies with the manufacturer. Some computers such as the Hewlett-Packard and the DEC models are especially easy to interface, whereas the automatic channels of the SDS Sigma computers and the ordinary IBM machines (e.g., the 360 series) are very difficult. The IBM machines require an expensive control unit. It is said that before a competent engineer could order plug boards for Sigma interfacing he would have to study the system for a month or two. However, once interfaced, these machines permit rapid input of data. Interfacing a $5000 Calcomp plotter to the automatic channel of an IBM or Sigma series machine may cost much more than the cost of the plotter.

11. Typewriters

Many small computers use teletype machines as console typewriters. The ASR-33 teletype has not performed well, but it has recently been improved. The ASR-35 and KSR-35 have excellent records, and the newer ASR-37 and KSR-37 (15 characters/sec) are very good. The IBM Selectric has had a mixed reliability record which is, however, improving. In every case, expert routine maintenance is required.

12. Index Registers

These are a valuable asset to efficient programming. At least one, and preferably more, is desirable, especially in the medium and large computers.

13. Line Printers

These are of great use for obtaining a permanent ("hard copy") record, especially when large volumes of output are produced; however, they are expensive, usually costing $20,000 or more (including interfacing). In order to avoid tying up a large central processor during typewriter output of masses of data, a line printer is not only very useful, it is essential for efficient operation (and to spare the typewriter). A line printer can be immensely helpful and can save much time in the process of developing and debugging programs. The cost, however, will often preclude its addition to a modest system. If the system has an IBM-compatible tape drive, the computer output can be written on tape and later carried to a computing center for printing. Several industrial concerns are known to be working on new types of printers, some being dry-copy, nonpercussive types. One type which has already been marketed, the Inktronic printer, operates by spraying ink at the paper from small tubes. The characters are well formed. It operates at about 120 characters per second and costs $5600. Conveniently, it requires standard Teletype interfacing, and it can be ordered with an optional keyboard. Although it has exhibited a few new-product ailments in its first 8 months or so of use, it shows promise of becoming a very useful device. Another printer operating on a similar principle has just appeared—the A.B. Dick Company's Videojet printer, priced at about $4900.

14. Plotters

The overwhelming favorite is still the incremental machine called the Calcomp plotter. It costs about $5000 and is easily interfaced to many computers. It is very accurate (about 0.01 in. in each direction) and provides valuable output to the experimenter. It can be programmed to plot experimental points and theoretical curves together on white paper in India ink, relieving draftsmen of considerable work and doing a more precise job. Other incremental plotters are now on the market, e.g., the Houston Instruments version. Varian has developed an electrostatic plotter to sell for about $15,000.

15. Cathode-Ray Tube Displays

At least four types are in use. The standard scheme involves the displaying of bright spots under control of the computer, which has generated appropriate words to cause x and y deflections of the spot when those words have been transformed by ADC's in the CRT unit. The pattern is rewritten continuously. A light pen held against a particular part of the display pattern can be used to signal the computer. This scheme works well but may produce a flickering image if the computer is interrupted frequently to handle higher priority jobs or if the display is so complicated that the rewriting period exceeds 1/30 sec. The expensive hardware option called a character generator is considered not worthwhile unless large amounts of text are to be displayed. On a 10 in. x 10 in. raster a matrix of dots 1024 x 1024 is sensible.

A second scheme involves a disk or drum on which the computer writes the words to generate the pattern. Separate reading heads send the words to the CRT unit. Thus the display, automatically rewritten over and over, is updated from time to time by the computer. The light-spot cursor and joy-stick method replace the light pen in this case. (In passing, it is worth remarking that a light pen is only as effective as the computer program allows it to be, that the effort of programming for light-pen control is usually not trivial, and that a substantial amount of core storage may be required. A means of display control perhaps not so popular as it should be is sense-switch control.)

A third scheme makes use of a modern storage CRT. The computer sends the pattern to the CRT only once, and the display can persist until erased. This method is flicker-free and inexpensive, but the pattern is not so distinct and sometimes not so bright as in the above schemes. However, it is cheap. Furthermore, the storage tube can be used alternately as an ordinary CRT with quite satisfactory resolution. A storage version is thus possible which reverts to the standard scheme, for high-resolution inspection, when a button is pushed. The storage-tube scheme is probably the best buy for use in a typical small system. The Tektronix Company has recently announced a storage-tube device, Type 4501, which is said to generate a continuous video signal suitable for driving large-screen television monitors.

A fourth scheme involves the generation of a video (analog) signal corresponding to the display, written on a disk or drum by the computer. Reading heads then send the video information to a CRT having a TV raster synchronized with the rotation of the medium. This is a good scheme where many displays are needed, but it is too expensive for many applications, costing upwards of $20,000 for the first unit. (For example, the Data Disc System 6500 Display costs about $23,000.)

One display feature considered desirable by many nuclear physicists is rotation of isometric data plots. This can be accomplished in one of two ways: recomputing every displayed dot or using an appropriate analog device (potentiometer). Because the latter is so cheap, clearly its use is more desirable than the recomputation of the rotated view. Also, using a light pen on a recomputed display is especially difficult because the inverse computation has to be performed in order to maintain proper correlation with the original data. However, it should be noted that the TV raster technique is limited in this respect: rotating potentiometers cannot be used, and the image must be recomputed. The technology of displays is developing rapidly.

16. The Role of External Devices

In many cases, especially where typical standard operations are involved, it is preferable to use external devices to handle preliminary selection and sorting of events, rather than to ask the computer to do the entire job. For example, particle identification by use of signals from two counters involves one or two multiplications and additions, which can be carried out almost instantly by a fairly simple external analog device, whereas a small computer would likely require at least 500 µsec for the job, assuming calculation, and perhaps 40 µsec, assuming table look-up.

17. Time Sharing

Computers as small as a PDP-8 have been successfully time-shared by several users in special applications. The justification given is that all the peripheral hardware can be shared also, so that the added constraints and programming difficulties are balanced by savings in hardware costs. Computers have also been shared for simultaneous on-line data-taking in low-data-rate experiments. In working out the economics of time-sharing, the added hardware (such as CRT's and remote consoles and memory protection) needed to allow simultaneous access by more than one user, as well as the extra memory space needed by the time-sharing monitor, should be considered. The greatest costs, however, lie in the added constraints placed on each of the users and in the greatly increased cost of programming. In many cases the use of two or more identical computers is preferable. However, in large, expensive systems time-sharing can be very useful.

18. Software That Should Be Supplied by Manufacturer

Complete documentation should be provided, including listings, step-by-step user instructions, and some fully worked out examples.

a. Hardware diagnostic routines: To test memory addressing, instruction set and to test correct operation of every peripheral and special hardware feature.

b. Systems to edit, assemble, and debug programs in symbolic machine language: These should efficiently use any special I/O device such as magnetic tape, disk, or line printer.

c. Efficient subroutines should be provided for operation of any special peripheral device purchased from the computer manufacturer. Symbolic language source tapes or card decks, listings with comments, and examples of use should be included.

d. Conversational Fortran-type programs provided by some manufactures are useful for supplemental calculations.

NOTE: The following points apply particularly to the medium and large machines and become increasingly important as the computer becomes larger and more complex.

e. Fortran compiler and operating system, with convenient method to insert machine language instructions and subroutines. Good compile and run-time diagnostics are essential.

f. Mathematical subroutines should be provided in binary and source language.

g. Complete specifications and documentation for the programming system should be supplied, so that programs prepared by users can be made compatible. It may be objected that this will cost too much, but not to do so will be very costly and frustrating to many users.

19. Note on the Cost of Programming

Experience at Brookhaven and Berkeley has shown that a programmer can produce between 10 and 20 debugged and documented lines of program per day, depending on such factors as experience, when he is working on reasonably straightforward programming. When working on a complicated monitor system he would be considerably less productive. System programming is obviously very expensive, therefore the average person exploring the computer market would be well advised to consider the software support along with the hardware offered in each case. Manufacturers vary greatly in this respect. A major contributing factor to the persistent popularity of the PDP-8 is that the software support is so extensive. In general, the newer a computer, the less software is likely to be available.

[Chapter 2]
DATA-ACQUISITION SYSTEMS

A. INTRODUCTION

1. History

The movement toward computer systems began in earnest about 1962. Much of the early work depended on the use of magnetic tape for storage of data, either raw or partially digested, the analysis of data being carried out later, off-line. More recently, computers have been used increasingly for on-line processing. The early work is well known and will not be described here. Some of the more recent systems are basically very close descendants of one or another of the early systems. Many varieties are now in service. Most incorporate small or medium-sized computers, however, extensive new experience has been gained during the past two or three years of operation of a few large time-shared systems, in particular those in the tandem Van de Graaff accelerator laboratories at Yale and at Rochester, perhaps the first large systems in operation which were planned systematically for nuclear research. Both operate with multiprogramming monitor control, background calculations being possible, on a low-priority basis, simultaneously with data acquisition.

2. Possible Systems

Simple rules for the design of various types of data-acquisition systems cannot be stated, but some examples of possible systems can be given. ([See Figure 1.])

a. A simple system for pulse-height analysis work can be assembled from a small computer, a 5-in. Tektronix CRO, an ADC unit, and a teletype with paper-tape attachment for a cost of about $30,000, providing that a competent engineer is available, not counting programming and engineering costs. A Calcomp plotter could be added for about $6000. To maintain and operate the system at least a half-time technician-programmer would be required.

FIGURE 1 Basic data-acquisition system.

b. A general-purpose system for use in an accelerator laboratory can be assembled from a medium-sized computer, two typewriters, four 13-bit (0-8192) ADC's, six 18-bit (0-262,144) counters, a 10 in. x 10 in. CRO display unit with light pen, two tape drives (for IBM tape), a Calcomp plotter, and a fast paper-tape reader for about $175,000 plus the cost of engineering service and programming. At least one full-time technician-programmer would be needed for maintenance and programming.

c. A large shared-time system of the smallest configuration which makes much sense consists of a large computer with a 16k memory, two typewriters, a fast punched-tape reader, four DEC tapes (or the equivalent), one IBM-compatible magnetic tape, one CRO with light pen, one incremental plotter, input devices for experimental data (ADC's, counters, etc.), plus an interfacing system to link the external input-output devices to the computer. The interfacing system may include a fixed-wired "front end," such as that used at Yale, a small computer, such as that used at Rochester, or both. The hardware would probably cost over $400,000 exclusive of engineering, and to this must be added a large expense for programming, even if the manufacturer supplies a satisfactory shared-time monitor plus all the usual software. Three men would be needed to assemble, maintain, and operate the system: an engineer, technician-programmer, and a full-time programmer, or some equivalent combination, assuming use of the system in a large laboratory with an active and continually developing research program. Thus the cost of this "stripped-down" system must be expected to reach $500,000 before it is in full operation, and the cost of keeping it going, including salaries, overhead, and replacement parts will likely exceed $50,000 per year, although this could perhaps be trimmed somewhat once the system is running. Furthermore, to run efficiently, the system would need additional components: another 16k (at least) of core memory, another IBM-compatible magnetic-tape drive, and a line printer. A rotating memory device would also be helpful. These would raise the cost by well over $100,000. It is apparent that large time-shared systems are so expensive that they can ordinarily be justified only in the largest, most lively research establishments.

3. Small Computers as Satellites

In medium and large systems the use of small computers for coupling input and output devices to the main computer offers a number of attractive advantages, especially now that mass production and competition have brought the prices down so low that a large amount of hardware nearly ideal for the purpose is available at a bargain. Some advantages: (1) The small machine can control data acquisition, accumulating blocks of data while the large machine is doing background calculations, interrupting those calculations only occasionally to transfer raw or partially processed data. (2) The small machine can continuously control the monitor CRO. (3) It can control output devices such as a plotter, line printer, rotating memory, or tape drive. (4) It can carry out many logic operations on the incoming data. Experience has shown that such operations are numerous, and from the economic point of view they should not be allowed to tie up the larger machine, which, at the same time, can better be engaged in complicated calculations. In some cases the use of two small satellite computers can easily be justified. The chief disadvantage: Programming can be complicated. However, if the larger machine already has a time-shared monitor which recognizes the small machine as a typical input-output device (as is the case with the PDP-6 plus PDP-8 system at Rochester) the programming problem is not bad.

In the following five sections descriptions of a number of data-acquisition systems of various types and sizes will be given in order to illustrate concretely some practical system configurations. In each case a breakdown of costs and a discussion of the lessons learned in connection with planning, construction, and operation will be included. The systems are of the following types: two small, one medium, one large, two multiple-CPU, and one process control.

B. A SMALL TIME-SHARED DATA-ACQUISITION SYSTEM BASED ON A PDP-7 COMPUTER

1. Introduction

In 1966, two identical computer systems based on PDP-7 computers were set up in two different locations at the Lawrence Radiation Laboratory (LRL), to be used by several groups of experimenters (see Figure 2). Assembly of the hardware for the first system was completed 6 months after delivery of the computer. Assembly of the second system required only 3 months. Two years after operation commenced, the first satisfactory time-sharing monitor was completed and put into service. The basic use of these systems is pulse-height analysis. In principle many other types of operation are possible.

FIGURE 2 PDP-7 data-acquisition system at Lawrence Radiation Laboratory

FIGURE 3 A switch panel used for data taking and control of CRT display in conjunction with the PDP-7 computer. The switch-setting codes can be read into the PDP-7 accumulator under program control and are used to select branch points in the program. As many as eight of these units can be connected to the system. The lights are used to indicate program status.

2. Operational Features

Data-reduction jobs currently possible in the shared-time operating mode include spectrum stripping, normalization, smoothing, storage and retrieval of data from magnetic tapes, graph plotting, printout, energy calibration, background fitting, peak integration, and transfer of data from a remote analyzer. Remote control of the computer from up to eight experimental locations is possible using inexpensive switch panels (Figure 3). Remote slave CRT display is also provided. Multiparameter pulse-height acquisition and analysis can be done on a time-shared basis but often requires all the computer's time and memory.

3. Hardware

The hardware configuration is shown in Figure 2. The PDP-7 computer was supplied by the Digital Equipment Corporation with an 8k memory (18 bits) extended arithmetic hardware, microtape (Dectape), paper tape, teletype, and cathode-ray tube (CRT). The other items were built or interfaced at LRL.

Automatic memory increment and memory-protection hardware, together with suitable programming, allow a user to carry out simple data-reduction jobs with a live CRT display while two other users are independently acquiring separate, 2048-channel, pulse-height spectra in part of the computer memory, with computer-controlled-gain stabilization. ADC dead time per pulse is less that 40 µsec. Up To 6144 words of the memory can be used for data (one PHA channel per word), while machine language programs fill the remaining 2048 words of memory.

4. Lessons Learned from Operating Experience

The system works well for pulse-height analysis, but for new applications, e.g., nuclear magnetic resonance magnet control, it needs additional hardware and programs.

Two groups of experimenters, doing chiefly pulse-height analysis experiments are very satisfied with the system. Another group, with a wider range of interests, has been dissatisfied because of the time lag to implement new experiments.

One programmer is now engaged full time preparing more programs.

Lack of free computer time has become a limitation for both users and programmers.

Provision for programming at the outset was inadequate. One full-time systems programmer should have been assigned to these systems for 18 months.

Experimenters need Fortran or similar language capability. A disk, or more core memory, would make this practical.

The memory size is totally inadequate for multiple users because of the large amount of data space needed for the high-resolution spectra now obtainable with Ge(Li) detectors. An external 8k memory is being acquired for data acquisition in each system so that more of the computer memory can be used for computing.

Memory CRT's are needed to provide independent displays for each user. A separate teletype for each user would be invaluable.

A disk memory is needed for rapid overlay of programs and for sorting of multiparameter data.

A "czar" should have been appointed for day-to-day assignment of facilities, consultations with users, and routine maintenance and upkeep of the hardware and programs. The "czar" could be a good electronics technician interested both in programming and in physics.

5. Costs

System costs for the PDP-7 with time-sharing are given in Table 1. Fabrication time is included as a dollar cost. Engineering and programming times shown are one half those charged against two identical systems.

Additional special-purpose experimental equipment commonly used with the system includes gain stabilizers, analog pulse derandomizers, amplifiers, pulse pileup rejection, low-noise preamplifiers, and Ge(Li) detectors.

C. A SMALL SYSTEM BASED ON A PDP-8 COMPUTER

1. History and Hardware

This second example of a small computer system is also taken from experience at LRL. It was planned in February 1967 and first put into operation in the summer of 1967. Data were first taken with the aid of the system in the spring of 1968, and the system programming was completed in May 1969. The system is used extensively in experiments with the Bevatron.

FIGURE 4  PDP-8 data-acquisition system at Lawrence Radiation Laboratory.

The computer-system hardware consists of the items shown in Figure 4. The PDP-8 has 4k of memory. The disk is a Data Disc unit with a removable disk (128 tracks and a movable head) on the same shaft as a smaller disk and three fixed data heads. Two of the fixed-head tracks are devoted to the display: they drive a hardware-translator continuously. A single display track is used when the number of points does not exceed 2000. For larger displays two tracks are used alternately.

The display is controlled from the switch panel. On-line operating functions may be controlled both from the switch panel and from the teletype.

2. Programming

Three classes of programming have been completed:

System Programs: Symbolic Text Editor, Assembler, a general-purpose library system—all disk oriented.

Data-Taking Programs: These cause the ADC to be read, control elementary sorting, update histograms resident on the disk, write raw data on tape, and monitor the beam. The Bevatron has approximately 1 sec of beam every 6 sec. During a beam pulse the computer is devoted entirely to acquiring data, saving raw data in core, on the disk, and on tape. After a beam pulse, the 13-disk histograms are updated, then the display programs are read into core memory and the display is updated. The system is designed to be capable of accepting over 1000 events per beam burst, and it has met this requirement.

Simple Data-Analysis Programs: These compute displays (linear, log, isometric, and contour—all double precision), read out the sealers, monitor the real-time clock, allow resorting of raw data from tape, and generate tapes for remote plotting.

3. Lessons from Operating Experience

The system now functions as originally intended and does its job very satisfactorily.

The experimenter relies heavily on the main computer center for data processing.

In assembling this system now, one would buy the disk already interfaced by the computer manufacturer; furthermore the manufacturer now offers programs that would greatly reduce the programming costs.

A memory scope would eliminate the need for a disk-to-CRT display interface.

Less-expensive magnetic tapes are now available. However, it would be better to buy the tape already interfaced by the manufacturer of the computer.

The added cost of buying a 16-or 18-bit computer would have been almost completely offset by savings in the cost of programming.

The addition of a fast printer (e.g., Inktronic $5600) would have paid for itself in time saved during programming but would not be of much use in experiment.

4. Costs

The costs of the PDP-8 are given in Table 2.

FIGURE 5 Block diagram of EMR 6130 data-acquisition system at Columbia University.

D. A MEDIUM-SIZED ON-LINE COMPUTER SYSTEM

1. Introduction

An EMR 6130 computer system has been installed and is being prepared for use with Columbia University's Neutron Velocity Spectrometer data-acquisition and analysis system. The spectrometer is characterized by high data rates and many events per burst. At present, peak arrival rates are approximately 10⁶ events per second, with 40-60 events per burst and a burst rate of 70 Hz. The arrival distribution is random; therefore, 10 percent of the interarrival intervals are 100 nsec long, and 1 percent are 10 nsec long. In the future, peak arrival rates of 10⁷ events/sec and 400-600 events per burst are possible, with a burst rate of 300 Hz. With an appropriate time-of-flight "front end," the 6130 will be able to handle the anticipated faster rates.

The EMR 6130 is a 16-bit, 775-nsec computer. The memory has a multibus structure which permits each bus to communicate simultaneously with a separate memory module. Up to four memory buses may be purchased. The Columbia system has two memory buses. If a high-speed buffered data channel is used, block transfer may occur at memory cycle speeds. With two buses, data may be stored in two memory modules at rates up to twice memory speed. Alternatively, one bus, channel, and one or more memory modules may be dedicated to data acquisition, while the central processor and standard peripheral devices, using the second bus, simultaneously operate in the remaining memory modules.

2. Description of the System

A block diagram of the Columbia system is given in Figure 5. The system has three 8k core modules. Memory bus 1 is dedicated to a high-speed channel serving the time-of-flight acquisition system. Memory bus 2 serves both the central processor and a second high-speed channel. Low-speed input-output devices, such as the operator's console, teletype, card reader, and plotter communicate directly through the processor. The high-speed input-output devices, namely, a magnetic tape unit, line printer, fixed head disk, and interactive CRT display, communicate through the channel.

The box designated as "time-of-flight system" represents special-purpose electronics, including a 50-mHz clock, time-quantizing circuits which "clock" an input event from one of the detectors to the nearest clock pulse following its arrival, a 50-mHz counter, and a 16-word derandomizing buffer capable of storing a new word of data (i.e., arrival time) every 20 nsec. The number of channels, nominally 16,000, is limited not by the front end but by the amount of core available for histogram storage in the 6130 system. (For the high data rates anticipated in the future, the time-of-flight clock speed and derandomizing buffer data acceptance rate will be increased to 100 mHz. At the same time, an accumulating buffer of several hundred words capacity, with a 20-mHz data acceptance rate, will be added to empty the derandomizing buffer and store temporarily the time-of-arrival data prior to its transmission to the 6130 system.)

FIGURE 6 Diagram illustrating mode of utilization of core memory in the Columbia System shown in Figure 5.

3. How the System is Used

During the time-of-flight experiment, memory is utilized as follows (see Figure 6). The channel dedicated to data acquisition writes on alternate bursts, into two buffer regions, of approximately 100 words each, in the top of memory module 3. The remaining parts of memory module 3 and all module 2 will be devoted to histogram storage (i.e., time-of-flight channels). Module 1 will contain a stripped-down monitor program and all data-handling programs, including buffer regions for the external devices other than the time-of-flight front end. Programs will be capable of referring to all module 1 or 2 in full concurrency with data acquisition. Reference to module 3 will also overlap data acquisition, except for a period of high input data rate of 100-to 200-µsec duration per burst. With the type of memory allocation described, the system will permit the use of all standard I/O devices, concurrent with the essential operations of input data buffering and histogram generation. Thus, new data may be stored on, or old data retrieved from, the disk or magnetic tape; either new or old data may be displayed on the CRT; and the same or other data may be output with the plotter or line printer. Control information will be input from the teletype, the operator's console, or from special-purpose switches. The importance to the physicist is that hard copy output is immediately available during data acquisition and may be used to monitor, or modify, the experiment.

Subsequent to the input data increase, a high-speed memory incrementing channel will be used to input time-of-flight data directly to the histogram area. With this channel the buffer area in module 3 will no longer be required. Histogram data will be stored in all modules 2 and 3, and no program intervention will be required for histogram generation.

Between data-acquisition runs, the system will be used for data analysis.

4. Present Status

The computer, with two memory modules and one channel and bus, was delivered in July 1968. The remaining memory module channel and bus were delivered in the fall of 1968, the CRT arrived in June 1969, and the line printer (which was not purchased from EMR) came shortly afterward. The first time-of-flight run with this system was scheduled for December 1969. During the period from delivery to the first run, one full-time programmer and approximately half the time of one physicist were devoted to the debugging of manufacturer-supplied programs and the writing of the on-line programs required for the run.

It has been hoped that the system would be used extensively for the analysis of previously acquired data, beginning shortly after delivery; however, very little such use has proved possible, essentially because of the unreliability of the 100-cpm card reader supplied by EMR. The lack of a line printer was also a factor. A more reliable reader has been purchased. The delivery of a line printer should rectify the second need.

The development of high-speed, buffered, time-of-flight front ends has been a continuing interest at Columbia. It is therefore difficult to estimate the precise costs of the time-of-flight system developed for use with the 6130. A rough estimate of the design and development time is approximately 3 engineer man years.

5. Lessons from Development and Testing Experience

Columbia chose to order the EMR 6130, even though at that time (1966) it was not in production, because it seemed a very powerful machine which matched the needs of the system planners. The alternate possibility open was to order a larger, much more expensive machine of proven capability. As it turned out, difficulties in the development of the 6130 caused a delay of over a year in the delivery of the main frame and of over two years in the delivery of the CRT display. (When these delays became apparent, EMR loaned Columbia a 24-bit computer and also a small display for use during the interim period.) The EMR 6130 is perhaps the most powerful 16-bit computer available today, in spite of one or two changes in the original specifications, but in order to get it Columbia apparently traded time for money.

6. Cost

The costs of the Columbia University EMR 6130 system are given in Table 3.

E. A LARGE SYSTEM BASED ON A SINGLE COMPUTER (THE YALE-IBM NUCLEAR-DATA-ACQUISITION SYSTEM)

1. Introduction

Since early 1966, Yale and IBM Research have been engaged in a joint study in the application of computers to nuclear-data acquisition. The main goal was the production of an integrated hardware-software system which is fully under the control of the experimenter in the sense that he can define his entire data acquisition and analysis process with a Fortran program.

The joint study may be divided into four areas: (1) Development of a suitable general and powerful data-acquisition interface and control unit (front end) with a set of compatible nuclear instrumentation modules (scalers, ADC's, and general-purpose input registers). (2) Development of a suitable display system. (3) Development of a data-acquisition language (as an extension to Fortran) and the necessary library routines to support this language. (4) Development of a general-purpose multiprogramming system for the selected computer (the IBM System/360, Model 44) into which the data-acquisition system could be incorporated.

The first three areas became operational in July 1967, within three months after delivery of the computer, using the standard batch programming system for the 360/44 as a basis. Development continues on the multiprogramming system, which has now reached a state where users inexperienced in using the system can compile and execute Fortran programs, but the data-acquisition components are not yet operational.

2. Description of the System

Nuclear data are input by means of a general-purpose nuclear-data-acquisition interface and control unit, organized around the concept of an event, an occurrence in the real world which causes the outputs of a group of instruments selected by the user to be read into computer memory. Sixteen independent events are provided for, with each of which may be associated any or all of 30 different instruments (scalers, ADC's, or general-purpose monitor registers) by means of a diode plug board matrix. The instruments themselves, also designed and built by IBM, are modular and completely interchangeable and enable the experimenter to configure his experiment in any way desired, i.e., to determine not only which instruments are to be read but also in what order. Exclusion logic is provided to prevent processing of certain events if and when other defined events occur simultaneously (Figure 7).

The cathode-ray-tube display unit provides a 1024 x 1024 point plotting oscilloscope with seven levels of intensity, character-generation hardware, a light pen, and a programmed function keyboard, by means of which the user can call in programs by pushing buttons. Such programs can perform any function from changing displays gains to curve-fitting. They may be system-supplied or user-written and may be (and usually are) written Fortran. A parallel, high-resolution photographic system permits computer assembly of publication quality illustrations.

The data-acquisition and display-programming system is composed of a group of subroutines which may be called from Fortran programs for performing the various processes in data acquisition and display. For this purpose, a considerable number of additional statements have been added to the Fortran language. These statements perform such functions as defining multidimensional pulse-height analyzers in the computer memory, performing pulse-height analysis using incoming data as channel numbers, and defining separate programs to process each of the 16 classes of input events. All the special statements that make up the new data-acquisition language are implemented by means of a preprocessor which converts them into Fortran coding, which the standard 360/44 Fortran compiler then processes.

FIGURE 7 Block diagram of the IBM 360/44 system at Yale.

3. Software System

The general-purpose multiprogramming operating system is a multilevel priority system designed to provide access to the system simultaneously by an, in principle, unlimited number of users, each with unique priority. Unlimited means that there is no arbitrary restriction on the number of users; the nth user can always get access if the facilities his particular job requires are not already in use. Two types of user are recognized by the system: The basic unit of execution is the logical user, or task. Each logical user has a unique priority level. Switching between users is carried out as a response to I/O, timer, or external interrupts, at which time the highest priority user in a position to execute gains control of the central processor. The basic unit of memory protection is the physical user, composed of one or more logical users engaged in a common cause. Physical users correspond to real people doing independent work simultaneously. By dividing his work up among a group of logical users, a person may take advantage of the parallel processing capabilities of the system in a natural way. Since logical users within a physical user are not memory-protected against each other, they may communicate rapidly, at full machine speed. Communication between different physical users is also possible, via real or simulated I/O devices. While this system by no means guarantees execution time to any but the highest priority user, it is adequate in a single-experiment environment. The assumption is that the experimenter, who has actual control of the computer at all times, loads his logical users in the order in which he requires their priorities. Following this, other users load their jobs, getting whatever memory and I/O facilities remain. The amount of processor time available to the other users varies inversely with the experimenter's counting rate and the amount of processing he does on his data. In most experiments, the experimenter uses significantly less than 50 percent of the processor time simply because those experiments requiring the sophistication of the computer also have rather low counting rates. A priori, it is estimated that the simple priority algorithm described above is not only adequate but pays a dividend in terms of reduced system overhead time as compared to a more elaborate algorithm. It also guarantees that no data will be lost due to the lower priority users being in the machine.

In general the new system will provide all the facilities of 44PS within the multiprogramming framework, including execution of the Fortran compiler simultaneously with data acquisition.

4. Costs

To the laboratory interested in developing a system of the magnitude of the Yale system, but not a copy, it must be reiterated that neither Yale's out-of-pocket costs nor the cost of copying the system represents the total cost of development. IBM's development costs are not known, but they may be assumed to be very large.

From IBM's viewpoint, the ADC and scaler project is the least successful part of the whole project. Although those instruments are technically excellent, IBM is either unwilling or unable to sell them at a price competitive with the costs of front-end and interface equipment available from the traditional nuclear instrument manufacturers. However, ADC's and scalers available from the traditional sources can easily be interfaced to the front end (whose price is in keeping with its power and versatility). The University of Maryland has followed this procedure.

We therefore present the cost of copying the Yale system by some other laboratory. In Tables 4 and 5 following the example of Maryland, we have not selected IBM ADC's and scalers but rather less expensive components from traditional manufacturers, together with suitable adapters available commercially. The prices shown are to be considered strictly reference numbers and in no way constitute price quotations.

5. General Comments on Experience with the System

Starting by producing an operational data-acquisition software system running within the standard batch programming system for the 360/44 enabled the system to become operational within three months of the delivery of the computer. This not only enabled it to do useful work almost immediately but also enabled important experience to be gained which is being applied to the development of the multiprogrammed version.

One of the main lessons so far is that a batch-oriented system barely begins to tap the real-time potentials of a computer such as the 360/44. In a batch system, whatever analysis is needed during data acquisition must be somehow tied to the processing of events. If this is not possible, it is necessary to stop data acquisition in order to do analysis even though, on a millisecond time scale, plenty of CPU time is available during acquisition. Multiprogramming software is necessary in order to utilize this available time. This means that multiprogramming not only makes the machine available to several people at a time, but, more important, it makes large amounts of parallel processing power available to the experimenter.

It has also been shown quite conclusively that the ability of the physicist to program his own experiment (in Fortran) gives him enormous power, power which simply would not be available on a suitable time scale if he has to queue up for the services of a system programmer.

While the generalized event structure gives the experimenter considerable ability to deal with complex experimental situations, it has an overhead associated with it which limits it to about 5000 events per second. This is, of course, adequate for all experiments that demand such an event structure. For simple pulse-height analysis, it is unnecessary overhead, but it can be "turned off" in a trivial way, by simply defining the completion of filling of the buffer as an event and calling a special pulse-height-analysis program to process the entire buffer, bypassing the event sorting. This allows for close to 100,000 pulse-height analyses per second.

There are, however, few situations that justify using a computer as powerful as the 360/44 in a manner just described (i.e., doing nothing but simple addition 100,000 times per second). Therefore, such simple experiments will shortly be handled by means of a link between an existing multichannel analyzer and the computer.

The system does not suffer from having the front end directly connected to the 360/44. The data channel on the 360/44 is sufficiently sophisticated so that it performs all the functions that one might relegate to a small CPU placed between the front end and the 360/44, without any interference with the program currently running in the CPU. The one application described above, which does warrant a separate processor, is handled best by attaching the processor to a separate input port rather than by placing it between the front end and the computer. This enables it to do its intended job without acting as a bottleneck in jobs requiring the power of the front end. It also, incidentally, will function as a completely separate data input terminal if two simultaneous terminals should ever be required.

The particular display system employed has worked very well. Because the display list is in the main memory of the computer, programming of light-pen and other manipulative actions is extremely easy, but at the price of large amounts of memory being tied up. It is clear that the system cannot support two such terminals if they are to be truly independent of each other. It is equally clear that the display is as useful in data analysis as it is in data acquisition. A second display terminal is therefore being added. The selected unit (built by Computer Displays, Inc.) is oriented around the Tektronix 611 storage oscilloscope. It provides both alphameric and graphic display, as well as an interactive device (a cursor moved by means of a joystick) for a price of $8000 to $12,000 plus the cost of interfacing to the computer.

F. MULTIPLE-COMPUTER SYSTEMS

1. Introduction

At the Rutgers-Bell (RB) nuclear physics laboratory, work has been done with two different two-CPU systems. The first of these represented essentially two duplicate processors (Figure 8), and the second, now in the process of implementation, two processors of different size and capability (Figure 9). While full data are not yet available on the actual performance of the second system, an outline of the experience to date will be given.

FIGURE 8 The two-central-processor system of Rutgers-Bell.

2. Two Equivalent Processors

The initial success of the original RB SDS 910 data-acquisition system was soon tempered by a result of its popularity: during most experiments the computer was unavailable for program development or data analysis. Since most experiments required the use of displays and light pens in at least one stage of data analysis, the computer center could not handle the work.

FIGURE 9 The new Rutgers-Bell Sigma 2-Sigma 5 system.

The solution adopted was to acquire another computer with the same instruction set (an SDS 925) and to provide switches such that the line printer, card reader, and plotter could be run from either computer. No provision was made for direct transfer of data from one computer to the other.

3. Lessons from Operating Experience

In practice this system worked out quite well. There was complete interchangeability of programs from the 910 to the 925, which differed only in being five times faster. Normally the switchable peripherals were run from the 925; when the group taking data wished to print or plot current spectra, they consulted with the 925 users, then used the peripherals with little more difficulty than permanently attached units would have involved.

A further advantage of the switchable peripherals, in addition to the cost saving, was that the experiments associated with the 910 could proceed while the peripherals were being serviced. The 910 is exceedingly reliable, averaging less that one main frame failure per year, and the 925 is nearly as reliable. The vast majority of service calls have been occasioned by the peripherals and have competed with data analysis but not with accelerator utilization.

In addition to the switched peripherals, both computers were equipped with two magnetic tape transports, electric typewriter, and high-speed paper-tape reader and punch. While these units were also subject to downtime, the paper-tape system and the typewriter could be exchanged between the 910 and 925. Only the magnetic-tape transports required the use of the 910 CPU during servicing, and the presence of two transports has usually meant that the second one could carry the load until the weekly accelerator maintenance period.

While the reliability record of the central processors has been excellent, that of many of the peripherals has not. Here is an excellent justification for renting computing equipment: if units do not work well, they can be returned. For a time, a low-cost card reader (100 cards per minute) built by NCR for SDS was used. It was unacceptable in reliability and was replaced by the Univac reader which came with the 925. Another unit returned was a cartridge magnetic-tape system built by SDS. The Ampex TM-4 magnetic-tape transports on both the 910 and 925 have been consistently poor in reliability, but no other unit has been available to replace them. A manufacturer's name does not seem to be a guarantee of good or bad quality—the line printer, also made by NCR, has been excellent both in reliability and print quality.

4. Limitations on a Twin-Computer System

While the two-computer system generally rated high in user satisfaction, considerations of performance have led to the design of a larger and more powerful system with totally new components. The 925, without wired multiplication or floating-point operations, was too slow for theoretical computation or for many types of data analysis such as those using Monte Carlo methods. Interactive methods of analysis, using a display and light pen, have been found very effective in the cases where the 925 could accommodate them but have not been available through either the Bell Laboratories or Rutgers computer centers.

A further limitation on the earlier system was that only one person could use the 925 at a time. The generation of a display involved the full time of the CPU, and while multiprogramming might have been able to divert some CPU time, the 8k memory size did not permit it.

Data acquisition on the 910 was limited in array size to the capacity of the core memory. For multiparameter experiments, three, six, or even twelve 4096-channel arrays have been stored in core, but the advantages of live display available with core storage have discouraged anyone from handling large arrays by logging raw data on magnetic tape for analysis later. Memory expansion would have been desirable, but the necessity of making the expansion on both the 910 and the 925 effectively doubled the cost.

Limited flexibility, then, is a major drawback of this type of system. As long as only two users needed to be accommodated, and each could adapt to exactly half of the total core storage, it was satisfactory and provided redundant facilities to guard against experiment downtime due to computer failures.

5. New Directions

In ordering a new computer powerful enough to handle most of the nuclear physics laboratory's data analysis and theoretical computing tasks, cost ruled out the acquisition of a pair of program-compatible computers. It was recognized that desirable features of the original system would have to be obtained in new ways. Accessibility of the system for programming could be improved by running a simple time-sharing monitor on it. Reliability could be enhanced by avoiding bargain peripherals and using only items of demonstrated high quality and by the capability of running the peripherals on either computer.

The use of a separate CPU for data collection still seemed particularly desirable, however. A combination of a large (by present standards) computer with a powerful small computer as a front end was designed. It includes a display disk for refreshing displays without CPU attention, as well as for storing data arrays too large to be kept in core. The computers selected were a 32k, 32-bit SDS Sigma 5 and a 12k, 16-bit Sigma 2.

The new system, with separate and nonequivalent computers, will have advantages over the old system in data analysis and general computation, because these will be done on the larger computer, either in time sharing or batch mode. Time sharing should enhance the flexibility of the system by making it easier to generate and debug new programs, in addition to improving the accessibility.

For the data-collection computer, RB will lose the advantage of a separate computer on which complete debugging of programs may be done. This loss can be tolerated since the fraction of the load carried by the Sigma 2 will be less than that carried by the 910 in the old system. In the old system, very few distinct data input or display programs were written. A few subroutines and their calling parameters sufficed for all needs for six years; the logic and I/O operations unique to each experiment were written in Fortran by the experimenters.

In the new system, the Sigma 2 will be concerned with the operations used in the data acquisition and formating of displays; most of the rest can be left in the Sigma 5, with routines sent over to the Sigma 2 under the time-sharing system. If the user should prefer, he can operate the Sigma 2 directly and make use of the Sigma 5 only for data storage.

Until very recently, program development on the Sigma 2 has been slow because it lacked means of getting program listings quickly. We have now developed an assembler for the Sigma 2 which runs on the Sigma 5. The availability of card reader input and line printer output has greatly speeded Sigma 2 software development. The loading of Sigma 2 programs is also much more convenient, since they can be stored on the Sigma 5 disk and loaded exactly as Sigma 5 programs. It seems highly desirable to have an assembler for any small data-acquisition computer capable of running on another machine; the means of transporting the object code to the small computer is of less importance.

The reliability of the new equipment has been excellent. Only the card reader has had any downtime of consequence, and modifications seem to have resolved its problems. The Sigma 5 main frame has had no failures in 12 months, and the Sigma 2 has had only one in the past year. If this record continues, the loss of the redundancy inherent in the old 910/925 system will not have any serious effects.

6. Computer-Independent Data Bus System

One component of the new system is taking on an increasingly important role, although it had not been a part of the original planning. That is the computer-independent data bus consisting of system controller, bin controller, and register units. Only the system controller is specific to a particular computer; moreover the same system controller design could be used on both the Sigma 2 and Sigma 5 by restricting the data path to 16 bits. The register units are used to interface external devices to the computer quite cheaply; a typical register used here to interface an existing Calcomp plotter to the new computers costs about $300 in parts and labor. Similar units are used to interface the Sigma 5 to the Sigma 2 and to the 910, to drive a temporary core-resident display on the Sigma 5, to read pushbutton inputs, and to read ADC's. The display disk controller now under construction uses these registers to furnish control information, although the data go directly to and from the core.

At the present time, the registers are read and written under program interrupt control, but the design is not limited to program-controlled operation. By substituting a controller designed to operate automatically (directly to core or to the I/O processor) speeds approaching 1 or 2 µsec per word transferred could be obtained. Such interfaces have been built for various computers using the European CAMAC bus system, which is conceptually similar.

The system is highly modular and is built into NIM bins with modified back connectors. Exchange of modular units has been very helpful in debugging the system, and presumably it will also be helpful in case of failures in operation. This is a much more satisfactory situation than that which was obtained with the ADC interface on which RB collaborated with Brookhaven. The latter unit was built with computer-type construction: commercial logic cards and wire-wrapped back panel. Debugging of that unit was exceedingly laborious because of the lack of modularity in its components.

The computer-independent bus system has not been expensive in manpower. It has required about 9 man-months in design and debugging and somewhat less time in construction. The registers cost about $300, as mentioned, and the controllers $1500 to $2000 depending on the need for cable drivers.

7. Costs

The costs of the RB multiple-computer system are given in Table 6. The figures are approximate and not the result of detailed accounting.

G. A PROCESS-CONTROL SYSTEM: THE BROOKHAVEN MULTIPLE SPECTROMETER CONTROL SYSTEM (MSCS)

1. Introduction

In 1965, a system based on an SDS 920 computer was put into operation at the Brookhaven National Laboratory to control data-acquisition processes involving eight neutron spectrometers and one x-ray spectrometer. The neutron spectrometers are located on the floor surrounding the High Flux Beam Reactor (HFBR); the x-ray spectrometer was placed in the same building in order to facilitate linking it to the computer. The system can control the execution of experiments on all nine sets of apparatus simultaneously, yet each experimenter feels that he is working essentially independently of all other users. The system controls all angular rotations of crystals and counters, all detector counting, the data displays, the input and output operations, and automatic error responses.

It can also perform most of the calculations necessary for real-time guidance of the course of the experiments. For example, the experimenter can mount a crystal on a goniometer at approximately the correct angular orientation, then he can specify to the computer where several peaks should be found, whereupon the computer will direct the execution of a trial experiment to find where the peaks do, in fact, occur, executing least-squares calculations in the process, after which the error in crystal orientation is known and the angular scales are automatically corrected. In another example, the computer is given as input information the crystal constants (unit cell) and the zone orientation of the crystal on the goniometer and is asked to produce a scanning of a given part of reciprocal space. The computer then calculates where to look, turns to a correct angle to check the intensity of a central peak, and performs the other necessary steps, making many decisions as it controls the execution of the entire experiment.

2. Description of System