D. C. Engelbart
3 March 1961
SRI Confidential

ROUGH-FORM THINK PIECE REGARDING RESEARCH POSSIBILITIES AT SRI IN AREAS OF MAN-MACHINE COMMUNICATION MEANS, AND AUTOMATED PSYCO-MOTOR SKILL TRAINING

I INTRODUCTION

In line with an overall program we are developing at SRI, we have isolated the research plans listed below as being those which would (1) develop immediate byproduct techniques directly useful to industry and military and (2) provide the sort of techniques, facilities, and experience here at SRI that are very nicely geared to the longer-range needs of the program (the "program" in question is described in Proposal EU 60-251, entitled "Augmented Human Intellect Study" and submitted to AFOSR on 13 December 1960, resulting contract begun 1 March 1961).

The work described below has several facets that could, in fact, be developed quite separate from one another. On the one hand we have the development of special keysets for more useful man-to-machine information transfer, as well as development of compatible machine-to-man signal-transfer channels. On the other hand we have the development of special techniques for automating the teaching of psycho-motor skills. As it turns out, the early parts of projects in these two areas can be coordinated quite nicely with one another. This is true because in any event, we have to evaluate the "learnability" of the psycho-motor skills needed to utilize these new communication channels, and also because they seem to be very good kinds of skills with which to do the early experimenting on the automated teaching techniques.

II COMMUNICATION CHANNELS TO AND FROM HUMANS

A. Quick and Rough About What We Want To Do

First off, we want to try binary keysets, initially with alphanumeric codes suitable for transmitting text and data roughly in the realm of what the teletype code allows. We would like to check on visual-binary input for humans, to see really what human capability for reading text in this fashion seems to be. How hard are these (transmitting and reading) skills to learn, what is their plateau proficiency, etc. We would like to investigate other physical-channel binary inputs too, and work on other means to actuate the transmitting "keysets" than by fingers alone. Then on to work on more sophisticated, wide-band channels, with much-expanded symbol vocabulary and specially developed shorthand for non-ambiguous machine-handleable transmission of text and data at high rates (i.e. initial goal, competitive with speech, or perhaps stenotype -- later, explore for limiting human rates).

B. Some of Our Considerations for Pursuing This Course of Research

We humans are developing an ever-widening array of equipment with which

SRI Confidential
Page 2

we exchange information. Both the degree of information-handling sophistication and the number of these equipments are increasing rapidly (perhaps explosively is a more appropriate term). The physical means by which communication takes place between human and equipment have usually been dependent strongly upon the special nature of each piece of equipment. Since our earlier equipment was mostly mechanical, where location of knobs, levers, etc. had to conform to limitations of mechanical-design and manufacture, the physical means by which we communicated with the equipment was restricted, and necessarily different from one piece of equipment to another. Much of the newly developing equipment is electrical, from which we are deriving not only a much-increased sophistication in functional utility, but also from which stems new freedom in designing and physical means by which interchange of information takes place.

As the sophistication of the equipment increases, we find that the quantity and nature of this exchanged information changes. In our early days of equipment development, interchange consisted entirely of the human telling the machine what he wanted it to do (control), and of the equipment somehow indicating to the man what it was doing (performance monitoring -- usually just watching the contraption, but maybe having dials or indicators, too). We can be talking of our early looms, locomotives, cars, or what have you. Then we began to develop equipment to help us operate on information, and some of the inter-communication therefore began to involve the information to be operated upon and the results of that operation (in addition to control and monitoring inforpation).
With the development of automatic self-sequencing information-handling equipment has come a new situation, in which there often is not a clear-cut difference between information that is to be operated on and information that directs the equipment's activity. In the future, we can expect a tremendous increase in the amount of information that the professional-type human will want to exchange with equipment, and the trend will be toward having all of the information (to be processed, has been processed, controlling, monitoring) be interchanged in a common form such that it all could be processed by equipment.

It seems that it is time to give serious attention to the development of really effective means by which this interchange of information between human and equipment can be achieved. The means so developed must be compatible with the way in which they are to be used. Also, it would be preferred if these communication means could be more-or-less universal, allowing each person to

3/31/61 DCE
SRI Confidential
Page 3

learn to use almost any piece of equipment with a minimum of re-adjustment -- i.e., he need only learn the special nature and form of the information composition, and does not also have to learn a new physical skill to affect the information transmission.

Stipulating that the means of communication are to be compatible with the way in which we want to use them implies that we should know about these uses. The fact here, though, is that we don't know just how the uses are going to develop. But it is the underlying purpose of the program at SRI, from which the work here discussed is being stimulated, to concentrate on the ways and means by which equipment and procedurds can be developed which will increase the effectiveness of humans in their professional problem-solving pursuits. Other research activities within this program are going to be working on such procedures and special equipments, aiming for the development of practical developments within a few years (the longer-range problems being the concern of yet another activity here). We therefore are going to be giving detailed consideration to the way in which helpful equipment is to be used, and the consideration that we already have given to this matter leads us to the research possibilities which are the main topics of discussion of this paper.

With the kind of flexible, general-use procedures for which I see a big payoff in the human use of machines, compatibility of the means for information interchange between human and machine within these procedures will not be obtained with only fixed visual displays (machine-to-man) and large, fixed keyboards (man-to-machine). It will be a long time before communication with a machine is the only necessary activity for the human in his professional role, and in the meantime we must allow for the other types of activity, (e.g., reading from hand-held paper, scratch-paper thinking, dropping into a colleague's office for an earnest chat, sitting in a meeting, learning something interesting over the phone, arguing with someone in your own office, etc. or, in different frameworks, moving around out in the plant or in his command area) in providing as much help from information-handling mechanisms as possible. Our considerations of compatibility and preference for potential developments of communication means are based upon plans for giving the human some real help in a much more general sense than could be achieved if he must sit down at a special station every time he exchanges information with equipment.
We know that many people place ultimate dependence for man-to-machine communication upon efficient speech-recognition equipment. Besides being most

3/3/61
DCE
SRI CONFIDENTIAL
Page 4

probably too expensive to consider in the medium-long-range application planning, it is a communication means that does not meet our compatibility requirements too well. Principally, it would disrupt concurrent discussions and would not provide privacy, both of which are important drawbacks to our way of thinking. We also feel that something other than a big, fixed keyboard would be desirable,,to give the user more physical freedom. Similar considerations lead us to believe that supplementary, non-visual, machine-to-man communication means should be developed to permit humans to use machine help in a wider and more effective range of procedures than would be provided by visual display alone. Probably it would be better if these supplementary means were other than auditory, for similar but less critical reasons.

The above discussions lead us to the position that (1) serious exploration is needed for really efficient means for interchange of information between humans and their service equipment, (2) the communication means should possibly be compatible for transmission of all of the normal types of interchanged information (perhaps pictorial or analog forms warrant special means), (3) the communication means must be compatible with the manner in which the humans will want to make use of the equipment with which they are exchanging information. These in turn lead to (4) man-to-machine communication channel should be non-vocal, and needs to be capable of as high an information-transfer rate as possible, (5) machine-to-man communication should have a supplementary non-visual, non-auditory channel (which, since it supplements the very-high-rate visual channel, will not need a high transmission rate so badly), and (6) the channels of both Items 4 and 5 should allow the human considerable freedom as to the positions and circumstances under which he can operate them.

G. Specific Things We Have In Mind

The foregoing discussion develops a picture of a general research need. We have some specific ideas about "things we'd like to try" that we feel would make an excellent starting place in the general pursuit of better communication means between humans and the machines that they want to make use of. These are listed and discussed below, in what appears to be a reasonable order of attack.

1. Develop a 5-key binary keyset, and a reasonable encoding for alphanumeric transmission. To be one-hand operated. Set it up with something like a Flexowriter and learn how well people can learn to use it, in competition with conventional keyboards. (Realizing that to obtain full facility, we have to provide for case shifting.)

3/3/61
DCE
SRI CONFIDENTIAL
Page 5

2. Make a keyset for each hand to give each hand equal capability for transmitting any character. See how transmitting speed goes up if we allow alternate use of the two hands. See how much utilization flexibility is provided by operator having choice of transmitting with either hand or with both hands.

3. See how conveniently such a keyset could be designed so that it is more or less tied to the hand, or at least free to move with the hand. This could leave hands freer to participate in other tasks concurrent ~ with intermittent needs to transmit. For instance, this keyset might take the form of special caps on the ends of the fingers connected through flexible wiring to equipment in some unobtrusive manner, so that transmission can be affected by tapping the fingers against any convenient surface. Means must be provided, of course, to de-activate the keying devices or transmission link when the hand is being used for other activities. Several other possibilities are on our minds, too. The general idea is to have the hands relatively free to do other tasks, but yet be almost instantly available for communication without re-positioning the body or limbs appreciably.

4. See if we could develop a small, portable (pocketsized) recorder, to which one of the 'portable keysets" can be connected. This could give the capability of recording information in machine-readable form (assumedly on a tape of some sort) while one is completely mobile -while talking with people (without distracting or interfering with conversations, while traveling, listening to lectures, etc. When back home one could slip the tape into a tape reader and have his recorded information typed out without human transcribing, for his editing and use.

5. See what there is of training one's toes so that a keyset could be built into a special sock (or shoe), to allow intercommunication with machines and at the same time leave, hands, arms(and legs?) completely free for other applications.

6. See about developing skill in reading such as five-bit code patterns, to see if humans couldn't read this ultimately as fast as Arabic symbols. W'hat forms of direct binary-pattern presentation would best facilitate this? Even if this reading skill wouldn't match Arabic-symbol reading skill for speed and convenience, it would allow much simplified man-machine systems -- simpler, more reliable equipment. For instance, one could scan back through his notes on the above mentioned pDrtable keyset recorder.

3/31/61/
DCE
SRI CONFIDENTIAL
Page 6

7. Perhaps we could develop a very small, simple printing mechanism that could be used with one of the above keysets (hand or foot) to print the binary patterns in hand-directed locations on ordinary notebooks and papers, to replace typewriter in some instances in the neat recording of information -- and to have the information in the form that is most simple for machine reading.

8. See about developing skill at interpreting tactile signals applied in parallel binary coding. For instance, in conjunction with five-key keyset, apply tactile stimuli to fingers, using the same binary coding (finger-combination selection) as used for transmitting with keyset. What is the likelihood that a human could usefthis as a useful information receiving channel? Indeed, what is the likelihood that a useful, two-way, telegraphic transducer-transmission system could be developed~directly between bumans or with any amount of signal storage, manipulation, or switching between c? (e.g., for field reconnaissance systems, for operators of equipment that can be interrogated, etc.)

9. Higher-speed channels: We have an idea right now of developing a ten-key keyset, for one-hand used where each finger has two keys so positioned that)independent of other-finger activity, it can arbitrarily strike either key or both (the Stenotype keyboard makes use of such an arrangement). Each strike of the hand then would allow depression of an arbitrary combination of ten keys, which provides 1,023 unique coding symbols. (We might possibly choose to try the following utilizati of ten-key keyset with a more direct, two-hand ten-key arrangement.) We would like to see what sort of coding of the material to be transmitted might be developed to give an effective shorthand, in which unambiguous designation of English text could be produced by a human at a rate perhaps competitive with-normal speech rate. Ideas about this now are that (a) normal alphanumeric designation would be provided by use of only five of the keys)to obtain complete compatibility with the five-key coding, (b) something like the 150 most common words that comprise one-half of normal English test could be each given unique 10-key codes, so that these words (and the spaces ol1 each side of them) would be designated by a single hand stroke, (c) give some of the most common word endings each a unique 10-key code so that each (as well as the succeeding inter-word space) can be designated by a single handstroke, (d) give similar unique 10-key code

SRI CONFIDENMAL

page 7

to other most common letter and word combinations. Could also develop special "vocabularies". For example, a computer programmeroperator might be given the spectral words and symbols in a problem-oriented language for which the machine has interpretive or compiling programs, so that the human can designate his desires in minimal time. Such special languages could each be developed in a fresh field of say 1,000 out of 1,023 code patterns, and the remaining 23 code patterns could be reserved for common usuage in all languages for designating a shift from one language (case) to another. It might be very interesting to see if a human could learn a thousand unique fingering combinations for the one-thousand three digit decimal numbers, so that one of the languages could in this way allow a human to designate a three digit number at each hand stroke. What about resulting transmission rate, error rate, fatigue, etc.

Could such a shorthand allow alternate hand transmission, if each hand were skilled in transmission and had independent keysets, to give significantly greater speed or versatility? Could the toes operate with such a ten-key shorthand vocabulary -- either one key per each of ten toes, or with ten keys per foot?

III AUTOMATED PSYCHO-MOTOR SKILL TRAINING
A. Quick and Rough About What We Want To Do

We want to develop equipment and techniques for providing pre-programmed tactile stimulii for subjects learning psycho-motor skills. These stimuli would be provided in such a way that the Subject is guided through the coordinated sequences of primitive actions that comprise the desired skill actions -- with the objective of increasing speed and effectiveness of training. We would like to develop means to monitor Subject's performance, to make decisions (both automatic and with human-coach interaction) which alter the guiding stimuli in ways that adjust to Subject's performance changes during the learning process. We want to do research which evaluates the various equipment and technique possibilities,and from which we can establish effective procedures for the training of given psycho-motor skills. One can also in this manner fairly well establish the relative
efficiency in learning rate, achievement-level attainment, transference to on-the job proficiency, and retentivity of the new techniques, as compared with other training procedures.

SRI CONFIDENTIAL
Page 8

B. Relevant Discussion

In learning to coordinate physical skill now, the Subject first has to memorize the reaction patterns associated with each primary stimulus involved in the skill. He needs to do this so that he can jondition his automatic reactions to the stimulus by directing his body through the appropriate coordinated sequence of primitive actions whenever this primary stimulus is present. Other than his own memory, a subject learning a coordinate physical skill today generally has no mechanism to guide his association of the desired physical response with the stimulus. Also, he is not skilled in interpreting where his reactions went wrong; his own procedural feedback and unskilled guidance is clumsy and is limited by his higher-center capability for remembering what should be done, and at the same time consciously controlling his body, monitoring its performance, and establishing corrective changes in his mental and physical processes. A human coach, observing the Subject's progress, can help considerably in suggesting changes in his way of doing things, but these changes still have to be envisioned accurately by the Subject and integrated into his other conscious controlling activity by highercenter processes. Also, there is often a problem due to the limitations in the communication means by which the coach gan give corrective guidance during realistic practice. Where action is quite complex, the coach can only describe the changes that should be made, and let the Subject try to incorporate these changes in succeeding practice trials, within the limits of his own interpretation and conscious; controlling capability.

Our notion here is to relieve the load on the Subject's higher-center faculties by means of Cueing signals, applied cutaneously at judicious points on his body, to guide him through his practice motions. This should remove the need for his memorizing the details of the reactions to which his automatic system is to be conditioned. Also, this should provide an improved communication means by which a coach (human, electronic, or combination thereof) can introduce detailed prompting and corrective information during the actual practice activity, to allow a much refined coaching process.

We plan to achieve this by using signals from tactile stimulators, controlltd generally by electronic means, to provide action cues to the Subject. Each of these signals is initially to be associated in the Subject's mind with one of the primitive-action components that comprise the physical action of the skill to be learned. The nature and location of each cue stimulus will be chosen to make as natural as possible its association with the primitive action which it is supposed to prompt. Using the typewriter for a very simple example, let us say

3/31/61
DCE
SRI CONFIDENTIAL
Page 9

that when the subject is to strike the letter j he will receive a shock or a
buzz on his right index finger. Thus, instead of having to associate mentally the symbol j with the proper response, that of striking with his right index finger, he will receive an action cue on that finger to show him how to respond. Reflex actions might possibly be used to advantage for cueing, but this may not be generally convenient, and some of the primitive actions might be less suited to reflex cueing than to learned-response cueing. In any event, we assume that the Subject will requ~re a certain amount of preconditioning as to the particular primitive actions that are to be associated with each cue signal. The signals might be either mechanical or electrical (i.e., vibration, pressure, or electrical current) and perhaps slower-acting thermal signals would be of use on occassion,too.

As a more specific example, consider teaching the operation of the five key binary keyset discussed in the preceding section. Here, the primary stimulus would be the presentation (probably visual) of a given alphanumeric symbol, and the skill response is to be the automatic depression of the appropriate combination of five keys that represents the binary code for that symbol. Means for providing a cueing signal to each finger are controlled by a programming device which is coordinated with primary stimuli symbol presentation, and with the Subject's response thereto. He is preconditioned to these cue signals (in this case, a fairly trivial matter) so that he associates each cue signal with the striking of a particular finger on its key. After the simple stimulus-response reactions to the cueing signals, are learned, the cueing signals could then be presented along with the primary alphanumeric code symbols, thus teaching the Subject automatically to associate a particular alphanumeric symbol with the depression of a particular set of keys. There are many subtle variations to the way in which primary stimuli, Subject response, and cue signals can be interdependent. A straightforward program would provide prompting to the Subject, after presenting him with each primary stimulus syhbol, to guide his responses. He would not have to memorize the alphanumeric code beforehand, nor hesitate overlong before determining what the correct response should be. Monitoring of the Subject's progress can allow modification of the cueing process (e.g., diminishing cue-signal strength, delaying cue signals, or skipping cues for some syhbols) and the rate and sequence with which he is introduced to new primary-stimuli symbols, to match the growth of his skill.

For more complex skills, cues may need to indicate more than just whether a limb is to move (i.e., direction and velocity may have to be indicated, too). A next higher level of skills might be such as typpng, where cue signals not only prompt selection of a given finger, but also the selection of one of several alternative keys for that finger to strike. Here, for instance, a cue signal at the

3/3/61
DCE
SRI CONFIDENTIAL

Page 10

base of the finger might prompt that finger to strike the home key. If accompanied, however, by a signal on the top of the finger, say halfway to the end, the signal prompts the striking of the next key up from home position. Similarly, if the accompanying signal is on the "bottom" or "inside" of the finger, the cue prompts the striking of the key below home position. Other cue-signal locations on the finger or hand could provide other modifying information to guide the selected finger. Preconditioning here would take a little longer than it would with the five-key binary keyboard.

A next higher level of skill might involve the simultaneous coordination of several different primitive motions. Here the relative timing of the cueing signals gains a new inportance. Here also we are beginning to benefit more from the "new comdunication means to the Subject than we were in the simpler skills. When an activity becomes a more complex coordination of primitive actions, it becomes increasingly hard to communicate to the Subject, by normal means, just what is expected of him. The multi-channel cue-signal system should make it much easier to communicate this type of information to the Subject -- and in a consistent form to which the Subject might well develop automatic primitive responses.

This latter concept, the learning of automatic primitive responses to the cue signals so that a Subject san be steered through complex responses with little demand upon his higher-center facilities, can be extended further. It could probably be that very complex skills can best be taught in stages, where the first stage involves the conditioning to respond to a first-level set of cues with a first-level set of primitive responses. The second level of training would use these cues to guide the conditioning to respond to a set of second-level stimuli with a set of second-level responses, composed each of coordinated combinations of first-level (open primitive) responses. The third level of training would use the second-level stimuli as cues in guiding the conditioning bo a set of third-level stimuli with a set of third-level responses, composed each of coordinated combinations of second-level responses. I guess this could go on and on.

The degree of sophistication of the monitoring and control system, the selection of cueing signals, the complexity of the skills for which automated training is attempted, and the degree of skill to which a Subject might be trained, seem from our present stage of training-method development to be practically boundless.

B. Some Specific Possibilities in Our Minds

1. A Starting Point

A reasonably good test of the basic concepts of cue-signal prompting could be centered around the 5-key binary keyset. As mentioned above, the cueing

3/31/61
SRI CONFIRENTIAL

Page 11

need not be too sophisticated, since each cue signal need only tell a finger whether or not it is one of those which should strike its key in response to the associated primary-stimulus symbol. The skill is not terribly hard to learn by normal code memorization and practice procedures (my seven-year~old daughter and I have mastered such a code well enough to carry on a conversation). Skill development should be smoothly measurable by reasonably simple means, so that relative progress of different trainees by different training methods would be easily obtainable.

A quite adequate teaching-machine system could be built up rather easily. A commercially available punched--paper-tape reader could provide the preprogrammed controlling of cue signal transducers and key-actuation-response checking. The Subject could be reading an open-text equivalent of the practice string of symbols punched on the tape, and the tape reader would pace itself automatically to his rate of 'typing'. Many variations in procedure would easily be available with quite simple controlling circuitry. For instance, when the Subject is ready to go on a given string of symbols, he pushes a start button, whereupon he experiences a set of cue signals that prompt him to strike the code for the first symbol. When he strikes the keyboard with the correct key combination, the tape reader advances automatically, and he is provided with the cue signals for the second symbol, etc. If he should strike the wrong key combination at any time, an error signal (light, buzzer, or even a special signal of the same sort as the cue signals) would be energized, and the tape reader would not advance to the next symbol until the proper code was struck on the keyset.

After the Subject had begun to learn the correct responses, an adjustable delay period could be introduced, after each correct response and before the next cue signals were applied, to give the Subject a chance to produce the correct response on his own before he is prompted. Or, the cue signal for the succeeding symbol could start at a sub-threshold level immediately after the preceding correct response, and build up gradually. These variations are mentioned to bring out some of the possibilities that can entsr into the experimentation. There are many others.

Instrumentation to provide automatic measurement of specific aspects of skill development would be fairly simple to attach to the equipment described above. Response time for keying entire symbol strings, or for particular symbols, could be indicated or recorded automatically, as could the number of errors over a given string of symbols or a given time, or the specific nature of the errors. This could help considerably in measuring relative effectiveness of variations in
training technique.

3/31/61
DCE
SRI CONFIDENTIAL
Page 12

It would probably be worthwhile to procure a Flexowriter, Teletypewriter, or other commercially available electrical-input typewriter. This would allow Subjects once in a while to exercise their skill in a more directly rewarding manner, by typing out their exeriise passages, with or without prompting.

The objective of the above described activity would be to get a reasonable measure of the relative effectiveness of the cue-signal prompting technique for the teaching of psycho-motor skills. If these new techniques prove relatively ineffective, we might be impelled to find out why in continuing research. If they prove to be promising, we should like to extend research both into the development of useful practical implementations for different kinds of skills and into the extension of the theory of learning to accommodate the findings of this research and to help guide it.

2.a. Further Work

a. General Experimental Facilities

If the basic techniques prove promising enough to warrant their extension into more sophisticated types of skill training and learning-theory study, it would seem most wise to spend extra time and money at the outset to develop some good, general facilities for further experimentation. The heart of the experimental facilities would be the equipment which controls the application of the cueing signals, programs the symbol sequences, measures detailed performance of the Subject, perhaps modifies procedures automatically according to the measures of performance, and perhaps automatically processes the performance measurements to establish meaningful record of the Subject's training history and progress. Peripheral to this, for different skill-training situations, would be the particular array of cue-signal transducers, the primary-stimuli "transducers" or display, the operational facilities associated with practicing (or training) that skill, and the transducers (measuring devices) that convert performance features into signals useable by the controlling equipment.

There would be several possible approaches to building up this kind of testing facility, where provisions want to be made for changing flexibly from one kind of procedural arrangement to another (and where the specific needs undoubtedly can't be completely anticipated). On the one hand, we could develop special purpose switching and control equipment that incorporates enough special features to make it reasonably adaptable for what we might want. This would not be cheap at all. On the other hand, we could build up our experimental facilities around a samll, general purpose digital computer, which could provide all the services of control and modification of detailed experimental procedures as well as performance data analysis and record keeping. Each new kind of experimental setup then would

3/31/61
DCE
SRI CONFIDENTIAL
Page 13

only require new development for the practice operation facilities for the signal and monitoring transducers and for the computer program. One computer could be shared among any number of such setups (probably handling only one experiment in
real-time operation at once), and would provide a maximum of all-round experimental flexibility. It seems to me that if serious pursuit of automated skill training is to be undertaken, the development of experimental facilities around a computer should be given very serious consideration.

b. Cue Stimuli Research

The nature of the physical stimuli to be used for cueing signals under different raining situations would need some specific research attention. This would have to be coordinated with research on the possibilities for transducers that could convert the controlling electrical signals into the particular physical stimuli being considered. Some cueing signals have to be energized and elicit reaction in a very short time, which would tend to rule out chemical and thermal types of cutaneous excitation. However, these may prove useful for situations where speed is not important. Electrical currents would perhaps be the most convenient from an equipment-implementation point of view, but they sometim produce unpleasant sensations which might negatively affect the learning process. (Though, since sometimes sensations are not unpleasant, we may be able to develop satisfactory electrical cueing techniques.) Research on cue-signal realization methods should begin 8Oon after a decision is made to pursue cue-prompted training automation in a serious manner.

3. Advanced Possibilities

We should like, for instance, to see how complex can be the skills for which these cue-prompting techniques could provide significant training help. Typing has been mentioned as something of a next step past the binary keyset, and there are other keyboard skills (such as stenotypy) which are perhaps more of a challenge. How about piano playing, or other musical instrments? Would these techniques be useful for training operators of special equipment, such as a pilot of an airplane or submarine, operator of a crane or rolling mill, etc. How about skills as refined as high jumping, pole vaulting, skiing, etc. -- could the monitoring and analysis of performance and the control of guiding cues proceed in such a rapid and refined manner, and could the Subject learn to be guided so sensitively and dynamically, that best-form performance could be established significantly more effectively than it can now. Almost all of us when we have watched people perform who have devoted many, many hours to practicing have marvelled at the high degree of psycho-motor skill that a human can develop. It is very intriguing to contemplate the new heights in skill development, and the new availability of skill to all of us, that would come about if we could indeed develop significantly better ways to teach psycho-motor skills.

3/31/61

DCE

SRI Confidential
Page 14

IV. COMMON POSSIBILITIES

Here are some miscellaneous possibilities for research that are more or less common to both the Communication-Channel and the Automated-Skill-Training areas. These are mostly based upon the possibilities for using the stimuli that have been providing cue signals for more general information transmission to the human.

We mentioned, in the discussion of Communication Channel research, the possibility of tactile signals, applied in the same coding as used by the fingers in transmitting with the binary keyset, being used for receiving coded information. In the Skill Training Discussion we described the use of tactile signals for cue prompting while training the Subject to transmit coded information on the keyset. It seems that very little special conditioning, if any, would be needed to give the Subject (who has already been trained, by cue prompting,to transmit on the keyset) the additional capability to turn his attention on the cue signals and to recognize the information that they would, in their cueing capacity, otherwise be directing him to transmit on the keyset. In other words, once the basic codeconversion skill has been learned, the erstwhile cueing signals could be used, not for prompting, but for receiving information. This dual send-receive capability would seem to be a byproduct of cue-prompted training for any kind of data transmission (including the faster-than-talk Stenotype system).

What sorts of other uses might this machine-to-human data transmission capability have, relative to training or operation? For one thing' in the automated training of symbol skills' prompting via cutaneous cue-signal communication, channels might be very helpful. This could be on the primitive level of fairly direct associations, cue signal to concept, or could be by actual coded-text prompting. More-flexible teaching machines could be realized cheaply if this "binary cue-coded" communication channel were available. For another use, consider the training of operators of various kinds (from air pilot to fork-lift driver) where the actual psycho-motor skill involved represents only part of, the training problem (sometimes a small part). The operating procedures to be followed in different situations often represent the greater part of the training task. Suppose the Subject were pre-conditioned to some special cue signals (if cue-prompting training had been used for the psycho-motor skills associated with the job, this new conditioning could be quite brief) so that they prompted him to physical motions that would represent the beginning actions of different procedures. It would seem that this kind of prompting, as the subject goes through his procedural training exercises, would be quite effective in reminding him what his next step should be, or

3/31/61/
DCE
SRI Confidential
Page 15

telling him that what he is about to do is in error. While he is thinking hard about what to do next, spoken prompting would seem to be quite distracting (particularly if he has to be talking or listening as part of his procedures) as would visual prompting while his eyes are otherwise quite busy. Operationally, during the on-the-job application of his training, some of the cutaneous communication channels developed during his training might profitably be put to use to provide auxiliary information to the operator.

How about someone with a tracking job who could be provided with cue signals, to which he has been conditioned as indicating unusual transients in the target's behaviour, so that his reflexes to cue signal could quicken his reaction to a transient that otherwise would have to be "recognized" as suchfby his visual percdptionf from among the normal visual input signals? This presumes, of course, that the transient in question is generally detectable by electronic means.

Relative to the Communication Channel area, there are a few problems that would be of special interest to the Skill Training area. For one' it would be interesting to see if special keysets could be made for the toes, and what special training problems might arise in teaching people how to use them with reasonable proficiency. Another problem is more or less a question of saturation. It was brought out earlier' in our Communication-Channel disoussion, that one hand (or foot) could operate a ten-key binary ketset that would allow 1023 different key-depression combinations, each of which could be given a unique meaning.

We hypothesized that many different "vocabularies" could be developed for these codes, and that a sophisticated person might learn how to shift back and forth flexibly among a number of them. One such special "vocabulary" would be the first one-thousand three-digit decimal numbers. What is the saturation level for this sort of thing? At what point have you spread the human's conditioning out so thin that his error rate begins going way up? How much of the training given to one hand in such a skill transfers over to the other hand if we set out to make our subject ambidextrous in his transmitting skill? If he becomes ambidextrous in operating this type of keyset, can he easily learn to time-share the transmitting load between his hands by letting each send alternate codes, and can he increase his speed thereby?

SRI Confidential
Page 16

V. CONCLUSIONS

We would like to start project work in both of the areas discussed. The reasons that their discussions are included in the same think piece is that: (1) it seems quite probable that they would both be of interest to some potential sponsors (many of whom would like to have better man-machine information-exchange means developed, within some of the same use-compatibility constraints that we establish' and they also have psycho-motor skill-training problems, and further they might well also be interested in the development of the "common possibilities"), and other research projects would benefit if they could be planned (particularly in their early stages) as co-operative ventures. They could actually share experimental equipment, and even benefit from common experiments. The facilities and personnel for both research projects oould be enrichened if it were pre-planned that they be co-operative, and very valuable cross-stimulation would occur.

It is quite true, however, that neither project would be dependent upon the existence of the other, and that therefore we could go ahead in either area alone if that were all that could find support. We only point out that each research project would get extra value from the concurrent existence of the other.

The "specific ideas" mentioned for each research area are only offered to clarify our general discussions with examples, and to indicate how we would probably proceed toward the longer-range goals implicit in the discussions if we were free to proceed now We could be reasonably flexible in considering alternative approaches that might better fit potential sponsors' immediate needs, but we should remain aware in these cases of the implications therein relative to these long-range goals.

DCE/dg
3/31/61