definite conclusions about the state of the art of criterion development in psywar.

AN INTRODUCTION TO THE PROBLEM

The proof of the effectiveness of most of the techniques of applied behavioral sciences, including those of psychological warfare, rests on the ability to provide adequate criteria of effectiveness. The criterion is the crucial element in most studies of effectiveness, but it is surprising to note the dearth of adequate research or pertinent articles in the professional literature on this subject. Obviously not all the problems of criterion definition or criterion development are resolved, and yet there has been a diminution of interest in the area. For example, the index for the Psychological Abstracts shows the following number of entries under the heading "Criterion" for the following years:

This dearth of entries suggests neither attraction nor interest in this vital area. One investigator has remarked that we often act as though criteria are either God given, or just to be found lying around somewhere.1

This deficient state of our knowledge has been discussed in many quarters, and pleas for attention to research in the area are heard on every hand. It is frequently stated that much more attention is given to the construction of predictor elements, for example, than in the development of criteria. We find ourselves with predictors that have been refined to a psychological hair-reliability coefficients on the order of 0.90, successive attempts to increase reliabilities by a few points, and elaborate attempts to establish norms; but still we are tied to primitive criteria in most cases. In the sphere of psychological warfare, the problem is especially acute. A critical examination of methods used to evaluate psywar programs since World War II follows.

AN EXAMINATION OF METHOD USED TO EVALUATE PSYWAR PROGRAMS SINCE WORLD WAR II

Many psywar programs have developed haphazardly, and evaluation of their success, in cases where evaluation has been attempted, has not been scientifically rigorous. There are cases in which psywar activities have been totally unsuccessful. In one instance, an aircraft was outfitted with a loudspeaker system and used for four months before it was discovered that the broadcasts were totally inaudible to people on the ground.2 Other

cases are on record in which techniques produced results directly opposite to those intended. FM 33-5, "Psychological Warfare Operations", presents only a skeleton account of effects analysis.3 Obviously a need for a program of criterion development exists.

4

Daugherty has pointed out three major reasons why evaluation efforts of the past have been of limited value. First, those who have been responsible for psychological operations or for post-operation evaluation have not possessed a clear and consistent understanding of the nature and mission of the activity. Second, no clear or acceptable criteria exist for measuring psychological effects except in limited situations. Third, even if the problems of psywar were understood, and even if adequate tools were available for effectiveness measurement, there would be other factors that would greatly limit the success of evaluators, e.g., combat conditions, logistical problems, etc., which would bring about inevitable compromises with scientific rigor.

Assessing the Impact of Psywar Efforts

Among the methods of evaluating the success of psywar efforts have been the following: content analysis of intercepted mail, captured military documents, monitored radio broadcasts, newspapers and magazines printed in the target area; questionnaires and interviews of captured enemy personnel or of enemy civilians in areas recently coming under friendly control; and observation by nonparticipant or participant observers. Most of these methods suffer from two problems: the lack of representative sampling and the lack of an integrated approach to evaluation. Adequate samples can be drawn of enemy civilians living in areas controlled by friendly forces, but several problems are encountered: often they are unwilling to cooperate and in many instances a period of time has intervened between reception of psywar messages and their capture. The fragmented approach to evaluation could be improved if each method could be weighted according to its worth and employed accordingly. These methods are indirect, but are probably as direct as one could hope for since target populations are usually hostile and therefore unapproachable. One segment of the enemy population is approachable, however, and often serves as subject for research: that of prisoners.

Prisoners of War

Prisoners of war have been used in the evaluation of psywar materials, but the tendency has been to give undue weight to this group. The difficulty in generalizing from the prisoner group to the enemy population is that the prisoner group is not a representative sample of the total enemy population. The very fact that a man is captured may reflect an inherent difference between him and his comrade who has managed to evade captivity. Once captured, even the nearly successful evader may change psychologically. For him the war is over. Good food, safety, medical treatment, and so on may change his attitude significantly, espe

cially after the initial fear of death and torture at the hands of his captors has dissipated.

Shils and Janowitz 5 developed a taxonomy of social disintegration within the Wehrmacht during the closing days of World War II which could be of value in judging surrender behavior. . .:

1. Desertion

a. Individual

(1) After discussion with comrades

(2) After no discussion

b. By groups in concert

2. Active surrender (deliberately signaling, sending emissaries, etc.) a. Individual

b. By the group as a unit

(1) By mutual agreement

(2) By order of, or with approval of, NCO or officer

3. Passive surrender

a. By individuals acting alone

(1) Non-resistance

(2) Token resistance

b. By plurality of uncoordinated individuals

4. Routine resistance

5. Last-ditch resistance

This taxonomy could be used as the basis for a scale of "willingness to fight" which could be related to the intensity of their exposure to the psywar campaign.

Efforts to find grist for the psywar mill have sometimes involved interviewing prisoners to find why they surrendered. This technique can be effective, but there are potential problems in using it. It is often impossible to ascribe motives to a surrendering prisoner since he, himself, may be unaware of the reason for his surrender. Was it because of fatigue, hunger, lack of ammunition, or other "objective" factors; or was it because he became "fed up" with the war because of exposure to psywar ploys? It has been pointed out that prisoners will not admit to deserting or allowing themselves to be captured because of guilt feelings over dereliction of duty. They would rather admit to "falling asleep in a cellar and waking up in the midst of enemy troops" or "getting lost and blundering into the enemy" than admitting that they sought out the enemy and surrendered to him. There have been cases of enemy commanders refusing to surrender towns or fortifications until a single shell was fired into it. Such symbolic resistance is a balm to the conscience of surrendering forces.

"Most Like” and “Qualified Judge” Approaches to Pretesting

The psywar operator is faced with the problem of obtaining accurate information about large populations without systematic use of the populations. There are two possible approaches to solving the problem which

may be called the "most like" approach and the "qualified judge❞ approach. The most-like approach consists of using respondents who, while not being "the same as" the target population, can be considered most like it. The rationale is that people who are culturally similar will be psychologically similar-a tenuous assumption—but perhaps a necessary one. In order to evaluate the effects of broadcasting to [the People's Republic of] China, for example, the broadcasts could be pretested on refugee groups, citizens of Taiwan, or, if the budget was low, Chinese living in the United States. The second approach, the qualified-judge approach, involves asking a person who knows the group in question to make certain estimates about the inaccessible group. In the example above, a person grounded in Chinese history, language, religion, and other aspects of that culture may serve as the qualified judge.

Sometimes enemy prisoners are used as most-like groups, but there are pitfalls in this approach. Pretesting inevitably involves some question-and-answer procedure, but when the cultures of the interrogator and the subject are different, the reliability and validity of the responses may be compromised. For example, as Daugherty has pointed

out:

The average person in Asia, including those who serve in the armed forces of their country, is a simple-minded, semiliterate individual, who is seldom if ever consulted for opinions on any matter.

Thus when someone approaches him concerning his views on the contents of a proposed communication, he understandably may hesitate to go on record as either approving or disapproving a suggested course of action. He surely would hesitate to express an adverse opinion on any action he believed to have originated on a higher level in the social scale from that which he occupies.

Almost universally when such individuals are asked to talk about such matters, there is an apparent eagerness to say that which is designed to please the questioner. As a matter of fact, in all Oriental societies it is considered to be a mark of bad manners for one to appear to be vocally critical of the efforts of another, especially if expressed openly to his face. Among prisoners of war there is likely always to be the further consideration that one should not slap the hand that feeds him.

The people in Asia have been told for centuries what to do, and in many cases what to think. The democratic precepts of the worth of individual opinions are entirely foreign to an Oriental's mental processes. Among the more highly educated classes, basic concepts of politeness prevail to such an extent that they are apt to color any expressions of opinion that are made. Under the very best of conditions it is difficult to get more than a rough approximation of what an Oriental thinks on a controversial item by questioning him about it.

Working through native interpreters further complicates the procedure, for the interpreter, in common with his fellow countryman, will hesitate to be brutally frank at the cost of being seemingly impolite. There is also the further problem of being certain that the ideas are communicated accurately in interpreting what is asked and in reporting the answers given.'

The qualified-judge approach suffers from the obvious handicap that the judge may suffer from nearsightedness, bias, or any of a number of ills to which human judgment is subject. Yet, in the absence of a most-like group, or an actual sample of the target audience, the qualified-judge approach may be the only one available.

Examples of Effectiveness Research

There have been no systematic, i.e., controlled, studies of the effec

tiveness of a total psywar effort, but there are a scattered few that consider a portion of the total. One of the best examples of a controlled study of effectiveness is a study by Andrews, Smith, and Kahn. These investigators hypothesized that the fundamental effects of psywar can be characterized in psychological form, and that they are predictable in terms of the attitudes, motives, and experiences of the recipients. It was also hypothesized that psywar efforts can affect an individual only in certain optimal conditions. They developed scales that attempted to measure nine factors, including the degree to which the individual, before the Korean War, was in accord with the ideology and war aim of the Peoples Government; the degree to which, and frequency with which, the individual had experienced intensive fear during battle; the degree to which the individual felt he had been poorly treated and physically cared for by his own forces during the war, and so on. Two of the nine scales were criterion scales which measured his willingness to defect or surrender. The scales were intercorrelated (each scale was correlated with every other scale) and subsequent analysis appeared to corroborate the major hypotheses. This study was replicated using another target group, Communist Terrorists in Malaya, with essentially the same results."

A number of studies have investigated the effectiveness of radio broadcasting and films, and their methodologies are similar. In the usual case, groups are assembled and hear tapes of the broadcasts. After hearing the tapes, they either discuss the merits and shortcomings of the broadcast in a panel discussion or in individual interviews, or variations thereof. The lack of precision and control in such procedures is evident, but they could be useful in preliminary work.

Evaluation of psywar efforts has largely been a catch-as-catch-can proposition. Most of the evaluative follow-up work, when there has been evaluation, has come as an afterthought or as a more-or-less unplanned adjunct to psychological and non-psychological military operations. To date, there has not been a concentrated effort to develop criteria of success, and this is one of the reasons why previous efforts have come up with so little.

SUMMARY AND CONCLUSIONS

At the present time there are few, if any, technically acceptable criteria for the evaluation of psychological warfare programs, nor are there adequate techniques available for developing criteria for the evaluation of a total psywar program. Most psywar programs have developed haphazardly, and evaluation, if it has been undertaken, has not been scientifically rigorous. Those who have attempted to research the problem have not had the responsibility for psychological operations and have not possessed a clear and consistent understanding of that activity, and the very nature of war has prevented the precision and control necessary to obtain adequate measures.

Some methods of evaluation have been tried using content analysis,