This information has been archived for reference or research purposes.
Archived Content
Information identified as archived on the Web is for reference, research or recordkeeping purposes. It has not been altered or updated after the date of archiving. Web pages that are archived on the Web are not subject to the Government of Canada Web Standards. As per the Communications Policy of the Government of Canada, you can request alternate formats on the "Contact Us" page.
COMMAND AND CONTROL
J5PA Combat Camera photo IS2002-1622.j by MCpl Brian Walsh
Commodore Dan Murphy and Commander Gary Hatton reviewing Maritime Interdiction Operation Guidelines aboard the Canadian Task Group flagship HMCS Montréal prior to authorizing the boarding of a ship in the Gulf of Oman, October 2002.
SYNTHESIZING TWO APPROACHES TO DECISION MAKING IN COMMAND AND CONTROL
For more information on accessing this file, please visit our help page.
Throughout history, military commanders have been confronted with the problem of planning and controlling operations amid the confusion and constant change of the battlefield. Modern technology has not changed this,1 as uncertainty remains an inevitable feature of command. Yet, decisiveness is an essential requirement of effective command, especially as the tempo of warfare has increased. The changing global political situation is forcing military organizations, including the Canadian Forces (CF), to expand and change the ways they conduct operations. Thus, there is a continued need for the study of decision making in the military context, ranging from basic research on how people make decisions and solve complex problems to applied work on the development of training and support systems to aid commanders’ decision making in the field. This sort of research is especially important today because military operations are becoming increasingly demanding, fast-paced, and risk-filled.2
TWO APPROACHES OF DECISION MAKING
The purpose of this paper is to contrast two general but highly influential approaches to understanding decision making. These approaches differ in a number of key respects, but each has been applied, in some form, to the study of military Command and Control (C2). Classically, theories within the first approach of decision making are based on the premise that human decision making can be modelled in terms of formal processes predicted by normative theories of probability and logic.3 These theories have been popular in the development, by researchers, of support for military decision makers, perhaps because there is often an assumption that good decision making follows a rational approach in which decisions are based on expected outcomes and there is an attempt to select a course of action that will bring the optimal outcome.4
Within that last 15 years, however, there has been recognition that the nature of the real-world battlefield limits a human decision maker’s ability to implement truly analytic processes.5 This has led to the development of another approach that comprises what are called naturalistic or intuitive theories of decision making. This approach is based on the premise that people use informal procedures or heuristics to make decisions within the restrictions of available time, limited information, and limited cognitive processing capacity. Intuitive decision making, as we will see, offers a clearly different way to think about decision making than the analytic approach. We will later argue, however, that these diverging approaches can nevertheless be synthesized into an effective framework for understanding military decision making in its broadest sense.
J5PA Combat Camera photo ivd-02-058
by Sergeant David Snashall
A Princess Patricia’s Canadian Light Infantry captain discussing the defensive layout of Kandahar airport in Afghanistan with an American NCO, February 2002.
THE ANALYTIC APPROACH
In taking normative theories as bases for developing descriptive models of decision making, researchers in the analytic tradition have emphasized explicitly computable processes to take in information, code it symbolically, manipulate these symbolic representations, and generate some output.6 Analytic theories require some kind of formal comparison among decision alternatives using deliberate and procedural rules that quantify those alternatives.7 This assumes that all pertinent factors can be both identified and quantified in terms of their absolute or relative impact. Methods of comparison are based on various kinds of sequential or distributed, logical and/or mathematical algorithms,8 so that there is a direct link between facts and conclusions. Numerous specific procedures for comparing alternatives are known, most of which can be computationally modelled by production systems operating on a representation of the problem space.9 Many, for example, are based on Bayesian statistics and evaluate options in terms of base rates for different hypotheses and probabilities of the accuracy of different observations.10 Other analytic strategies include subjective expected utility analysis, single feature difference, and elimination by aspects.11
To select a decision alternative, the decision maker needs to specify one or more dimensions along which to compare the alternatives. Typically, researchers propose that these computations are based on compensatory algorithms in which information types are weighted and values of one type can compensate for the absence of other types.12 A popular general form of analytic theory is the linear compensatory model, which involves the computation of an overall score for each decision alternative based on the sum of relevant dimension values for each alternative, weighted by each dimension’s importance.13 Because the score of each alternative is based on all known dimensions, effects of large and small dimension values can compensate one another in determining the overall desirability of the alternative.14
A core assumption of analytic theories is that the goal of decision making is to reach an optimal decision as assessed along some criterion dimension.15 Optimality is a difficult concept to operationalize in the military (or any other) context, but it is generally defined in terms of maximizing benefits, such as enemy units destroyed and friendly units preserved in tactical situations, while minimizing costs.16 A premise related to achieving optimality is that all possible hypotheses must be analyzed because there is no a priori means of knowing what alternative will be optimal.17 As a consequence, analytic decision making entails extensive computations, except for very simple problems.
Although analytic theories have been tested with great experimental rigor, there are a number of conceptual problems with this approach.
Analytic theories generally assume some form of exhaustive option generation and evaluation, implying that people either have access to all possible options or are somehow able to generate a complete set. Such a comprehensive search, however, would be extremely time consuming, if not impossible, in real-world problems, given limitations on human knowledge and cognitive capacity. In fact, studies have found that expert decision makers tend to generate only a few potential solutions when solving complex real-world problems.18 Moreover, it may not be possible to construct a complete representation of the problem space, including their goals, the values of potential outcomes, and the probabilities of certain actions producing certain outcomes.19 This is especially true for a complex military problem. Decision makers may lack access to knowledge that would allow them to determine potential outcomes and their probabilities. In cases where the problem is not simple, the number of parameters and potential courses of action may make the process of just describing the problem space so complex that one would have to assume near-omniscient capabilities to process the data in real-time.
Tests of analytic theories have generally been conducted in the context of relatively simple problems, such as inferences with binary choices or problems with small amounts of data, and situations in which decision makers have sufficient time and access to all relevant knowledge and information.20 Such situations bear little similarity to the fast paced, high risk environment of military operations, thus raising questions regarding the applicability of analytic theories to this domain.21 In real-world domains, people are likely to deal with multiple pieces of information that may be ambiguous, highly inter-related, and potentially obscured or missing. Such limitations can make analytical theories intractable and highly implausible as descriptive models of human decision making.22
THE INTUITIVE APPROACH
Over the past 10 to 12 years, intuitive theories of decision making have become influential in explaining decision making in C2 and other complex, real-world domains. Intuitive theories, or Natural Decision Making (NDM) theories as they are most often called,23 are based on descriptive, rather than normative, models of the strategies employed by experienced decision makers in coping with real problems.24 Lipshitz, Klein, Orasanu, and Salas,25 for example, base assessments of the quality of decision making on the behaviours exhibited by experts rather than some formal model. In particular, intuitive theories dispense with the concept of normative choice and assume that human decision makers use much less formal but much faster strategies.
Three basic principles underlie intuitive theories.26 The first is that decisions are made by sequential, holistic evaluation of potential courses of action against some criterion of acceptability rather than by feature-by-feature comparison of multiple alternatives along multiple dimensions. The second principle is that the decision maker relies primarily on recognition-based processes to generate options and compare them to previous experiences (including actual on-the-job and training experiences). Thus, there is no exhaustive generation and comparison of alternatives. Instead, the decision maker identifies potential courses of action by first assessing the situation then recognizing past situations that are similar. Based on past experience, the decision maker can recall previously taken courses of action and determine their acceptability to the current situation. The third principle is that decision makers adopt a satisficing criterion, stopping the search when an acceptable course of action is identified, rather than searching for an optimal solution.27 Real world situations often demand very rapid responses and decision makers may have to accept a solution that merely works without considering whether a better solution exists.
An influential example of the intuitive approach to decision making is Klein’s28 Recognition-Primed Decision (RPD) model.29 Like all intuitive models, it eschews formal, logical processes and instead emphasizes pattern matching processes, in which the decision maker first appraises the situation to classify it as familiar or not, based on experience. The RPD model incorporates additional processes meant to complement recognition in cases where the decision maker is unable to recognize the current situation.30 A typical reaction is to seek more information and this sort of on-going situation assessment is a crucial component of decision making in the RPD model.31 In addition, diagnostic processes such as story building can resolve ambiguous situations. In this case, the decision maker deliberately notes features of the situation and attempts to create a detailed hypothesis, or story, that could explain that configuration of features, even in cases in which the availability of information is limited. Once the decision maker has diagnosed the situation, he or she can use mental simulation to form expectations about future events that serve to test the working hypothesis. If there are too many inconsistencies between the hypothesis and the situation, the decision maker must revise his or her hypothesis.
The major problem for research with the intuitive approach to decision making has been the generally vague level at which models have been formulated. Indeed, Todd and Gigerenzer32 have argued that researchers have deliberately avoided detailed theorization in the mistaken belief that decision making processes actually used in real-world environments cannot be formally modeled. Certainly, researchers in the behavioural sciences have traditionally sought general models of cognitive functioning, but this does not preclude in-depth modeling of specific cognitive operations in specific decision making settings. Without more detailed models, it will be difficult to develop meaningfully specific hypotheses to test intuitive models of decision making.
J5PA Combat Camera photo IS2002-0109a
by MCpl Danielle Bernier
Keeping a watchful eye over the perimeter of Kandahar airport during Operation “Apollo”, May 2002.
DECISION MAKING IN THE MILITARY CONTEXT
Both classes of theory have received empirical support in various contexts,33 and this precludes a simple conclusion that one is an inherently better theory than the other. The best that can be done is to consider the question of the particular military situations and conditions under which one type of theory might serve as a better model of human decision making than the other.
Although analytic models generally do a poor job of describing how people actually make decisions, they are highly explainable and prescriptive. The steps and processes involved in decision making are clearly specified and can usually be implemented as computer programmes. Consequently, these models are excellent tools for predicting and exploring military decision making performance. In addition, the focus of analytic models is on obtaining the optimal solution to a problem, which is, of course, highly desirable in any domain as long as it can reasonably be achieved. This suggests that analytic models might apply in situations in which there are well-defined goals and in which factors and options can be explicitly defined.
Intuitive theories have the advantages of being closely linked to what expert decision makers actually do in real-world situations and of being applicable to dynamic, uncertain, and high risk environments, as demonstrated in numerous empirical studies.34 Intuitive approaches certainly are better suited to situations where time and data are limited.35 Their usefulness as models of decision making, however, can be limited by their informal nature, which makes it difficult to develop specific, testable hypotheses for research.36 Also, intuitive theories are, by their nature, more descriptive than prescriptive, making them effective for understanding human decision making but less so for developing systems to aid/support actual decision making performance.37
In all likelihood, no single decision making approach can be considered best for explaining how people make decisions in all situations. Certain phases of a military operation, such as planning and preparation, afford more time, more complete data, and greater access to computing resources than do other phases, such as activities related to surveillance, threat detection, and threat response. Thus, what should be considered an appropriate approach to modelling decision making will depend on the particular operational context. In this case, we can perhaps see analytic and intuitive decision making not as competing classes of theories but as different kinds of decision making strategies. By strategy, we mean a specific mental procedure by which the decision maker computes or infers his/her choice of a course of action. Analytic strategies would represent an ideal set of procedures that can be used to approach optimality, whereas intuitive strategies would represent a more practical set of procedures suited to situations of limited time, information, and computational power. We assume that people have access to both types of decision making approaches, depending on their needs and the nature of the situation. Perhaps the key factor for choosing between analytic and intuitive strategies (i.e., a specific procedure to be applied to a specific problem) is whether one can accept a merely workable solution or whether one requires an optimal solution. It may be under only some circumstances in military command, such as planning and preparation, that decision makers have the time and high quality data needed to engage in truly analytic processing.
The idea that people have access to many different decision making strategies/procedures is not new, as others have distinguished ‘strong’ (formal) from ‘weak’ (heuristic or intuitive) methods of reasoning.38 We likewise believe that analytic and intuitive decision making theories actually represent distinct strategies or methods of decision making. Moreover, these methods fall at opposing ends of a continuum of possible decision making procedures, as illustrated in Figure 1. This continuum is based on the formalism of the decision making method, which is defined by how exhaustive, quantitative, and compensatory the procedure is, and by how many options are evaluated, and whether the options are considered sequentially against a criterion or concurrently through feature-by-feature evaluation. Analytic methods are those in which multiple options are evaluated through an exhaustive, quantitative, and compensatory feature-based analysis. Intuitive methods are those in which alternatives are evaluated holistically through a sequential comparison process that is generally qualitative, non-compensatory and not exhaustive. Thus, there are many kinds of analytic procedures and many kinds of intuitive procedures, all varying in their degree of formalism.39
Figure 1: Proposed continuum of decision making strategies and its relation to the amount of resources available in the situation.
Both analytic and intuitive methods seem to be valid, as empirical evidence suggests that people have access to at least these two broad classes of decision making processes. A particular decision making procedure can be selected as needed, depending on the problem and situation.40 Although truly analytic processing is difficult to apply, except to simple problems, the constraints of knowledge and time vary in real-world problems, allowing applied decision making strategies to vary in terms of their analytic formalism. Because military planning typically takes place under (relatively) light time pressure and sometimes (if not always) with relatively large amounts of high quality data (especially if various decision and computation tools are available for use over extended periods of time), analytic strategies may be better suited to this stage of operations. When planning, decision makers are more likely to have the time and access to resources needed to clearly define the problem and consider multiple courses of action. Also, there is greater emphasis on determining the best possible solutions rather than merely satisfactory ones because the goal is to develop the best possible plans and procedures to achieve mission objectives. When the enemy is engaged, however, commanders may be forced to rely on intuitive methods because their time and knowledge are limited. The reliance on intuitive decision making strategies would not be surprising in tactical situations, given the advantages of speed and information processing efficiency conveyed by these strategies.
Some studies suggest that analytic and intuitive decision making strategies interact in complex problem domains.41 Roth,42 for example, found that nuclear power station crews exhibited both situation assessment activities, consistent with intuitive decision making, and procedural activity in which crews attempted to apply pre-planned responses when responding to simulated emergencies. This is, in a sense, an intuitive strategy but it relies on analytic approaches having been performed in a preparatory phase; that is, operators developed links between anticipated events and the responses that would be appropriate. In this case, crews spent time before confronting the emergency developing procedures to deal with a range of potential events and situations. Intuitive strategies, in contrast, seem better suited to dealing with fast paced, evolving problems or events in the field. Here, there is greater time pressure and the goal is focused on accomplishing the mission in an effective manner more than finding the optimal way to accomplish the mission.
The Roth study highlights the importance of not focusing only on crisis situations when studying decision making, but also considering the decision making done during planning prior to entering specific situations that require action. Mission planning and preparation is a distinct phase of operations that is defined by the need to acquire information about expected threats. The plans developed prepare the force to meet anticipated threats by providing detailed analyses of enemy and friendly force capabilities, the political and civil situations, neutral forces, and parameters of the mission (rules of engagement, political resolve, etc.). The key point to make is that plans serve as bases for intuitive decision making. When time is available individuals can study potential events and conditions to make them easier to recognize during more time-stressed situations. In addition, they can develop pre-planned courses of action to aid in the generation of acceptable responses. To date, there is no clear indication of the extent to which people employ analytic strategies in planning. It is likely that here, too, there are certain constraints that limit the comprehensive analysis of a problem and alternate solutions. Nevertheless, it seems fruitful to consider analytic and intuitive decision making as synergistic styles that together can greatly enhance decision making.
A potential problem, however, is that planned responses are not helpful unless the decision maker on the spot recognizes the relation of the current situation to the planned contingency. Decision makers in the field are likely to rely on intuitive, recognition-based strategies to retrieve suitable course of actions during time-stressed events. If they have not themselves participated in generating pre-planned responses, it will likely be much harder for them to match the situation to planned contingencies and retrieve the pre-planned response. Because intuitive decision making procedures often do not generalize widely across different decision problems,43 a decision maker may fail to see the connection between the current situation and previously planned responses, including the degree of modification required. This failure occurs when the situation is not exactly the same as the situation anticipated in planning – even if the planned contingencies are similar to or even formally equivalent to the current situation.
A discrepancy between decision strategies used in planning and in the field is not, in and of itself, a bad thing. But we must recognize the synergy between analytic and intuitive strategies to enhance overall decision making success. Specifically, analytic strategies could be used in planning to generate high quality courses of action to anticipated problems so that these courses of action can be rapidly selected and implemented by intuitive, recognition-based decision making during critical events. In this way, the benefits of analytic decision making can be indirectly imported to intuitive decision making under stress and time pressure. Thus, a major area of opportunity for the design of decision support is improving the connection between planning and action.
DND Photo IS2002-1703a by MCpl Brian Walsh
Debriefing of a naval boarding party after a small boat inspection in the Gulf of Oman, November 2002.
CONCLUSION
Traditionally viewed as competitors, the analytic and intuitive approaches differ in many respects and have distinct strengths and weaknesses. It seems unlikely that an entirely analytic approach or an entirely intuitive approach will serve as a useful model for every aspect of military decision making. Consequently, effort should be directed at employing the advantageous aspects of each approach.
In practical terms, the most immediate use for our knowledge of decision making is to guide research into the development of decision support through the provision of tools, information systems, computers, displays etc. Specifically, the CF should work towards identifying the demands and constraints (lack of information, lack of time) placed on decision making in different aspects of military operations, then determine specific decision making strategies best suited to these conditions. Tools and procedures that clarify the link between analytic and intuitive decision strategies have the potential to enhance both plans and decisions made in the field. With such an analysis, the CF would be better placed to support, through training and technology, the employment of effective decision making procedures in appropriate situations.
The CF will likely be called upon to serve in non-traditional environments and perform a wider range of missions in the future. This will create a large number of problems for effective C2 (e.g., greater uncertainty, greater constraints on movement and action). Thus, it makes sense to pursue research into ways of supporting decision making in this environment. Recognition of the continuum of decision making strategies will be crucial to asking the right questions – questions concerning how to make planning and the communication of plans more effective, how to build common intent within units and teams,44 and how to design effective decision support at all levels of command.
Dr. David Bryant is a Defence Scientist with Defence R&D Canada, Toronto, where he is pursuing research on inferential processes involved in situation assessment and tactical picture compilation.
Dr. Robert Webb is Principal Consultant of Humansystems Incorporated, where his recent military work includes analyses of cognitive and information flow issues in a naval frigate operations room.
Carol McCann heads the Leadership and Trust Group in the Command Effectiveness Section at DRDC Toronto, where she has been involved in extensive investigations of human factor aspects of command and control.
NOTES
1. J. F. Schmitt and G. Klein, “Fighting in the fog: Dealing with battlefield uncertainty.” Human Performance in Extreme Environments 3 (1998), pp. 57-63.
2. J. O’Neill, “The Role of Problem-Solving in C3I Systems.” Proceedings of Second International Symposium on Command and Control Research and Technology. (Market Bosworth, Warwickshire, UK, 1996), pp. 348-355.
3. G. Gigerenzer, “Bounded rationality: Models of fast and frugal inference.” Swiss Journal of Economics and Statistics 133 (1997), pp. 201-218.
4. A. B. Dahl, “Command dysfunction: Minding the cognitive war.” (Unpublished thesis. School of Advanced Airpower Studies, Air University. Maxwell Air Force Base, AL, 1996).
6. J. R. Anderson, The Adaptive Character of Thought. (Hillsdale, NJ: Lawrence Erlbaum Associates, 1990). See also K. R. Hammond, “Functionalism and Illusionalism: Can integration be usefully achieved?” In R. M. Horgarth, eds., Insights in decision making (Chicago, IL: University of Chicago Press, 1990), pp. 227-261.
7. Anderson, 1990; G. Klein, “An Overview of Naturalistic Decision Making Applications” in G. Klein and C. E. Zsambok, eds., Naturalistic Decision Making (Lawrence Erlbaum Associates Inc, Mahwah, NJ, 1997), pp. 49-59; and A. Newell, Unified Theories of Cognition. (Cambridge, MA: Harvard University Press, 1990).
8. J. St.B. Evans, D. E. Over and K. I. Manktelow, “Reasoning, Decision Making and Rationality.” Cognition 49 (1993), pp. 165-187; J. W. Leland, “Informal reasoning in decision theory” in J. F. Voss, D. N. Perkins, et al. (Eds.), Informal Reasoning and Education (Hillsdale, NJ: Lawrence Erlbaum Associates,1991), pp. 209-223; and R. M. Roe, J. R. Busemeyer and J. T. Townsend, “Multialternative Decision Field Theory: A dynamic connectionist model of decision making.” Psychological Review 108 (2001), pp. 370-392.
9. Anderson, 1990; K. J. Holyoak, “Problem solving” in D. N. Osherson and E. E. Smith (Eds.), Thinking: An Invitation to Cognitive Science, Vol. 3 (Cambridge, MA: The MIT Press, 1990), pp. 117-146; and H. J. M. Tabachneck-Schijf, A. M. Leonardo and H. A. Simon, “CaMeRa: A computational model of multiple representations.” Cognitive Science 21 (1997), pp. 305-350.
10. G. Klein, Decision Making in Complex Military Environments. (Klein Associates, Fairborn, OH: Final Contract Summary Report, prepared for Naval Command, Control and Ocean Surveillance Center, San Diego, CA, 1992).
11. J. F. Yates, Judgment and Decision Making. (Englewood Cliffs, NJ: Prentice Hall, 1990).
12. Anderson, 1990; E. Hunt, “Problem solving” in R. J. Sternberg (Ed.), Thinking and Problem Solving. (San Diego, CA: Academic Press, 1994), pp. 215-232.
13. R. M. Hogarth, Judgment and Choice: The psychology of decision. (New York: John Wiley, 1980); See also Yates, 1990.
14. H. Ben-Zur, “Dimensions and patterns in decision-making models and the controlled/automatic distinction in human information processing.” European Journal of Cognitive Psychology 10 (1998), pp. 171-189.
15. P. C. Fishburn, “Normative theories of decision making under risk and under uncertainty” in D. E. Bell, H. Raiffa et al. (Eds.), Decision Making: Descriptive, normative, and prescriptive interactions (New York: Cambridge University Press, 1988), pp. 78-98. See also D. R. Luce and H. Raiffa, “Utility Theory” in P. K. Moser (Ed.), Rationality in Action: Contemporary approaches (New York: Cambridge University Press, 1990), pp. 19-40.
17. S.G. Hutchins, Principles for Intelligent Decision Aiding. (Naval Command, Control and Ocean Surveillance Center, 1996).
18. G. Klein, S. Wolf, L. Militello, C. Zsambok, “Characteristics of Skilled Option Generation in Chess.” Organizational Behavior and Human Decision Processes 62 (1995), pp. 63-69.
19. P. M. Todd and G. Gigerenzer, “Precis of Simple heuristics that make us smart.” Behavioral and Brain Sciences 23 (2000), pp. 727-780.
20. G. Gigerenzer, & D. G. Goldstein, “Reasoning the fast and frugal way: Models of bounded rationality.” Psychological Review 103 (1996), pp. 650-669.
21. Klein, 1992; Klein, 1997 and Hutchins, 1996.
22. Gigerenzer & Goldstein, 1996.
23. Although the term “Naturalistic Decision Making” (NDM) has achieved widespread acceptance, we prefer the term “intuitive decision making” because it more clearly distinguishes the assumed processes underlying this kind of model from those proposed by analytic theories.
25. R. Lipshitz, G. Klein, J. Orasanu and E. Salas, “Taking Stock of Naturalistic Decision Making.” Journal of Behavioral Decision Making 14 (2001), pp. 331-352.
26. Klein, 1997; and D. K. Leedom, L. Adelman and J. Murphy, “Critical Indicators in Naturalistic Decision Making.” Fourth Conference on Naturalistic Decision Making. (Warrenton, VA: Klein Associates, 1998).
27. Klein, 1992 and Klein et al., 1995.
29. Klein et al., 1995; and Leedom et al., 1998.
32. P. M. Todd and G. Gigerenzer, “Putting naturalistic decision making into the Adaptive Toolbox.” Journal of Behavioral Decision Making, 14 (2001), pp. 353-384.
33. L. Adelman, Evaluating Decision Support and Expert Systems. (New York: John Wiley & Sons, 1992); G. L. Kaempf, G. Klein, M. L. Thordsen and S. Wolf, “Decision making in complex naval command-and-control environments.” Human Factors 38 (1996), pp. 220-231; Leedom et al., 1998; Newell, 1990; and Yates, 1990.
34. J. A. Cannon-Bowers and H. H. Bell, “Training Decision Makers for Complex Environments: Implications of the Naturalistic Decision Making Perspective” in G. Klein & C. E. Zsambok (Eds.) Naturalistic Decision Making. (Lawrence Erlbaum Associates, Inc, Mahwah, NJ, 1997), pp. 99-110; Kaempf et al., 1996; Leedom et al., 1998; and R. Pascual & S. Henderson, “Evidence of Naturalistic Decision Making in Military Command and Control” in Klein and Zsambok (Eds.) Naturalistic Decision Making, pp. 217-226.
35. Lt Col. J.J. McMenamin, Operational Decision Making: The Impact of Time and Information. (Naval War College, Newport, RI, 1992).
38. A. B. Markman and D. Gentner, “Thinking.” Annual Review of Psychology 52 (2001), pp. 223-247.
39. It may be difficult to clearly distinguish where one class of decision making strategy ends and the other begins because the separation along the continuum becomes somewhat arbitrary.
40. Gigerenzer, 1997; and Todd and Gigerenzer, 2000.
41. Y. Xiao, P. Milgram and D.J. Doyle, “Capturing and Modelling Planning Expertise in Anesthesiology: Results of a Field Study” in Klein and Zsambok (Eds.) Naturalistic Decision Making, pp. 197-205.
42. E.M. Roth, “Analysis of Decision Making in Nuclear Power Plant Emergencies: An Investigation of Aided Decision Making” in Klein and Zsambok (Eds.) Naturalistic Decision Making, pp. 175-182.
43. M. L. Gick and K. J. Holyoak, “Schema Induction Analogical Transfer.” Cognitive Psychology 15 (1983), pp. 1-38; and Klein, 1997.
44. See C. McCann and R., Pigeau, The Human in Command: Exploring the modern military experience. (New York, NY: Kluwar Academic/Plenum Publishers, 2000).
