How to approach the horse problem

  • Posted on: 12 March 2009
  • By: Michael Brydon

A Decision-Making Methodology

I make my living researching and teaching decision-making.  As such, I have some strong beliefs regarding how a complex issues such as "the horse problem" should be tackled.  The first element of my approach is a structured approach consisting of five distinct stages:

  1. Gap analysis:  Is there a problem?  Is there a gap between the desired state of the world and the actual state of the world?
  2. Root cause analysis:  What is the cause of the gap?  How certain are we that we understand the true source of the problem?
  3. Identifying goals, objectives, and trade-offs:  What are we seeking to accomplish?  What are we willing to give-up to solve the problem?
  4. Generating alternatives: What are the different ways that we can solve the problem in a manner consistent with our goals and objectives?
  5. Making a decision:  What are the expected consequences of each alternative?  Which set of consequences is best given our goals and objectives?

The five-stage process described above is fairly standard in the decision-making and problem solving literatures (e.g., Kepner & Tregoe, 1965; Hammond, Keeney & Raiffa, 1999).  The advantage of a structured approach is that that the objectives, desired outputs, and appropriate methodologies at each stage are clear to all participants.  For example, the process of generating alternatives is essentially creative: decision makers are asked to come up with many different ways of tackling the problem.  Evaluating the alternatives, in contrast, is a more analytical and may require specialized modeling techniques.

All stages of the decision process are important.  However in local government two stages require special attention to ensure that stakeholders are involved: (3) Identifying goals, objectives and trade-offs and (4) Generating alternatives.  The rationale for involving stakeholders in setting goals and objectives should be readily apparent.  Goals and objectives provide the basis for generating and selecting from among alternatives.  If, for example, residents of a certain area value “controlled development” more than “economic growth” whereas these priorities are reversed in a different region, then it is natural expect significant differences in “optimal” outcomes in the two regions.  Stakeholder input is also critical during the alternative generation stage.  Clearly, the quality of a decision is bounded by the quality of the alternatives considered by the decision makers.  The objective is to draw on as much experience and creativity as possible when generating possible solutions to a tricky problem.  Recall that evaluation of these alternatives (Is it practical? Is it cost efficient?) does not occur until the subsequent decision making stage.

Mechanisms for Stakeholder Involvement

Currently, local governments rely primarily on group meetings for decision-making.  There appears to be a general belief that a meeting that is properly conducted (e.g., according to Roberts Rules of Order) will result in a high quality decision.  Unfortunately, this view conflates “order” and “rationality”.  The organization and management literatures continue to unearth significant theoretical and practical problems with group meetings as a means of decision-making.  For example “groupthink” (a false consensus that results from specific group dynamics) has been shown to be a barrier to high quality decisions. “Focusing” (making a premature commitment to a binary decision concerning a single alternative) is another common decision making problem. And, of course, there are group meetings that just end up going sideways due to interpersonal conflict.

The Delphi technique was developed by the RAND Corporation in the late 1950s to help create forecasts in the highly uncertain Cold War environment.  The technique is being slightly modified for our purposes in order to elicit more than just technical forecasts of (say) the probability of a nuclear first strike by the USSR. One way to think of the technique is as a series of surveys to be filled out by experts and/or stakeholders.  Surveys are used instead of group meetings in order to elicit the independent contributions of individuals and avoid the barriers to expression and creativity imposed by meetings.  The process is iterative—that is, there is a series of surveys—so that consensus has an opportunity to emerge.  The aggregate responses from each round of the survey are revealed in the subsequent round so that participants on the Delphi panel can see where there is consensus and where there is disagreement.

The availability of high quality, open source survey software makes it relatively easy and very inexpensive to run Delphi surveys over the Internet.  I have done this before in a research context and participants found the scoring process to be simple and convenient.  The most difficult task is compiling the results of each stage.  This task is made easier if the panel participants understand the process.