Guest Post: The Decision Command Fallacy (Part 2)
How to avoid unintended consequences of decisions in complex environments
Decision failure patterns in complex environments
The two decision failures I covered in Part 1 have different causes, but follow similar patterns. The hit man example is a case of “throwing the baby out with the bathwater”: the unintended consequences of achieving the outcome end up nullifying that very same outcome, as shown below:
If the Joe in our story had taken the time to think through the possible outcomes that negatively impact the desired objective, he could have instructed the hit man to eliminate any witnesses, unless they included Joe’s mistress, in which case he should not go ahead with the operation (but of course, that wouldn’t have made for much of a psychological thriller).
The prohibition example follows a similar pattern, and is one of the most difficult to plan for. It involves an adaptive system that has built into it a way of evaluating how “favorable” its current state is. Here, the system is in control, and if you just perturb it you’ll likely get to a place you hadn’t anticipated.