Welcome to the age of unintended consequences

So one of my favorite things about decision intelligence is its promise to help to overcome unintended consequences. As a way to capture both mental models, as well as providing an ongoing infrastructure to gather evidence to support and refine what start out as mental models and end up as sophisticated systems models, I’m tremendously excited about the future of what we can do.
[bctt tweet=”Look at failures through the lens of unintended consequences, then fix broken systems.”]
I’ve been maintaining a scoop.it site with interesting examples of unintended consequences, and starting to detect systematic patterns:
Exploit: a new system is created, and certain persons discover that it can be used to achieve an outcome that was not intended by its designers. See this link about how a government transparency initiative led to opposition research firms overloading its opponent with requests.
Single-objective tunnel vision: The idea here is that we focus on just one objective, and don’t realize the other, less desirable results of our decisions. See this example where a focus on law enforcement created a number of undesired knock-on effects.
Ignoring intangibles: Here, an intangible aspect of a decision – often a psychological or behavioral component – has a bigger effect than the tangible, measurable things. But since the tangibles are easier to measure, they get over-emphasized in decision making. I see this one everywhere. I’ll leave it as a challenge for you to find an example and post in the comment, please.
Ignoring feedback effects: Here, we don’t realize that many systems have “tipping points”, which lead to a rapid escalation. Arms races are this. Perhaps Ferguson is as well.
Misunderstanding catalyst / synergistic effects: For instance, dispensing a new medicine without training doctors on how to administer it. Or, more subtly, fixing one necessary component of a system (say, law enforcement) without understanding that it will have limited traction if others are not fixed (e.g. corruption). Here’s an example where school standards were raised in an effort to improve education, but it only increased dropout rates.
Bad link understanding: This is a basic one. It’s when we think that a cause-and-effect link will go one way, but it goes another.
Spaghetti: This is when the rules that make up a system – legal or otherwise – become so complex that it begins to break down, creating an incentive to band-aide patch it, as I talked about in my blog article on this topic last week.
[bctt tweet=”Cure unintended consequences using a systems model, informed by crowd expertise and data.”]
Can you identify any other unintended consequences patterns? Or just good examples of specific unintended consequences? Please feel free to add comments below.
Are you passionate about this topic? I’m looking for a few new curators for the Unintended Consequences scoop.it site. You’ll see that each entry has a decision, intended consequence, unintended consequence, and analysis which often identifies one of the above patterns. It’s fun work. Drop me a line if you’re interested.
Once you start looking, you’ll see these everywhere. Fixing them gets to the heart of many broken things.
Image credit: Graham Roumieu
Computer scientist Dr. Lorien Pratt is one of the pioneers of artificial intelligence and is credited with inventing transfer learning. Following a 36-year career delivering applied AI systems, along with leadership and program committee responsibilities at the industry-leading NeurIPS conference, Pratt is recognized alongside Marie Curie and others by the Women Innovators and Inventors Project. Today, Pratt is chief scientist at www.quantellia.com, where she continues to push the boundaries of technology as a creator of and evangelist for Decision Intelligence (DI), a cofounder of www.opendi.org, and through building Large Language Models (LLMs) integrated with DI and ML. Pratt’s The Decision Intelligence Handbook (O’Reilly, 2023) is receiving worldwide attention. With recent publications in Foresight, Futures, and Frontiers, Pratt's DI innovations connect AI to human decision making in a way that is transparent, auditable, and accessible, bringing this important technology to the masses, and reducing information inequality.