What to do when complexity starts to eat you alive: in law, government, and organizations

I had a house once with a lot of stuff, and a big basement. Sometimes when I needed to find something quickly – a Philips screwdriver maybe, or a glue gun – there were just too many places to look, and it would have taken hours. So, I’ll confess, there were times I just went and bought a new screwdriver. Which added to the pile, and made it worse for the next time around. At the time, I always seemed to be too busy to take the time to organize things so this would stop happening. So for a few years it just kept getting worse.

I’m starting to see this pattern more broadly.

In a recent consulting project, I learned that a large organization in a slow-moving industry was suffering from the same problem with its procedures manuals. These documents specified how certain tasks were to be performed, and they’d reached the point where there were contradictions. Instead of taking a deep look at the inconsistencies, this organization sometimes created new rules meant to clarify the old ones, but the result was just a basement full of more stuff: a band-aide patch, that over time just made the problem worse.

In The Rule of Nobody: Saving America from Dead Laws and Broken Government, Philip K. Howard explains that the same is happening to modern government. If you think of government as sort of a “legal machine”, of legal gears and presses, getting ever larger with new regulations and programs, then Howard claims government leaders are not decision makers but rather act more like mechanics, “scurrying here and there to attend to the legal gears and machinery”. The theory is that, somehow, decision-making can be relegated to an automatic process, encapsulated in law, and that rule of law can be obtained by simply turning the crank.

As a software engineer, I realize I’ve seen this situation before.   It’s what code looks like before it’s been refactored. Because our systems must work efficiently for our companies to survive, software engineers have developed a comprehensive architectural best practice, starting with the concept of code smell (an indicator of the need for redesign) and including best practices that are captured in a number of books and resources on the topic, including a menagerie of cross-functional design patterns to get it right. The ability to refactor well is an important discipline to control the otherwise exponential increase in complexity.

So why haven’t we used these great best practices in the more vexing problems of organizational design, policy creation, and improvements to government regulations? I believe it’s because we haven’t come up with a way to think about these systems in a way that lends itself to good analysis, continuous improvement, evidence-based analysis, and collaboration. These and other engineering disciplines depend on a cross-situational way to think about these problems.

After having worked with a few organizations within these complex situations, I’ve found a solution. First, you need to cut through the complexity to find the critical causal paths through the system. What are the outcomes that we wish to achieve? What is the chain of events, data, and/or reasoning that lead from actions to outcomes?   Usually the links are a hodgepodge, with some links being a process (this-before that), others being a calculation (this-is-part-of-that), and more. You can’t impose an unfamiliar formalism or keep within a specific way of expressing things (e.g. formal logic, as we used to do in the Expert Systems days).  This needs to feel more like visual mind mapping than writing code.

Finding these paths depends critically on an expert who’s lived within the system for years, and knows where the important gears are in the machine: the linkages that matter. Indeed, this is the core skill for any smart leader in a complex organization.   A good decision engineer can work with them and their team to draw a map of this chain, and then to gather the evidence and data to understand it, and then to improve it.

And my house full of screwdrivers?  Ultimately, I gave it all away.  It felt wonderful.

Are you facing a problem like this?  I’d love to help.

Pratt has been delivering AI and DI solutions for her clients for over 30 years. These include the Human Genome Project, the Colorado Bureau of Investigation, the US Department of Energy, and the Administrative Office of the US Courts. Formerly a computer science professor, Pratt is a popular international speaker and has given two TEDx talks. Her Quantellia team offers AI, DI, and full-stack software solutions to clients worldwide. Previously a leading technology analyst, Pratt has also authored dozens of academic papers, co-edited the book: Learning to Learn, and co-authored the Decision Engineering Primer. Her most recent book: Link: How Decision Intelligence makes the Invisible Visible (Emerald Press) was published in 2019 and was a finalist for the prestigious PROSE award. With media appearances such as on NPR<,CSPAN, TechEmergence and GigaOm, Pratt is also listed on the Women Inventors and Innovator’s Project. Pratt blogs at www.lorienpratt.com.

You may also like...