ChatGPT Does Decision Intelligence for Net Zero
Unless you’ve been hiding under a rock, you’ve probably heard about ChatGPT from OpenAI, which answers your text questions in the style of your choosing (including rock songs, poetry, and screenplays), based on language it’s obtained by reading the internet. ChatGPT’s performance is both uncanny and disturbing, both because it seems so intelligent but also because it gets so many answers wrong.
After some initial brain melting, I’ve been left wondering, “How can ChatGPT help us the most?” and “Is there value here?” After all, a computer that has ingested a substantial chunk of human knowledge can probably be a great assistant for some tasks. But how?
Based on the analysis I’ll describe in more detail below, I’ve come to believe that Decision Intelligence (DI) is a “killer app” for Large Language Models (LLMs) like ChatGPT: DI is a discipline in which ChatGPT can provide substantial value. This post is about how I’ve reached this conclusion, and includes a session with ChatGPT about Net Zero GHG emissions that illustrates my point. And thanks to the brilliant Jeffrey Williams, tech lead for the UC Berkeley bConnected team, for his help with the prompt engineering.
By way of background, DI is a discipline that helps you answer the question, “What should I do to achieve my desired outcomes?” DI software goes beyond simple data dashboards and visualizations to explain the path from your actions to your desired outcomes. “What price should I charge for my product?”, “Is it safe to stop wearing a mask?”, “Should I offer this course in person or online?”, and “How should I spend my budget?” are examples of DI questions.
Some people see DI as the next generation of AI, as a ML model orchestration framework that connects ML models as “raw ingredients” into decision simulation and optimization. Others see DI as the next generation of Business Intelligence (BI): helping organizations on their digital transformation journeys to get to the next level to connect KPIs to business outcomes.
Both are valid lenses. In any case, DI is projected to be a $36B industry, is featured on three Gartner Hype Cycles and two “Impact targets,” and is the focus of an emerging vendor market.
As I teach in the new Getting Started with Decision Intelligence (ChatGPT edition) course, we’ve found that ChatGPT is very helpful, in particular, in creating a Causal Decision Diagram (CDD): a map of the connections between actions and outcomes. CDDs are used to both get humans on the same page regarding the structure of a decision, and are also used as specifications for software systems to run decision simulations and to monitor data-driven decision making.
Here’s what I’ve tentatively concluded:
- ChatGPT is often wrong, BUT
- ChatGPT has access to lots of the internet’s text, SO
- It turns out to be a pretty great collaborator when it comes to surfacing ideas for decision elements you might not have thought of, if you use a structured process (actions to take, externals to measure, unintended outcomes that might be achieved, plus the links between them), BUT
- These require a human to curate / evaluate for correctness AND
- We’re finding anecdotally that, because recognition is easier than recall, there’s a BIG benefit to doing it this way instead of depending on humans alone to create Causal Decision Diagrams (CDDs) from scratch without this super collaborator.
Check out this dialogue (based on a use case described in more detail in the DI Handbook) to see what I mean. (Note that my side of the conversation contains a few typos; I’ve left them in so you can see that ChatGPT understands me nonetheless.)
Me: I work for a central bank in Europe. We are considering how we can reduce our greenhouse gas (GHG) emissions as an organization, with a focus on reducing travel. But we want to maintain our relationships with our clients. I would like to use a decision intelligence approach, specifically creating a causal decision diagram. Please ask me a series of questions, one at a time, in interview fashion, to elicit my knowledge about this decision. First ask me about outcomes, then actions, and then help me to determine how those actions might lead to outcomes through intermediates. Also please suggest to me what actions I may not have considered that Net Zero experts might consider, and what outcomes – including unintended consequences – may occur.
ChatGPT: Great, let’s start with outcomes. What are the desired outcomes that your organization is aiming to achieve by reducing GHG emissions and travel?
Me: We would like to be climate Net Zero by 2030, while maintaining the quality of our relationships, and not incurring additional costs, year-over-year.
ChatGPT: That’s a great goal. Let’s talk about potential actions you could take to reduce travel and GHG emissions. What actions are you currently considering or have you already taken to reduce travel and GHG emissions?
Me: We’re considering creating a new policy specifying who can travel by air, and who should attend meertings and conferences remotely
ChatGPT: That’s a good start. Can you tell me more about the potential actions you are considering to reduce GHG emissions and travel? Are there any other actions under consideration?
Me: Why don’t you suggest some, please?
ChatGPT: Sure, here are some additional actions that Net Zero experts may suggest for reducing GHG emissions and travel:
- Encouraging the use of public transport or low-emission vehicles for local travel.
- Implementing telecommuting policies to reduce the need for travel.
- Encouraging virtual meetings through video conferencing or other means.
- Implementing a carbon offset program to offset the GHG emissions produced by travel.
- Providing training for employees on sustainable practices and promoting sustainable behavior.
ChatGPT: Have you considered any of these actions, and are there any additional actions you would like to add to this list?
Me: yes, we are looking at 3 and 4. But we’re not sure how effective virtual meetings are, nor how to make the right tradeoff
ChatGPT: That’s a good point. Virtual meetings can be effective, but they may not always be the best option for building and maintaining relationships.
ChatGPT: Let’s move on to the causal decision diagram. How do you think the actions you’ve mentioned might lead to the desired outcomes? In other words, what are the intermediate variables that might link the actions to the outcomes you’re hoping to achieve?
Me: I think that the type of meeting that we hold will impact our reputation worldwide, and the quality of our relationships, but I’m not sure exactly how much. Reducing air travel will reduce travel costs, as well, but perhaps too much, because our relationships could suffer more than the air travel savings benefits us
ChatGPT: Those are all important points to consider. To build a causal decision diagram, we need to break down the variables and relationships between them. Here are some variables and relationships that may help you create the diagram:
- Actions: This variable represents the potential actions you’ve identified, such as implementing a new travel policy or promoting virtual meetings.
- Intermediate variables: These variables represent the outcomes that may occur as a result of the actions you take. For example, promoting virtual meetings may lead to reduced air travel, lower travel costs, and reduced GHG emissions. On the other hand, it may also impact relationships with clients, reputation, and quality of communication.
- Desired outcomes: These variables represent the ultimate goals of your organization, such as becoming Net Zero by 2030, maintaining relationships with clients, and not incurring additional costs.
- Unintended consequences: These variables represent the unexpected outcomes that may occur as a result of the actions you take. For example, reducing air travel may lead to increased reliance on local travel, which may increase GHG emissions if not managed effectively. Similarly, promoting virtual meetings may lead to reduced travel costs but may also impact relationships with clients if not managed well.
ChatGPT: To build the causal decision diagram, you would start by identifying the relationships between these variables, including any feedback loops, trade-offs, or synergies. Does that help you get started?
Me: yes. Can you try drawing the CDD in ascii text please
ChatGPT: Certainly! Here is an example of a causal decision diagram in ASCII text:
ChatGPT: Note that this is just an example, and the actual diagram would depend on the specific variables and relationships that you identify in your decision-making process.
Me: OK that structure looks like a good start. Can you identify any models I can use to inform the behavior of the causal links (arrows) in the diagram? Please give me specific references, especially if there are some public models or data available through an API.
ChatGPT: Certainly! There are a few models that you can use to inform the behavior of the causal links in your decision diagram. Here are a few examples:
- Carbon footprint calculators: These calculators can estimate the carbon emissions associated with various activities, such as air travel or video conferencing. You can use this information to estimate the potential GHG emissions reduction associated with reducing air travel and promoting virtual meetings. Some examples of carbon footprint calculators include the EPA’s Carbon Footprint Calculator and the Carbon Trust’s Footprint Calculator.
- Climate scenarios and models: Climate models can help you assess the potential impacts of GHG emissions reductions on climate change. These models can also help you evaluate the feasibility of your Net Zero targets. Some examples of climate scenarios and models include the IPCC’s AR6 Climate Scenarios and the Climate Interactive’s En-ROADS simulator.
- Social impact assessments: Social impact assessments can help you identify the potential social and economic impacts of your decision, including impacts on relationships with clients, reputation, and quality of communication. Some examples of social impact assessment tools include the World Bank’s Social Impact Assessment Tool and the International Finance Corporation’s Social Impact Assessment Guidelines.
- Economic impact assessments: Economic impact assessments can help you evaluate the potential costs and benefits associated with your decision. Some examples of economic impact assessment tools include the World Bank’s Cost-Benefit Analysis Tool and the EPA’s Economic Analysis for Proposed and Final Clean Air Act Regulations.
ChatGPT: I hope these resources are helpful!
I spent a few more minutes trying to explain to ChatGPT how to create a better (formally, “well-formed”) CDD. I didn’t have a ton of luck, but I did get it to generate this view, which I think is a bit better than the one above:
Stepping back from the above dialogue, I asked myself, “Is this useful?” and “Is this wrong?” I evaluated the various actions, outcomes, externals, and models that ChatGPT suggested, with the results shown below:
This chart shows my analysis of the various suggestions made by ChatGPT, compared to the previous elicitation method, which used only human researchers, and which resulted in several CDDs, one of which is shown below:
CDD for a government agency looking to reduce GHG emissions through air travel policy.
Reprinted with permission from The Decision Intelligence Handbook. Copyright (c) 2023 Quantellia LLC
Here’s the raw data underlying the bar chart above, showing suggestions from ChatGPT. I manually removed redundant elements, categorized them into the various decision element categories (sometimes ChatGPT’s categorization was wrong), googled a few things to see if they were correct or not, and manually labeled each suggestion, as shown:
- As shown in the “new idea” column, ChatGPT suggested several decision elements that were both in scope and also potentially useful for the project.
- I was surprised that none of the ChatGPT suggestions were wrong, as far as I could tell.
Given that the prior human-user-only CDD work required substantial time and effort, I think that ChatGPT has enormous potential to A) save time during CDD creation and B) surface new ideas for the CDD that were not previously considered.
Why is this useful? Because a CDD is one of the most powerful ways to connect human decision-makers to technology, to solve the most complex commercial and human problems of our time.
And, of course, the use of DI goes well beyond this Net Zero use case, to many problem domains, as I wrote about in Link: How Decision Intelligence Connects Data, Actions, and Outcomes for a Better World.
A side note for my data scientist friends: most work in causal modeling involves learning causal connections (like the ones in CDDs) from data. Although valuable, this formulation places an unnecessary limitation on immediately driving decision-making value. Obtaining causal information from people—and, as shown here, human-generated text on the internet—is a much quicker and more general way (because there’s lots of causal information that is not contained in data sets) for organizational outcomes to benefit from new technologies.
Want to learn more? Check out the Decision Intelligence Handbook and my new Getting Started with Decision Intelligence (ChatGPT Edition) 8-week online course, starting soon.
PS: I couldn’t resist…