From the book "You are Solving the Wrong Problem"Chapter 10: Cognitive Bias
My favorite cognitive bias is the Gambler's Fallacy. Perhaps, this is because I enjoy going to the casino. It turns out that humans are exceptionally terrible at calculating probability. Not only are people bad at it, they fail to understand the most basic concepts. Casinos take advantage of this fact every day.
Most modern roulette tables feature a digital display that presents an extensive array of statistics regarding recent table activity. You can observe the previous 10 to 20 numbers that have been hit, the percentage of red versus black outcomes from the last 300 spins, as well as the hottest and coolest numbers. All this information is readily available as you prepare to place your next bet.
So, why would the casino provide you with all this information?
Let's consider the following scenario: you're seated at the roulette table, and the last 9 spins have resulted in red numbers. You are about to wager on a color, which is essentially a 50-50 bet. To be more precise, the presence of green zero and double-zero slots slightly tilts the odds in favor of the house. For now, let's ignore this factor.
Naturally, you might think that black is due to hit next after witnessing a string of reds. As a result, you place a significant bet on black, based on your hunch that it must come up after several consecutive red outcomes.
Was betting on black the smart move in that situation?
The previous explanation may seem counterintuitive, but it is crucial to understand that each spin of the roulette wheel is independent and unaffected by previous outcomes. The outcome of the next spin is not influenced by the color that has appeared in the previous nine spins. The probability of the next spin resulting in black is not more or less likely than it is to result in red. Therefore, there is no "smart" or opportune moment to place a bet on either color.
This concept can be challenging to grasp because it contradicts our initial intuition. It does not align with what we observe on the surface. However, over an extensive period of time, it is true that the ratio of red to black outcomes will approach a 50-50 balance. However, it is highly unlikely for this difference to ever be precisely zero. In most cases, the number of one color will surpass the other. Nonetheless, the crucial point remains: the outcome of any given spin is equally likely, excluding the impact of the zero and double-zero slots.
It is possible to explain this principle to people, especially seasoned gamblers who are already aware of it. However, when you are sitting at the roulette table and confronted with the display of past spins, there is a strong inclination to create your own "system." You might choose to play a combination of hot numbers or your personal favorites. Alternatively, you may opt for the color that has been occurring frequently. Occasionally, your "system" might yield a successful outcome. At that moment, the casino has you. They will profit from your belief that you have discovered a winning strategy. However, this perceived success is merely coincidental. The numbers that appeared were as likely as any other outcome. The probability of your system succeeding remains the same as before. Over an extended period, the odds are likely to catch up with you, favoring the house.
Emotion plays a significant role in this dynamic. After all, we are all human and subject to our own biases and inclinations.
Emotional bias
You receive a notice in the mail stating that you have been summoned for jury duty. Deep down, you hope that your group number will not be called so that you can return to your work uninterrupted. However, fate has a different plan, and your group is indeed summoned. Now, you have the opportunity to fulfill your civic duty. The defendant in the court case is facing charges of property damage and auto theft. Strangely enough, he bears a resemblance to someone who used to live in your neighborhood when you were growing up - a notorious troublemaker whom you always tried to avoid.
The question arises: can you genuinely serve as a fair and impartial juror? In reality, you have no personal connection to the defendant, and the person from your neighborhood has no relevance to this case. In theory, you should be able to approach this responsibility with impartiality. You may convince yourself that you can overcome any biases.
However, despite our best efforts, cognitive biases can unconsciously influence our thinking patterns. Cognitive biases are factors that cause our perception of events to deviate from reality. The example of jury duty contains the potential for an emotional bias. Defense lawyers would likely try to identify and eliminate such biases during the jury selection process.
Emotions play a significant role in how our brains process information. We tend to favor evidence that aligns with our existing beliefs - a phenomenon known as confirmation bias. This bias poses a significant challenge in the realm of problem-solving, as it can hinder objective analysis and decision-making.
The greatest hits of cognitive bias
It is imperative to acknowledge that cognitive biases operate without our awareness most of the time. As a result, it is our responsibility to train our minds to recognize and identify these biases, allowing us to approach situations objectively. Understanding cognitive bias is a crucial step toward overcoming its influence.
One of the most prevalent biases is confirmation bias, wherein people tend to favor information that aligns with their existing beliefs or hypotheses. This tunnel vision restricts the exploration of alternative solutions and leads to the dismissal of contradictory information. We often create excuses to explain away data points that don't fit our preconceived notions, convincing ourselves that they do not apply to our specific situation.
To mitigate this bias, it is essential to intentionally maintain an open mind, particularly at the beginning of any problem-solving process. Don't hesitate to challenge your own assumptions, whether they are explicitly documented or reside in the recesses of your mind.
Another bias that commonly skews decision-making is risk aversion. Our natural instinct tends to push us away from uncertainty. However, embracing calculated risks can often lead to more innovative and rewarding solutions. Recognizing this bias allows us to approach challenges with a more open mindset. While we should not be careless, being willing to take calculated risks as part of our prioritization can yield breakthrough outcomes, even if it requires confronting our fear.
The sunk cost fallacy is another bias that can mislead us into persisting with a failing course of action due to the investments we have already made. It is crucial to remember that, although we try to avoid it, backtracking and pivoting are sometimes necessary. This bias clouds our judgment and hinders our ability to recognize when a change in direction is warranted.
In group settings, individual brainstorming before coming together is recommended. Allowing each person to contribute their ideas independently helps prevent groupthink, where other people's ideas may bias our own or discourage others from sharing their thoughts. Additionally, be mindful of the order in which ideas are presented, as having more senior staff offer their ideas first increases the likelihood of groupthink.
To be effective problem solvers, we must break free from groupthink and detach ourselves from the influence of sunk costs. Each problem or subproblem should be approached with a fresh perspective, including the willingness to pivot when needed. Navigating the complex landscape of cognitive biases is crucial to achieving our best selves. As the saying goes, when armed only with a hammer, the temptation is to see every problem as a nail.
If bias is suspected, it is worthwhile to ask oneself, "What other perspectives can I take on this problem?" It may be helpful to step back, take a break, and return to the problem with a refreshed mindset in order to avoid being solely constrained by existing thinking patterns.
What type of problem solver are you?
Not everyone approaches problems in the same way. Some people are more intuitive, while others are more analytical. Some people are more open-minded, while others are more focused. People may be adventurous, or they may be cautious.
In her book Problem Solver, Cheryl Strauss Einhorn describes a Problem Solver Profile (PSP). It may help you to overcome cognitive bias by identifying your profile.
Note that these types are not mutually exclusive, and you may find yourself identifying with more than one of them. However, by understanding your dominant type, you can learn how to leverage your strengths and overcome your biases, as well as appreciate the diversity of other problem solvers.
The five types of problem solvers are:
-
Adventurer: You are quick, decisive, and trust your gut. You enjoy taking risks and exploring new possibilities. You are optimistic and confident in your abilities. However, you may also be prone to overestimating your chances of success and ignoring potential pitfalls. This is called the optimism bias.
-
Listener: You value input, feedback, and opinions from others. You are empathetic, collaborative, and respectful. You seek to understand different perspectives and learn from others. However, you may also be influenced by your emotions and social pressures. You may tend to favor people you like more or people who have more authority or expertise. This is called the liking bias and the authority bias.
-
Detective: You rely on facts, data, and evidence. You are logical, rational, and objective. You use systematic methods and tools to analyze problems and find solutions. However, you may also be influenced by how information is presented or interpreted. You may tend to look for evidence that confirms your existing beliefs, or ignore evidence that contradicts them. This is called the framing bias and the confirmation bias.
-
Thinker: You need time, space, and process to solve problems. You are careful, thorough, and deliberate. You weigh the pros and cons of different options and consider the consequences of your actions. However, you may also be influenced by your aversion to losses and your tendency to compare alternatives. You may tend to avoid making choices that involve losses, or choose options that are better relative to others, but not necessarily optimal. This is called the loss aversion bias and the relativity bias.
-
Visionary: You seek to create unique, different, and innovative solutions. You are creative, imaginative, and original. You challenge the status quo and break the rules. However, you may also be influenced by your attraction to scarce or novel ideas. You may tend to prefer options that are rare, exclusive, or bold, but not necessarily feasible or effective. This is called the scarcity bias.
How do you know if you are right?
"Until we know we are wrong, being wrong feels exactly like being right." – David McRaney
One of the most important skills in problem-solving is to evaluate your own solutions and determine if they are correct. In cases where absolute data does not exist, this can be a challenge. Some people suffer from overconfidence, while others have the optimal solution in hand but doubt their own abilities.
There may be multiple ways to approach a problem, different criteria to measure success, and various sources of uncertainty and error. How can you be confident that your solution is the best one, or at least a good one?
There are a few strategies that you can use.
- Test your solution with different scenarios: After you have a solution, you should test it with different scenarios or cases. These are the situations or examples that illustrate how your solution works in practice. For example, you may test your solution with normal, extreme, or edge cases. You should compare the expected and actual outcomes, and check if your solution meets the requirements and objectives of the problem. If you find any discrepancies or failures, you should debug and improve your solution accordingly.
- Seek feedback and opinions from others: Seek feedback and opinions from others who are familiar with the problem or the domain. If you need to convince others in your organization, talk with the stakeholders. These are the people who can provide you with different perspectives, insights, and suggestions. You may seek feedback from your peers, mentors, or experts. Listen to their comments and critiques, and evaluate them objectively. If you find any gaps or flaws, you should refine and enhance your solution accordingly.
Sinking software sales
Imagine your company has launched a new software product, but initial sales have been disappointing. As the product manager, you need to uncover why sales are below expectations and identify the key problems to solve.
- What types of confirmation bias would you need to be aware of or overcome while doing this analysis?
- What process could you use to objectively deal with the issue?
Confirmation bias would be a significant risk. As the product manager, you have a deep understanding of the product. You likely have some level of emotional attachment to it, since you were involved in creating and defining it. Thus, emotional bias is also a concern. It can be difficult when something you have created does not perform well. You must do your best to separate yourself from these biases and look objectively at the data. This is the best way to succeed. It could be that only a minor change makes all the difference, but a fresh perspective will be required to discern this.
Here is an objective process you could follow:
- Write down your initial hypothesis for why the new software is not selling well.
- List out 3-5 possible reasons for the poor sales. These should include your initial hypothesis as well as alternative hypotheses.
- For each hypothesis, objectively list out data points or evidence that could confirm or disconfirm it. Do not just focus on confirming data.
- Gather the relevant data/evidence to evaluate each hypothesis. Track both confirming and disconfirming information.
- Review all evidence for and against each hypothesis.
- Based on the full body of evidence, determine which hypothesis holds up best. This may or may not be your original hypothesis.
- Identify your key findings and the main problem(s) to solve that are supported by the objective data.
It can be more difficult than we think to understand the true nature of the problem. Next, we examine why this is the case. The Problem Paradox will help to unravel the complexity found within our toughest problems.
Chapters
2. Step out of Auto-Pilot Mode
3. Your most important skill: Asking Questions
4. Solve any problem using the Five Whys
5. AI Prompting like a Pro
6. Invention over Convention
7. How to Reframe a Problem to your Advantage
8. The Diamond Pattern: First fan-out, then fan-in
9. The Problem Paradox
10. Break free from Cognitive Bias
11. From Complex to Simple
13. Use AI for Data Analysis
14. The Curiosity Rule
15. Improve productivity by eliminating distractions
16. Optimization Problems
17. Greenfield or work within current constraints
18. Managing the most scarce resource: Time
19. Challenge yourself with Puzzles
20. Next Steps
21. Solutions to Puzzles