Monty Hall, Storytelling, and Planning

---

---

In this post we are going to take two fascinating topics, the Monty Hall Problem and the Narrative Fallacy, and see what they can teach us about product planning. Enjoy!

1. Let’s Make a Deal

A lot has been written about the Monty Hall Problem, a famous version of Bertrand’s Box Paradox from 1889. If you’re not familiar, here is a quick rundown. The story goes back to the popular gameshow, Let’s Make a Deal, which first aired in the US in 1963, hosted by Monty Hall. On the show, contestants, called “traders,” would usually be offered an item of value and then given the option to keep it or trade it for an unknown item, which would either turn out to be of equal or greater value or what the show called a “Zonk,” a booby prize.

---

---

---

The “Monty Hall Problem,” as it is now commonly known, derives from a game where traders were shown three closed doors. Behind one was a cash prize. Behind the other two were Zonks (in this case goats). When the trader chose a door, Monty would have one of the other two opened, revealing one of the two Zonks. This left two closed doors. Monty would then ask the trader if they’d like to stick with their initial choice or switch to the other closed door…and at this point the world of mathematics erupted into bedlam. (I’m joking, sort of.)

In 1990 a reader posed this as a problem to Parade magazine’s “Ask Marylin” column. The controversial question here of course is, “Should you switch doors or stick with your initial selection?” Marylin vos Savant argued that you should always switch. Parade famously then received more than 10k letters from people arguing that Marylin cannot be correct, including 1k letters from people with PhDs.

Their argument is that when the first door is opened, revealing a Zonk, that removes it from consideration. Since there are only two doors left, there is then a 50/50 chance the cash prize is behind one of them. Thus, switching doesn’t matter. Further, many people want to “stick to their guns” and stand by their initial guess. This reasoning, which is wrong, is how most of us react to the problem. Incidentally, this why it is considered a “veridical paradox.” (It is also known as the “Monty Hall Paradox.”)

---

---

---

A veridical paradox has a solution which, though correct, seems too absurd to be true. The correct solution, as Marylin vos Savant argued, is that you should always switch doors. The odds for the initial choice are determined when that choice was made. They do not change later. When the choice is made, there are three doors. There is a 33% chance the prize is behind the chosen door and a 66% chance it is behind one of the others. Thus, when Monty reveals the first Zonk, there is still a 66% chance the prize is not behind the chosen door.

If this just seems wrong to you, don’t worry. Call it a cognitive illusion. There have even been mathematicians who have refused to accept this reasoning. So how do we know they are wrong? One way is to play the game repeatedly, which can be done as a computer simulation. An idea I had many years ago was to create a simulation where the number of doors increases every time you play.

At the time I worked in a lab conducting decision making research. We designed and ran an experiment and found this does help people see through the paradox, but only if the opened doors remain on the screen. To illustrate, imagine that instead of three, there are 50 doors! There is a cash prize behind one of them. Let’s say you choose Door 36. The odds the prize is behind this particular door are 1 in 50, or 2%.    

---

---

---

Just like the original game, Monty has all the other doors opened except one, revealing a staggering 48 goats. He asks if you would like to stick with your initial guess or switch to the other closed door. There are only two closed doors left. This does not all of a sudden mean there is a 50/50 chance the prize is behind either door. There is still a 2% chance the prize is behind Door 36…which means there is a 98% (49 in 50) chance it is behind Door 4. As Marylin correctly argued, you should always switch.

---

---

---

2. Tell Me a Tale

When some event occurs, knowledge of the outcome makes particular antecedent events stand out to us. We then link these together into a causal narrative. If the resulting story seems to connect these salient dots, then the known outcome feels like it “makes sense.” This feeling of plausibility is heuristically used to gauge diagnosticity. This is called the “Narrative Fallacy.”

Oddly, the selective directionality of this doesn’t seem to bother us: If our p à q conditionals correctly “explain” events afterwards, then why don’t they help us predict them beforehand? Perhaps out of what is an indeterminate causal matrix, specific relevant causes only become salient vis-à-vis some specific known effect. Chalk it down to the hindsight bias then. As Kierkegaard so nicely put it, “Life can only be understood backwards; but it must be lived forwards.” But there is something more going on here.

As Berndt Brehmer noted in his seminal article, “In One Word: Not From Experience,” the real world is often nonlinear and probabilistic, whereas the narratives we use to make “sense” of it are linear and deterministic. This is a category error. Failing to realize this, we get caught in a regressive trap: When one linear deterministic narrative gets knocked down, we just search for another that seems to again make “sense” of things. No amount of such experiences will dissuade us from this habit—we just go on telling our stories.

---

---

---

Nothing in this arrangement generates the healthy skepticism of Beckett’s Molloy, wondering if in telling his story whether he is “merely complying with the convention that demands you either lie or hold your peace.” It will not occur to us that there might be something wrong with this mode of explanation altogether, that perhaps the narrative is not a driving force as much as a hanger-on.

A story that weaves many points into a cohesive narrative makes them together seem more likely to occur, ignoring that, mathematically, this is impossible. Something that seemed unlikely is not somehow made more probable by a narrative explaining it. In fact, the many claims that make up the narrative are necessarily together less likely to be true than the initial event that seemed so unlikely. (Pause and consider what this implies for the entire field of history, which is largely composed of such ad hoc rationalizations.)

Cognitive psychologists call this the “Conjunction Fallacy,” which could be thought of as violation of Ockham’s Razor, the maxim which states that, “Plurality should not be posited without necessity.” The same of course applies in science: All else being equal, adding to any hypothesis in order to counter a challenge ironically makes that hypothesis less likely to be correct. In a Bayesian sense, each and every assumption added reduces the prior probability of the overall hypothesis.

Speaking of such fallacies, it is perhaps worth noting that these aren’t exactly new insights. Buddhist philosophers have long argued that narrative descriptions rarely accurately summarize reality. To close this section, as the great Berndt Brehmer so aptly put it, the way out of this trap is to reason probabilistically but, rather cruelly, you cannot detect probabilism in the world if you cannot already reason probabilistically.

3. The Crystal Gazer

As with the traps above, so goes product planning. We act like we have a crystal ball. We slap a timeline on a narrative and treat it like a truth-preserving deductive argument: If we do p, then q. If q, then…success. Unfortunately, the real world does not care about our Gantt charts. Planning is not some form of alchemy, transmuting complexity into orderliness. The predictability that traditional approaches to planning assume is largely mythical.  

---

---

---

The real problem with work, we tell ourselves, is all that pesky procrastination and lack of coordination. Without these dates as forcing functions no one would do anything! So, we coordinate and PM and shepherd along, all the while keeping the focus on the efficiency of output. We celebrate when the thing decreed upfront is delivered—whatever it may be—and seldom if ever look at the actual outcomes of all this Sturm und Drang.

It takes bravery to stand up to such theater. It’s so much easier to just play the game and then pat ourselves on the back. As Rob England and Cherry Vu point out in their excellent book, S&T Happens, as long we keep the focus on the output planned upfront, ignore actual outcomes, and collectively pretend an unreal result for appearances’ sake, we’ll likely be rewarded for yet another performance in the theater of “define once, execute perfectly.”

To counter this trap, we need to stop pretending like we’re not placing bets. That’s what a “plan” is, after all, a collection of bets laid out as a chain of prespecified options. It’s like stating all the moves in a game of chess while it’s still the opening. Traditional PM thinking argues this is necessary, else strategy will be left to chance—an obviously fallacious position. Strategy instead should be up-levelled to a proper focus on ends, letting the means evolve based on the latest ground intelligence, akin to the now-popular military concept of Auftragstaktik.

To paraphrase England and Vu, we’re not navigating to a star here, we’re catching a firefly. We’re operating in a VUCA domain, which England and Vu define as “volatile, uncertain, complicated, and ambiguous.” This means things are more nonlinear and probabilistic than linear and deterministic. Thus, all the lessons of the Narrative Fallacy, the Conjunction Fallacy, and the Monty Hall Problem come to bear. To focus only on output is to commit a category error.

As in the Conjunction Fallacy, one can take the riskiest bet in any plan and know it has better odds of succeeding than the plan itself. To ignore this violates Ockham’s Razor. The reality this leaves us with is closer to the Monty Hall Problem than many of us would care to admit. The more unopened doors are built into a plan, the less optimized it is. Product work should therefore be treated more like A/B testing. To know if an option is more optimal than another, it must at least be compared to alternatives. We should be generating and exploring alternative options to maximize the outcomes obtained, which is akin to looking behind various “doors.”

---

---

---

Just as in the Monty Hall Problem, it behooves us to switch “doors.” Insisting that we “stick to the plan” in light of later better information is akin to refusing to switch. This persistent but irrational honoring of psychological sunk cost robs us of any possibility of real agility and is a barrier to better outcomes. In fact, this is perhaps the single greatest hurdle to creating more value: the bizarre insistence that “success” be framed in terms of delivering something specified in the past, thus unnecessarily spending decision degrees of freedom in a state of greater uncertainty.

Software or no, the reality is the more decisions that have already been made, the fewer options realistically remain. Agility is fundamentally about getting smarter about how we spend our degrees of decision freedom. (A backlog, properly employed, is a decision degrees of freedom buffer.) Agility, at the end of the day, is more about the distribution of decision authority in the system than anything else. No decision degrees of freedom, no agility, period. To switch doors is to learn your way forward. This is agility, here leveraged to avoid the “Zonks.”

To close, consider that in the Narrative Fallacy, the feeling of understanding that compels us to adopt a narrative is not itself diagnostic of correspondence with reality. I would propose this applies equally to planning. The same heuristics and biases are at play. The same tricks that allow a good narrative to hijack our thinking will also lead us into believing a plan is more optimal and sensical than it really is. At the end of the day, it’s a category error.

---