Measuring Perceptions of Uncertainty

What is the difference between an event that is probable and one that is highly likely?

The two terms seem mostly interchangeable, but each individual’s interpretation is actually highly subjective. That means that when stakes are high, such as for the intelligence community or for high-ranking government officials, a slight misinterpretation in the meaning of these phrases could be a matter of life and death.

Sherman Kent and the CIA

Sherman Kent, often described as the “father of intelligence analysis”, was a CIA analyst that recognized the problem of using imprecise statements of uncertainty. Particularly, Kent was jolted by how policymakers interpreted the phrase “serious possibility” in a national estimate about the odds of a Soviet attack on Yugoslavia in 1951. After asking around, he found that some thought this meant a 20% chance of attack, while others ascribed an 80% chance to the phrase. Most people were somewhere in the middle.

Inspired by Kent’s work, a later study asked 23 NATO officers to assign actual numbers to terms like “probably”, “almost certain”, “little chance”, “unlikely”, and other words of estimated probability.

The results were fascinating:

Interpretations are all over the map. The words are not precise to begin with, but it’s also worth keeping in mind that people attribute meaning to these phrases based on their personalities, backgrounds, and prior experiences. Context also matters.

How Do We Interpret These Terms?

Although the consequences are less severe for us civilians, we are stuck in the same quandary today.

We’re almost certain a deal will go through, or there’s little chance a candidate will win the presidency. People interpret these terms differently, and these small differences still impact our lives.

Reddit user zonination set out to recreate the poll to see if perceptions of words today matched up with data from the study inspired by Sherman Kent. The results below are very similar, and can help us communicate more clearly, particularly when the stakes are high.

The same idea was also taken a step further, to look at potential misunderstandings that can occur when we use phrases instead of hard numbers.

For example, one person’s a few is another person’s several:

If you want to communicate with precision, it’s best to use numbers or specific odds.

Otherwise, be aware that a term like “improbable” can have a considerable range of interpretations – from 0% to 50% – depending on who you are talking to!