Simplifying Uncertainty Responsibly

written by Eric J. Ma on 2019-12-19 | tags: bayesian data science uncertainty decision making

I read an article from Brandon Rohrer titled "Oversimplify". The article provoked some thoughts.

As someone who strives to model uncertainty in a day-to-day setting, it’s easy to misread Brandon’s article as saying, "discard your modelling of uncertainty". It took me a few reads and a bit of thinking to realize that my misunderstanding would have been wrong. The gist of Brandon’s article is to communicate the fact that most people don’t like uncertainty, so simplify the communication of uncertainty.

In a Bayesian setting, I think this corresponds mostly to "minimize regret" when making decisions.

Here’s a classic example: weather forecast says "30% probability of precipitation" (assume this is rain). How does this fit into Brandon’s conception of simplifying the communication of uncertainty?

Simply telling someone the probability of precipitation doesn’t help. It’s like stating an unhelpful fact - unhelpful if the audience doesn’t have a thought framework for acting on that uncertainty.

Any good card-carrying Bayesian who also knows how to minimize regret in-line with pretty universal human values would say, "there’s a 30% probability of precipitation. Since getting wet is more miserable than carrying an umbrella on a cloudy day, take that umbrella, and throw in your waterproof jacket and boots while you’re at it."

By contrast, someone other consultant might take the same probabilities, and instead recommend, "Oh, there’s basically greater odds of not raining than raining, so wear your cotton jacket since it’s cold, but don’t bother about an umbrella." This consultant, we would say, is basically deaf to human psychology and ethics. (I will not go deeper on the question of "what makes ethics" - too much for this blog post.)

The key question most people are seeking an answer to is not "what do the data say", but rather "how ought I act?" Though the latter question is normative, it can be informed by quantitative reasoning; we just have to make sure it is not only informed by quantitative reasoning.

So to summarize: most people don’t like uncertainty, so simplify the communication of uncertainty by providing an actionable recommendation in-line with universal of values (as much as possible) and culturally-specific behaviours of your audience. In that way, our handling of uncertainty can be responsible and ethical.