Noise #1: What is it?

Errors in judgment. There are two causes for it: bias, and “noise”. In their book titled Noise, the authors start off by explaining what “noise” means in this context. Say 4 teams, A, B, C, and D go to a shooting range:

 

Team A is on target. B is “biased” (off-target but in a very consistent way). C and D are inconsistent – they are what is called “noisy”.

 

In the real world though, for most judgments, one doesn’t know what the target is. Or who missed and by how much. If that sounded wrong, consider a few scenarios of what the authors mean: Which movie should be made? Which candidate should be hired? What is the right premium for the insurance policy? In these scenarios, what exactly is the right answer? And how do you even gauge if the decision was wrong: after all, each of the decisions could go wrong for reasons that have nothing to do with poor judgment e.g. a freak earthquake, or a personal event in the employee’s life that derails his professional life.

 

Which is why the authors say real life scenarios look like this. It’s the same shot pattern as above but without the bullseye circles to tell you what the target was.

 

In real world scenarios, one can’t tell if teams A and B are on target, or biased. But one can definitely say that C and D are inconsistent, i.e., noisy.

 

Noise, conclude the authors, is easy to identify in real life. The diagnosis of doctors vary. Project teams come up with very different estimates of cost and time. Bail decisions aren’t identical. Which patents get granted doesn’t follow a pattern. Not only is noise everywhere, but worse:

“In real-world decisions, the amount of noise is often scandalously high.”

Which is why the authors of the book (and this series of blogs) focuses on the problem of noise in decisions involving judgment.