The Signal and the Noise – Nate Silver

January-February 2016

What do mortgages, economics, poker, earthquakes, baseball, weather, politics, chess and terrorism have in common? Actually many aspects if we look deep enough, yet the commonality that Nate Silver focuses on in The Signal and the Noise (Why Most Predictions Fail but Some Don’t) is noise. All these elements have recent and pertinent instances where pivotal information (signal) was indistinguishable from the myriad forms of excess (noise).

Silver’s ability to dissect each of these scenarios into relevant and comprehensible examples shed light into how easily we can become entrapped by information.

Information overload can prove fatal. Couple this with predictions that fall out of context, overconfidence, inherent biases, dynamic systems and over-fitting and we’re in the real world that many forecasters and analysts of varying industries are left to decipher outcomes from. Today’s world, where data points are everywhere, create the paradox where petabytes of information are readily available to make quicker, more decisive and rewarding decisions yet we often have to spend great deals of time and intellect parsing out noise from truly meaningfully signals. Furthermore, when dealing with predictions without perfect models or issues involving human behavior, academic and theoretical analyses become less valid when tested in real world context. There are often so many unknown unknowns we can’t possibly account for until after experimenting and testing. As forecasters and analytic professionals, we should try to focus less on absolutes and try to adopt more Bayesian like reasoning when analyzing situations.

Truth be told, we’re all learning and pushing agendas. Neither Nate nor the pundits who vehemently reject his opinions are completely right or wrong in their reasoning. The key takeaway is not learning new methodologies to instantly make great predictions, but to better understand and identify the tiny and innumerable nuances that go into every action in the world.