Predictive Analytics is an emerging field of technology and already being applied in business. Advanced algorithms are being combined with increasing computer power to find trends and relationships in data in order to automatically predict what’s likely to happen in the future. Even CFOs and their FP&A teams are becoming fascinated with the possibilities of applying Predictive Analytics to make the naturally uncertain tasks of forecasting and planning more certain, as well as easier.
I’m fascinated by it as well. Recently I’ve been talking to data scientists, marketing professors, and an economist about how Predictive Analytics works. And I’ve realized it can be as dangerous as it is powerful. Getting it right requires understanding analytic techniques and the data, and always questioning the relevance of data and results. Technology vendors may claim their solutions can predict the future with no human help, but computers still need some human supervision.
Here are a few things to keep in mind as you venture into the brave new world of Predictive Analytics – with a few common sayings to help you remember:
Past is not prologue
Your treasure trove of historical data certainly contains valuable insights worth learning. But does past experience predict future results? Remember that as you fit that regression line over a long history of data points, you might miss some very recent changes that could indicate a new trend, or not catch changes in your market influenced by factors you don’t see. A cautionary tale is the mortgage-backed securities crisis of 2008 – statistical models based on past experience didn’t warn of the increasing likelihood of defaults and the economic disaster that resulted.
Correlation is not causation
Regression models are great at finding correlations in your data, and may help you discover interesting and relevant relationships. But don’t substitute common sense for math and be misled by “false positives.” It’s worth the extra effort to verify you’re seeing causality and not coincidence. Someone actually found a very strong mathematical correlation between revenue generated by game arcades and computer science doctorates awarded in the US from 2000 to 2009 (from Spurious Correlations by Tyler Vigen)!
Garbage in, garbage out
The better the quality of data, the better and more reliable the analysis. Incomplete or inaccurate data (what we often call “dirty data”) can cause flawed results and expensively bad decisions. Metadata alignment is a significant challenge as we combine data from disparate sources. Predictive Analytics requires that the relevant attributes or drivers (the “columns”) are present and accurate, and that we have a sufficient number of records (the “rows”) as well. Despite our current fascination with Big Data, for Predictive Analytics it’s often much less about the quantity of data and more about the quality.
It’s the math, genius!
In Predictive Analytics it’s important to find and apply the right algorithms, based on the nature of your data and what you’re trying to learn and predict. This is where the math can get so complicated there aren’t enough regular math symbols and you have to start using the Greek alphabet! And I guess that’s why hiring people with doctorate degrees in data science is so much in fashion.
Are you considering applying Predictive Analytics in financial planning? Specifically in what areas would you trust a computer to do the predicting?