Marketing and Optimization
Recently, the importance of customer churn, value, channel preference, and other behaviors have come to the forefront as criteria to consider when planning marketing segmentation and targeting. Providing support for the marketing team’s understanding of these metrics has become a key deliverable for IT departments. These metrics help decision makers define goals and strategies for organizations. As part of this, big data has increasingly become more important.
The interest and use of big data become complex when leveraging the data to support predictive analytic and optimization techniques. The integration of predictive analytics and big data is powerful for marketers and opens up new opportunities for optimal targeting and planning. One of the biggest levers a marketer can pull to improve performance is to improve the targeting of their marketing strategies. The more marketers can predict customer differences and align strategies and tactics around these predictions, the more efficient and effective their marketing spending will become.
BI vs. Predictive Analytics and Marketing Optimization
The proliferation of Business Intelligence (BI) tools demonstrates the demand that exists for historical customer insight. BI tools offer a wonderful way to visualize data and understand customers and prospects. However, they fall short in their ability to take all of the information at once and determine the most effective way to weave it together in multi-dimensional fashion.
Generally, BI tools analyze one to three pieces of information at a time. But more data than that becomes too complex for a visual representation, leaving data on the table. On the other hand, predictive analytics uses modeling algorithms to examine data and determine the best combination of data elements and weightings to predict a behavior of interest. Going further, marketing optimization can then combine those predictions with hard business constraints, such as budget and limited resources, to expose the best way to spend a marketing budget.
For example, perhaps an organization is working to grow its customer base as a marketing goal for its third quarter, and then increase profitability the following quarter. Optimization can provide the blueprint for making budgeting decisions to satisfy each objective. In the end, it is the optimal use of a combination of all available data that makes predictive analytics and optimization, in conjunction with BI, such a powerful set of tools.
Big Data + Predictive Analytics = Computational Burden
Without a doubt, the era of “Big Data” has arrived. The proliferation of data provides more sources of information about customers, prospects, channels, and competitors. While it may seem overwhelming to consider the increased complexity this brings, the reality is it will improve marketing outcomes.
But, there are challenges. The data is often unstructured and can require enormous computing power to summarize and evaluate. In addition, advanced algorithms for modeling and optimization require more sophisticated computing resources than BI tools. The IT administrator has to make a decision about infrastructure planning, and often the first thought is to throw the kitchen sink at the problem, thinking that an increase in data size requires a requisite increase in computing power for advanced analytics.
Thankfully, many aspects of the predictive analytics solution can rely on core tenets from the realm of statistics. Specifically, statistical sampling and variable reduction techniques can provide computational efficiencies while sacrificing very little in terms of the accuracy or power of the results.
Big Data + Efficiency = Fast, Powerful Results
A way to simplify the process is using samples to analyze the data. Statistical sampling approaches ensure a random or stratified subset of all available data which provides sufficient information to draw essentially the same insight, as if the full file were being analyzed. The brute force method is to crunch the entire file, which in the case of big data can be billions of records.
Some might argue this is the only way to realize 100 percent of the information value. Without disagreeing, the fact is that as little as .01 percent of your data could be used to generate predictive results to within 99 percent or greater accuracy (depending on the situation). Closing the gap from, say, 99 to 99.5 percent accuracy can require exponentially greater computing power with very little gain. The law of diminishing returns is in full force when combining big data with predictive analytics.
In addition, in many cases, what makes big data “Big” is the width of the information. The number of variables, or pieces of information, available to us can often be in the hundreds, if not many thousands. More variables increase our power to build compelling models but also increases the computational needs for predictive analytics.
As we discussed above, there are tried-and-true techniques that can be employed to reduce the number of pieces of information used to develop predictive models with very little effect on accuracy. Variable reduction techniques such as correlation analysis and principal components analysis can determine subsets of variables that contain a high proportion of the interesting variability necessary for predictive modeling. Focusing on a powerful subset provides significant efficiency gains when working with big data.
Putting It Together for IT
We’ve attempted to draw your attention to the power of combining advanced analytic techniques such as predictive analytics and optimization with Big Data. For the IT practitioner, the key thing to remember is that computational resources can be saved or re-allocated by considering a few key concepts such as sampling and data reduction techniques. Certainly, these techniques do not solve all the issues and can’t always be employed. But, for some of the most challenging and resource-intensive problems in advanced analytics, they can help make the supporting IT infrastructure much more streamlined and efficient. This work has huge potential for marketing teams, and ultimately, an organizations profitability.
