Advanced analytics in Retail, father of data mining Colin Shearer's interview

We hear a lot today about how organisations should leverage their data assets, and apply advanced analytics to improve their business. What exactly do you mean by “advanced analytics”?

Traditional analytics – reporting, business intelligence, and the like – give a “rear view mirror” approach; they show you where your business has been and where it is now. These techniques usually present information at an aggregated or summarised level, and are heavily “user driven”. It’s up to you to spot something that looks interesting in the data, and drill down into it.

Advanced analytics are predictive, providing foresight as well as insight. The computer takes much more of the initiative; intelligent algorithms automatically find patterns in historic data, and from these they create “predictive models”. These models can be applied to any current or future case to give a prediction, and they work at any level of detail you choose – for example, predicting the level of interest a particular customer will have in a specific product at a given time.

Is this a new area?

Not really! Some of the advanced analytical techniques used today date back to the 1950s, and predictive analytics have been used commercially in a wide range of businesses since the mid 1990s.

So why haven’t we seen more of it in Retail?

Most of the early application areas were in customer analytics. Businesses with the richest customer data and closest customer relationships – notably telco and personal finance companies – were very well equipped to do this. Retail, however, while generating vast amounts of transactional data, for a long time lacked the ability to easily link that to individual customers – until the advent of loyalty schemes and e-commerce.

Where can this be used? What retail functions can be improved with such an analytical approach?

As I’ve said above, now that retailers “know” their customers, all aspects of customer relationships can be improved by applying advanced analytics: from attracting more and better customers, through growing customer value and share of wallet by smart cross- and up-selling, to understanding loyalty and retaining customers at risk of ceasing to purchase or winning back those who have “gone”.

But there are a wide range of application areas beyond that. For example, optimising all parts of the supply chain based on accurate demand forecasting; taking assortment planning to a new level by having unique assortments for each store; selecting and designing offers by accurately predicting what impact they will have; store network planning; optimised pricing; and identifying and reducing shrinkage. The list is almost endless – these techniques can be applied wherever there is a pressing business pain and relevant data.

Surely before a retailer starts this, they have to go through a large project of getting their data complete, well organised and managed?

In an ideal world, all retailers embarking on advanced analytics would start with complete and perfect data. But few (if any!) are in that fortunate position, and attempting to complete a data collection, integration and management strategy before starting analysis hugely delays time to value. (Data collection and management is a cost; it’s only when applying analytics to data and acting on the results that you truly generate value from it.)

The most effective approach is a pragmatic one. Start to apply advanced analytics in areas where you already have decent data, and reap the rewards. Then as you work towards collecting and managing more comprehensive data, new data sources you integrate enable analytical use cases in further areas. These additional data sources can also help you enhance existing predictive models. For example, integrating new data from survey research into am existing cross-sell model could substantially increase the model’s accuracy and the additional sales revenue it generates.

So the best approach is an incremental and evolutionary one: start with the data you have, and as you improve and expand your data platform, so you extend and enhance your use of advanced analytics and glean greater and greater value.

So is it just a case of connecting this technology to your data, and suddenly your business is “predictive”?

For some areas where advanced analytical applications have been packaged and automated – for example, in enhancing multi-channel marketing with smart recommendations – it can almost be a case of “load and go”, with virtually out-of-the-box offerings giving very rapid time to value.

More generally, though, it’s a case of identifying the initial areas where you could benefit from this technology; selecting the relevant data; building and evaluating the appropriate predictive models; and using the results to take better decisions and actions through the operational systems that support your key business processes. In all these stages, you need to apply the combination of business knowledge and analytical skills; you need to either develop these capabilities in house, or select an appropriate partner who can help you succeed in these projects.

What guidelines would you give a retailer for starting in advanced analytics?

With all the publicity around “big data” and “AI” and suchlike, it’s easy to be confused by the hype into thinking it’s a case of simply hiring some smart analytical people (“data scientists”), equipping them with free open source tools, and letting them loose on your data. - That’s a recipe for disaster.

Your use of advanced analytics should always be business driven. You should absolutely have a vision of what it would mean to have your entire business enhanced by advanced analytics, but aim to start small and make initial quick wins that justify your investment and help fund greater and wider use across the organisation.

Select use cases based on business need, and factors such as availability of data that make for short projects and quick time to value. The analysis itself must be conducted in the context of the business, not by a team of geeks who take the data away to do amazing mathematical things to it; create teams that bring together business and analytical skills. And from the very first stages of the project – even before identifying the necessary data – focus strongly on how the results will be applied to drive better business outcomes; working back from that will always be the best guide to what analytical results you need to deliver, and what data will be needed to produce those successfully. For organisations new to this area, finding a good partner to help them progress through these early steps can make a huge difference to their initioal success and formulation of an effective advanced analytics strategy.