Machine learning and personalisation 101

Matt Lacey, performance director at Code Computerlove, takes the jargon out of machine learning and personalisation, and gives some actionable advice for how you can start improving your conversion rates immediately.

Matt Lacey Code ComputerlovePersonalisation is one of those topics that gets talked about with a lot of enthusiasm as the ‘next big thing’, year after year. And yet, it hasn’t seen mass adoption yet.

Econsultancy’s conversion optimisation research has identified this as a problem for the last three years, with marketers citing personalisation as the top method that they plan to use, but don’t get around to. And for those brands that are doing personalisation, it is invariably relatively basic, focusing on emails, homepage and maybe, if they are lucky, a landing page or two.

So why are we not embracing personalisation? It could be down to a range of factors, be it not having a conversion strategy in place or a lack of available design and development resources. Invariably, however, the biggest factor is the need for a solid foundation of data.

The growing trend of machine learning highlights this. The process feeds off data, but the problem in most cases is not whether the data is available, but that the data sets exist in silos. By connecting and enriching your data – linking up your returns and purchase data for instance, or ensuring your back-end data feeds into your analytics platform – you can really begin to personalise the customer experience.

So what are the machine learning and personalisation techniques that you can use to up the ante?

Data mining
Data mining, or opportunity mining, is the practice of using an algorithm to search for opportunities in the data. The algorithms are designed to ‘mine’ large data sets to find patterns of behaviour for you. These can either identify issues with your current campaigns or find opportunities for future growth. So, for instance, the data may reveal that PPC traffic converts well at weekends and poorly during the week, so you may adapt your campaign spend based on that information. Or you may find that a campaign does not work well in a particular region, and adapt the messaging based on this information.

Data mining is useful for identifying patterns, but it does not provide solutions, so it needs to be used in conjunction with experimentation and other conversion optimisation methods to really make the most of it. In which case, why bother? The main reason is time: analysis is time-consuming and therefore costly. Data mining automates this initial insight work and can help you to identify opportunities that might otherwise be hard to spot. And if you have a tool that incorporates data mining as part of the service, it’s even easier and quicker.

Predictive algorithms
Predictive algorithms are what most people think of when they talk about personalisation. They are the algorithms that power recommendation engines along the lines of: ‘what others also bought’. Lots of commerce sites use these features, but not always in a sophisticated way. The predictive algorithms used by retailers like Amazon go several steps further than ‘other items you might like’.

Amazon uses its predictive algorithms to predict how motivated a user is to buy, the user interest and the current sales volume of the individual products. It then provides product recommendations based on this behavioural and operational data, so that you and I will each receive a different set of search results for ‘green hat’. It is also why in your search results you may sometimes see a product with few or no reviews above items with hundreds of reviews.

This personalisation tool accounts for 35 per cent of Amazon’s sales, and is obviously highly complex, requiring a vast amount of data maturity, but it shows that there are great opportunities if you have a strong data strategy.
Switch to ‘best match’ algorithms.

A term you might hear a lot from your conversion optimisation team is ‘multi-armed bandits’. Although it sounds like highway robbery, this is a method used in A/B testing to automatically favour a variation as the experiment progresses.

Say in a standard multivariate test you send 50 per cent of traffic to one version and 50 per cent to the control. Dependent on traffic volumes, you would usually allow the test to run for a period of time, say two weeks, then allocate the winner and direct all traffic to that variation. But if you are using a best match algorithm, it will automatically redirect more traffic to the winning test as soon as a leader becomes obvious This means that the test is more likely to reach statistical significance and you will capture value sooner.

The New York Times is employing this technique with its online headlines right now. When a new article is uploaded to the site, it is input into the CMS with more than one headline. The headlines are split tested, and the winning variant is favoured as the experiment progresses, until all the traffic is focused on the winning headline. It’s a process that usually takes less than an hour.

This type of algorithm is great for short-term promotions and campaigns, as you can see in the New York Times example, but it can also be used in long-term tests. For instance, say you have two versions of a homepage and one is much more popular. Over time, 95 per cent of traffic is directed to the winning homepage, but 5 per cent is reserved for the control. Then a seasonal event, such as Black Friday, happens. Now the page that was initially performing much worse might suddenly start performing better, at which point the algorithm would detect this and start directing more traffic to it so that, in an extreme example, over a short period of time the traffic might then switch, with 95 per cent directed to the page that was previously performing poorly. It would then automatically switch back once the seasonal event was over.

Data is everything
The above examples can only be deployed with the right data strategy in place. Joining up your data sources is a key step to allow for this more advanced personalisation. We know this is easier said than done, but if you can enrich your data to allow patterns to be drawn, then you will be in a much better position than the majority of your competitors to create a relevant and engaging customer experience, and improve your site conversion.