Have you ever wondered how online retailers seem to know what you want, even when you only type in a few words into the search box? Shops like Tesco aren’t using clever mind reading tricks to know what you want to buy – in fact they are using what are known as Search Algorithms, which are based on maths, to help you find what you want.
Algorithms are like a set of mathematical instructions which tell a computer what to do. Having the most efficient Search Algorithm can mean that customers get a better shopping experience and are more likely to buy the product which they are looking for. After all, no one wants to be recommended fish fingers when they have searched for a new sofa!
What do Search Algorithms do?
The task of a Search Algorithm is to identify the most relevant items to show a customer, given the text they just typed into the search box. Although it sounds pretty simple, the problem is actually quite complicated because there are no constraints on what can be typed into the search box.
For example, a customer could make spelling mistakes or use words which are confusing. If a customer types the word “light” it could have different meanings depending on the context, such as light bulb, Flora light spread or light blue colour.
Therefore, there are two main steps to a Search Algorithm:
(1) Interpret what the user is searching for.
(2) Identify the most relevant products to suggest for the customer.
In some cases the first step is hard. For example, if a user searched for “Kiwi”, are they looking for a Kiwi fruit or Kiwi brand shoe polish?
At other times, the second step is hard. For example, if a user searched for “Fresh Milk”, it’s clear what they are looking for. But as Tesco sells over 200 milk products, which item or items are most relevant?
Sometimes both steps are hard. For example, if a user searched for “wine that goes best with beef”, then we don’t know what beef dish they are thinking of or what they define as “best”, and of course Tesco sells over 500 different wines.
So these are both quite difficult steps even for a human to do, and hence it’s even harder to get a computer to do this automatically. The good news is that Tesco has a lot of data being generated by millions of customers searching for products on their websites, allowing them to improve and update their Search Algorithms all the time.
Fuzzy matching
In order to deal with spelling mistakes and abbreviations, the Search Algorithm carries out what is known as a “fuzzy match”, looking for words which almost match what was typed. In order to do this the Search Algorithm could, for example work out what is known as the Levenshtein distance between two words to decide whether there is a close enough match.
The Levenshtein distance between two words is the number of edits of one letter which you would need to make in order to turn 
one word into another. For example the Levenshtein distance between the mis-typed word “bannana” and “banana” is only 1, as we only need to remove one “n” from “bannana” in order to create the word “banana”. This means the Algorithm will be able to say with a high degree of confidence that someone searching for “bannana” actually means “banana”.
The fuzzy matching can also be extended further to look for synonyms (similar words), where the Search Algorithm would for example match “shower gel” with “body wash”.
Real world testing
Once Data Scientists at Tesco have created a Search Algorithm, it can only be properly tested using real world data – does the Search Algorithm actually help customers find what they are looking for? This can be measured in a number of ways – one of which is by counting the number of times a shopper adds items from the search results to their basket.
In fact the Data Scientists at Tesco found that their initial Search Algorithm, which was launched across the Tesco iPhone App with great success, led to a significant increase in revenue per week. It was only by using real world data that the success of this Search Algorithm could have been measured and its benefit shown to the company.
Mathematical Careers in Retail
Developments in online shopping and the use of computers has meant that retailers need to use sophisticated mathematical techniques in order to maintain a competitive advantage. This means that there is an increasing demand among retailers for people who have advanced mathematical skills. Typically roles such as data scientist or business analyst will require a mathematically rich degree and for the most technical research roles, a PhD in mathematics or statistics alongside substantial computer programming experience will be important.
As well as needing researchers such as data scientists, retailers also need mathematically skilled people throughout their businesses, such as accountants, project managers and logistics specialists. If you like the idea of applying mathematics to real world problems then a career in retail could be for you.
With thanks to Dr Ben Dias, Lead Data Scientist at Tesco for help in preparing this article.
Image Credit:
Kiwi Shoe Polish © Petaholmes / Wikimedia Commons / CC-BY-SA-3.0 /
