Everybody searches.
We’re in 2020, and search engines are plentiful and free-of-charge across the internet. From Google, to Amazon, to your ecommerce website, shoppers have multiple options when they seek information, compare products, and make purchase decisions. But as you can’t improve what you can’t measure, the key question we face is how to measure and assess the quality of search.
Well, the way it has always been done is by looking at relevance.
What Is Relevance?
Relevance is defined as “relation to the matter at hand.”
However, as intellectuals preparing to take you on a deep-dive into a complex topic in a short amount of time, we naturally have to make things more complicated and quickly shift your attention to the second dictionary entry: “the ability (as of an information retrieval system) to retrieve material that satisfies the needs of the user.” Simple, right? Just wait.
Relevance has always been, and still is, the most fundamental goal of search engines. The concept has been around forever, or at least as long as humans have tried to communicate and use information effectively. Modern computers have been around since the 1950s – with the Internet rising in the 1970s, the Web in the 1990s, and social media in the 2000s.
Part of the reason why relevance has endured as such a powerful concept over the past half decade or so lies in the seemingly apparent intuitiveness of the term. Nobody really has a hard time explaining it to anybody else in the world – search works when people can find what they had in mind. However, that intuitiveness is actually a simplistic misconception that some continue to hold onto even as the world changes – but we’ll get to that in a bit.
Relevance in Ecommerce
Relevance has become and remains the underlying criterion for measuring the effectiveness of ecommerce search, influencing the metrics that are used. When a customer searches your website for products, the idea is that results shown to them should be as close to what they are looking for as possible.
While measures like clicks and conversions can provide interesting proxies, the way to really measure search relevance is by using two primary metrics, precision and recall, which we can be seen as the systole and diastole of relevant search. The maximization of one does come at the expense of the other, but given they’re both essential to determining functionality, a balance is usually found between the two.
Recall refers to the ability of a search algorithm to properly classify and return all relevant products – nothing less.
Take a search query for “black boots” as an example. If there are twenty products that would qualify as black boots, but the system only returns sixteen, then the recall is 16 out of 20 or 80%. There might be several problems behind a low rate of recall, but you should probably start by checking your stemming, handling of synonyms and typo correction.
Precision on the other hand refers to the ability of a search algorithm to return only those products identified as relevant – nothing more.
For example, in the same customer search for “black boots”, if the results show ten different products – six of which are black boots, two of which are brown boots, one of which is black shoes, and one of which is brown socks – then the precision is 6 out of 10 or 60%. There may also be a variety of reasons for a disappointing rate of precision, but investing in your NLP capabilities to deepen your understanding of what people mean by what they say has proven to be very effective in addressing precision problems.
If you’re into ecommerce, here’s the rub: you may be assessing search the wrong way. For so many years, people have had one idea of relevance in mind when working on and evaluating search engines – giving customers what they need. However, changes in how we interact, the capabilities at our disposal, and what is expected, force a reconceptualization of what relevance is really all about in the world of product discovery today.
Relevance Gives Your Business What It Needs
In ecommerce it is necessary to broaden the narrow customer-centric notion of relevance. Organizations have interests other than just ensuring customer satisfaction and presenting the most relevant items to customers when they search.
Search experiences must also be informed by business priorities – with results being geared toward generating profit, clearing expiring inventory, and satisfying supplier relationships too.
This contrasts with what characterizes other search engines, such as Google or Bing where a narrower notion of relevance still works. While both web and ecommerce search engines aim to ensure user satisfaction by presenting the most relevant items to a user, web search engines do not need to account for the optimization of additional goals – such as minimizing inventory cost by selling items faster.
When it comes to your ecommerce site, your business strategy should determine your ranking strategy. Conversion rate, gross margin, customer loyalty, recent search terms, or a utility function comprising multiple values make the search experience more valuable to customers and the business alike.
This may seem like a lot to handle (it is), but you don’t have to do it on your own. Machine learning operates based on this broader notion of relevance and brings both customer and business signals to bear on automating ranking improvement efforts to strike a balance between customer and business optimization. With machine learning, you get the best for both worlds.
Relevance Gives Shoppers What They Need
When we talk about giving shoppers what they want, we also need to carefully consider what we mean when we say “shoppers” in the world of ecommerce today. There has been a rise in demand for more personalized ecommerce experiences, which means relevance needs to encompass more than personas, but individual people.
Especially with large commerce websites selling a wide variety of goods to very diverse buyer cohorts, there is no single best ordering of relevant results to search queries. In searching for a “Summer dress”, one user may prefer v-neck mini dresses, while another user may prefer floral maxi dresses. Relevance is in the eye of the individual beholder.
Long tail economics has arguably opened up important business opportunities for ecommerce players, shaping increasingly more complex catalogs. However, the focus on expanding SKUs in order to drive growth will be fruitless if companies are not also focused on capturing user preferences to provide a more personalized experience. More isn’t better if individuals can’t find what they need.
Give Shoppers (Just Enough) of What They Need
I know, it is a well-established axiom that more relevance means better search. But sometimes too much relevance won’t actually be that good for your search performance. In ecommerce, when your goal is to satisfy the needs of your customers, maximizing recall and showing your customers as many relevant products as possible won’t necessarily serve your purpose. In addition to being business-inclusive and individualized, relevance must be controlled.
Convincing ecommerce players to limit the choice they offer on their sites might be difficult, but the rationale behind this is actually quite straightforward and familiar to most of us. Simply put, providing too many options might lead to decision fatigue and shopping paralysis. There’s evidence for that.
Researchers in California ran an experiment in a local grocery store. They set up a stall in the store selling jam. Some days the booth would sell six flavors of jams, while on other days it offered 24 varieties. The results were significant. On days where they sold 24 varieties, they saw a mere 4 percent conversion rate. However, when they offered only six choices, that figure rose to a staggering 31 percent.
Less can actually be more, especially in ecommerce.
It’s 2020 and everyone searches. What are you doing to ensure that your customers find what they need in a way that works for them and for your business? Reimagining relevance is a good starting point – adopting the technology that operates upon this new understanding is what follows.