The answer is: recommender systems.

Most people are unwilling or unable to explicitly state their preferences. That process requires too much time, engagement, and self-awareness. Additionally, the platforms often don’t have an intricate enough interface or form for collecting such detailed information.

On the other hand, we would be constantly overwhelmed by the vast catalogs of available products and services if we based our search on simple queries and rankings. If you’re looking for that one book, then all you need is that title. However, if you wanted to find other similar books that you’d probably like, you would have to know their titles too (or a very good librarian).

To solve these issues, recommender systems — which are Artificial Intelligence algorithms — analyze our online behavior through our interaction with various products and generate personalized recommendations to help us discover new products, content, or people we might be interested in.

Thanks to recommender systems, e-commerce sites can suggest products we might love, video streaming services can offer us films and TV series that match our tastes, and social media can help us discover new friends and connections.

On platforms like Amazon, they serve up-selling and cross-selling purposes. They may suggest books that other people with similar tastes have bought, or promote compatible accessories to the laptop computer that has just been added to the cart. According to a statement by Amazon’s CEO in 2006, about 35 % of their sales originate from cross-sales. *

While in physical stores we can efficiently browse the shelves and look out for special offers, new products, or a surprise gift for our dearest, in online stores with tens of thousands of products we would have to scroll through flat images for hours. Therefore, inspiration served by algorithms is often gladly embraced.

Recommender systems greatly improve the customer’s experience by reducing the information overload problem and bringing better-targeted items to their reach.

1. They increase sales:

Promoting complementary and better-targeted products stimulates customer engagement, increases the average spend on the purchasing journey, and the positive experience improves the likelihood of building customer loyalty (and consequently of other purchases).

2. Data analysis also occurs after the purchase:

Historical data can be analyzed to predict which products or services may have higher demand in the future. This allows for the optimization of inventory, reduces the risk of unsold products, and improves inventory management.

Netflix, for example, disclosed in a blog post that “75 % of what people watch is from some sort of recommendation”, while in another report, the authors estimate the business value of recommendation and personalization systems for Netflix as more than 1 billion US dollars per year, thanks to a measurable increase in user engagement and a contribution in reducing customer churn over the years. *

Likewise, YouTube communicated that 60 % of the clicks on the home screen are on the recommendations. A report on the platform’s video recommendation system shows an increase of over 200 % in terms of the click-through rate when an algorithm based on co-visitation was used instead of an approach that only recommends the most-viewed items. *

To Make or to Buy, that is the question.

Nowadays, commercial off-the-shelf recommender systems are cloud-based services (SaaS). They are optimized towards vast, heterogeneous product catalogs typical in electronic retail and stand out with appealing features like seamless integration with e-commerce and ERP software, KPI dashboards, rule engines for sales campaigns, A/B testing support, and many more. However, basing a make-or-buy decision on the arguments of a glossy brochure is hardly ever faultless.

Here are some other decision criteria that regularly urge CTOs to abstain from cloud services and instead build, deploy, and operate their own recommender system:

Pricing Model

Cloud-based recommender services are priced based on their success. Due to extensive user tracking, the system knows which purchases originate from one of its recommendations. Imagine that the pricing model is such that 15% of the additional net sales is claimed by the recommender service vendor. This seems like a good deal as long as the shop is not too successful. For larger shops, however, the cumulative monthly fees quickly catch up with the investment for the development and operations of a recommender system.

Data Privacy

For most e-commerce applications it is usually not a big deal to send user interaction data to the cloud, where the recommender algorithms are carried out, and in return receive a set of items to display as recommendations on the current page. Some businesses are more reluctant though, think of banks, insurance, pharmacies, adult stores, etc. It clearly also depends on the kind of data. User ID along with page views seems harmless — location data or credit card transaction data does not.

Omnichannel Recommendations

Cloud-based recommender services are designed for an effortless integration with standard e-commerce software, but they can hardly ever spur other customer touchpoints. In line with a modern omnichannel marketing strategy, high-quality recommendations must supplement every customer interaction, from newsletters through call center interactions to personalized print products.

Homogeneous Product Catalogs

Cloud services are all about standardization. They target heterogeneous product catalogs of large retailers and, consequently, employ recommender algorithms that only learn from customer interaction data (e.g., ratings, purchases, page views, session data). No product information is being used, because there are too few common features among the products. Such collaborative filtering style algorithms are known to suffer heavily from the “cold start” problem: they cannot issue recommendations for new users or new products, due to the lack of interaction data.

Cold start issues can be mitigated with custom-built recommender algorithms that are trained to leverage product-related attributes instead of interactions. For homogenous catalogs found on car reselling platforms, real estate websites, cooking recipe platforms, or online dating portals for example, specifically tailored algorithms often do a much better job than what you can get from cloud service providers.

Unstructured Product Data

Issuing recommendations for catalogs with unstructured product information (text, images, videos) is usually well beyond the scope of cloud-based recommender services. However, with embedding models based on deep learning, such data can be leveraged to mimic homogenous features even in heterogenous product catalogs. Much like Google Image Search displaying semantically similar images to the one you are currently looking at. For instance, we built a recipe-to-recipe recommender for Betty Bossi that suggests similar recipes based on the instruction text and ingredients list.

Cross-Catalog Recommendations

Traditional recommender algorithms compute product suggestions with respect to the same catalog they also use for learning user preferences. This, however, may not apply to all businesses. Take the example of a cooking platform that attracts customers through its recipes while recommending and selling kitchen supplies or special ingredients. Computationally, user preferences are thus learned from interactions with recipes to issue product recommendations from a different catalog. Such business requirements are simply too particular to be addressed by a standardized cloud service.

Beyond E-Commerce

Leaving the ancestral use-case of electronic retail, more specific needs pop up that will necessarily require some tailoring of the recommender systems. For example, suggesting treatments to patients diagnosed with a certain disease requires transparency. In this case, recommender algorithms must always be able to explain why a certain therapy is being suggested over another, to avoid liabilities.

Here’s another example: in an online bookstore, every item can be sold many times and with every transaction, one can learn more about the type of customer attracted by this product and improve recommendations. However, original artworks or NFTs are one of a kind and cannot be suggested again once sold. A user preference for one painting does not necessarily carry over to other paintings, even if they are from the same artist, so technically speaking, in these cases, we encounter a cold-start issue with every item in the catalog. Once again, this can only be solved through custom-built recommender algorithms, that can leverage other data besides interactions.

Does a cloud-based recommender system sound like the right fit for you?

Artificialy can support you with the integration of third-party services, but we can also evaluate a large set of different state-of-the-art recommender algorithms and parametrization on your data to ensure that the performance scores of your favorite recommender system vendor are competitive. We are proud to have a team of engineers that have guided and advised many of the leading Swiss e-commerce players throughout this process.

Or maybe you think your business could benefit from a tailored solution?

Well, you’re in luck! For nearly 10 years the people behind Artificialy have been building domain-specific recommender technologies for various businesses and industries whose needs could not be addressed by standardized cloud services.

* The data reported in this article can be found in more detail in: D. Jannach, M. Jugovac. Measuring the Business Value of Recommender Systems. ACM Transactions on Management Information Systems, Vol.10, №4, Article 1. December 2019.