Mastering Recommendation Systems: From Collaborative Filtering to Deep Learning

Recommendation systems are the unsung heroes behind the internet. They power the products you see on e-commerce homepages, the news you read, and the ads you encounter. A blog called Stats&Bots recently detailed five ways to build a recommendation system.

Many companies now use big data to deliver related recommendations, driving revenue growth. Data scientists must choose the best algorithm based on business constraints and requirements.

The Statsbot team compiled an overview of existing major recommendation system algorithms.

Collaborative Filtering

Collaborative filtering (CF) and its variants are among the most common recommendation algorithms. Even a data‑science beginner can use it to build a personal movie‑recommendation system, which is a solid resume item.

To recommend items, the logical approach is to find users with similar interests, analyze their behavior, and suggest the same items, or to look at previously purchased items and recommend similar ones.

CF has two basic approaches: user‑based collaborative filtering and item‑based collaborative filtering.

Regardless of the approach, a recommender engine follows two steps:

Identify how many users/items in the database are similar to the given user/item.

Considering the total weight of similar users/items, evaluate other users/items to predict the rating for the target product.

What does “most similar” mean in the algorithm?

We have each user’s preference vector (rows of matrix R) and each item’s rating vector (columns of R), as shown in the figure below.

First, we keep only the elements where both vectors have known values.

For example, comparing Bill and Jane: Bill hasn’t watched Titanic, Jane hasn’t watched Batman, so we can only compare their ratings for Star Wars to measure similarity.

The most popular similarity measure is cosine similarity or correlation between user/item vectors. The final step fills missing cells with a weighted arithmetic mean based on similarity.

Matrix Factorization

This elegant algorithm factorizes the rating matrix into user interest vectors (u) and item parameter vectors (v). The dot product u·v estimates the rating x for user i on item j.

For instance, after factorization Ted’s vector is (1.4, 0.8) and movie A’s vector is (1.4, 0.9); their dot product yields a predicted score of 2.68.

Clustering

Both previous methods are simple and suitable for small systems, treating recommendation as a supervised learning task. For large‑scale systems, clustering becomes attractive.

When a new service lacks historical user data, clustering is the best starting point.

Clustering yields weaker personalization because it recommends the same content to all users in a cluster, but it can serve as a first step to narrow the candidate set for collaborative filtering, improving performance.

Each cluster groups users with similar preferences, and recommendations are computed per cluster.

Deep Learning

In the past decade, neural networks have made huge strides and now dominate many recommendation tasks, often replacing traditional machine‑learning methods.

For example, YouTube uses two deep neural networks: one for candidate generation and another for ranking, as described in “Deep Neural Networks for YouTube Recommendations”.

The candidate generation network takes a user’s browsing history and selects a small, highly relevant subset of videos from the massive library.

The ranking network scores these candidates using video metadata and user behavior, presenting the highest‑scoring videos to the user.

Through these two steps, YouTube can efficiently select relevant videos from a huge catalog and even incorporate content from other sources.

Key Points Before Building a Recommender

If you have a large database and want to provide online recommendations, split the problem into two sub‑tasks:

Select the top‑N candidates.

Rank them.

Beyond standard quality metrics, consider recall and precision for recommendation tasks. See “Information Retrieval” for details and the book “Recommender Systems in Software Engineering” chapter 12 for additional metrics.

When using classification algorithms for recommendation, generate negative samples; treat purchased items as positive examples and others as negatives.

Evaluate algorithms both online (live scores) and offline (historical data); models trained only on past data may become stale.

Source: Quantum Bit