User-Based Collaborative Filtering with Python: A Step-by-Step Guide

Collaborative filtering is a widely used technique in recommendation systems, typically divided into memory‑based (user‑based or item‑based) and model‑based approaches.

The tutorial focuses on a memory‑based user‑based method, where similarity between users is measured using cosine similarity on a rating matrix that is first mean‑centered and cleaned.

Data is loaded with pandas from movies.csv, ratings.csv and tags.csv. User mean ratings are computed and merged to obtain an adjusted rating column ( adg_rating).

import pandas as pd
import numpy as np
from sklearn.metrics.pairwise import cosine_similarity
from sklearn.metrics import pairwise_distances

movies = pd.read_csv("data/movies.csv")
Ratings = pd.read_csv("data/ratings.csv")
Tags = pd.read_csv("data/tags.csv")

Missing values in the user‑item matrix are filled either by the average rating of each movie or by each user’s average rating:

# Replacing NaN by Movie Average
final_movie = final.fillna(final.mean(axis=0))
# Replacing NaN by User Average
final_user = final.apply(lambda row: row.fillna(row.mean()), axis=1)

Cosine similarity is then computed for both filled matrices, with the diagonal set to zero to ignore self‑similarity:

# user similarity on replacing NAN by item(movie) avg
cosine = cosine_similarity(final_movie)
np.fill_diagonal(cosine, 0)
similarity_with_movie = pd.DataFrame(cosine, index=final_movie.index)
similarity_with_movie.columns = final_user.index

# user similarity on replacing NAN by user avg
b = cosine_similarity(final_user)
np.fill_diagonal(b, 0)
similarity_with_user = pd.DataFrame(b, index=final_user.index)
similarity_with_user.columns = final_user.index

For each user, the top 30 most similar users (or movies) are selected as neighbours:

def find_n_neighbours(df, n):
    order = np.argsort(df.values, axis=1)[:, :n]
    df = df.apply(lambda x: pd.Series(x.sort_values(ascending=False)
           .iloc[:n].index,
          index=[f'top{i}' for i in range(1, n+1)]), axis=1)
    return df

sim_user_30_u = find_n_neighbours(similarity_with_user, 30)
sim_user_30_m = find_n_neighbours(similarity_with_movie, 30)

Rating prediction for a specific user‑item pair uses a weighted sum of neighbour ratings multiplied by similarity weights, added to the user’s average rating:

def User_item_score(user, item):
    a = sim_user_30_m[sim_user_30_m.index==user].values
    b = a.squeeze().tolist()
    c = final_movie.loc[:, item]
    d = c[c.index.isin(b)]
    f = d[d.notnull()]
    avg_user = Mean.loc[Mean['userId'] == user, 'rating'].values[0]
    index = f.index.values.squeeze().tolist()
    corr = similarity_with_movie.loc[user, index]
    fin = pd.concat([f, corr], axis=1)
    fin.columns = ['adg_score', 'correlation']
    fin['score'] = fin.apply(lambda x: x['adg_score'] * x['correlation'], axis=1)
    final_score = avg_user + (fin['score'].sum() / fin['correlation'].sum())
    return final_score

score = User_item_score(320, 7371)
print("score (u,i) is", score)

Finally, the top‑5 recommended movies for a given user are obtained by scoring all unseen items and sorting by the predicted score:

def User_item_score1(user):
    Movie_seen_by_user = check.columns[check.loc[user].notna()].tolist()
    b = sim_user_30_m.loc[user].values.squeeze().tolist()
    d = Movie_user[Movie_user.index.isin(b)]
    l = ','.join(d.values)
    Movies_under_consideration = list(set(l.split(',')) - set(map(str, Movie_seen_by_user)))
    Movies_under_consideration = list(map(int, Movies_under_consideration))
    score = []
    for item in Movies_under_consideration:
        c = final_movie.loc[:, item]
        d = c[c.index.isin(b)]
        f = d[d.notnull()]
        avg_user = Mean.loc[Mean['userId'] == user, 'rating'].values[0]
        index = f.index.values.squeeze().tolist()
        corr = similarity_with_movie.loc[user, index]
        fin = pd.concat([f, corr], axis=1)
        fin.columns = ['adg_score', 'correlation']
        fin['score'] = fin.apply(lambda x: x['adg_score'] * x['correlation'], axis=1)
        final_score = avg_user + (fin['score'].sum() / fin['correlation'].sum())
        score.append(final_score)
    data = pd.DataFrame({'movieId': Movies_under_consideration, 'score': score})
    top_5 = data.sort_values(by='score', ascending=False).head(5)
    Movie_Name = top_5.merge(movies, how='inner', on='movieId')
    return Movie_Name.title.values.tolist()

user = int(input("Enter the user id to whom you want to recommend : "))
predicted_movies = User_item_score1(user)
print("The Recommendations for User Id :", user)
for i in predicted_movies:
    print(i)

The article concludes with a concise workflow: data collection, loading, preprocessing, similarity computation, score prediction, and item recommendation, emphasizing its applicability to large‑scale data scenarios.