Understanding Vector Similarity in Machine Learning: A Plain‑Language Guide

Dot Product

The dot product returns a single scalar that can range from negative infinity to positive infinity. A negative value indicates opposite directions, zero indicates orthogonal vectors, and a positive value indicates the same direction. Geometrically it is expressed as |a|·|b|·cos(θ), where |a| and |b| are the vector norms and θ is the angle between them.

Cosine Similarity

Cosine similarity considers only the angle between vectors, ignoring their lengths. It is computed as the dot product divided by the product of the two vector norms, effectively normalising each vector to length 1. This makes cosine similarity ideal when direction alone matters.

import torch
import torch.nn.functional as F

# Method 1: Using F.cosine_similarity
cos_sim_1 = F.cosine_similarity(vector1, vector2)

# Method 2: Manual calculation
cos_sim_2 = torch.dot(vector1, vector2) / (torch.norm(vector1) * torch.norm(vector2))

Document Comparison Example

Four document vectors based on word frequencies:

doc1 = [10, 20, 30]

doc2 = [100, 200, 300] (10× doc1)

doc3 = [30, 20, 10] (different distribution)

doc4 = [200, 300, 100] (longer document, different distribution)

Cosine similarities:

doc1 vs doc2 = 1.0

doc1 vs doc3 ≈ 0.7142857142857142

doc1 vs doc4 ≈ 0.7857142857142857

Dot products:

doc1 vs doc2 = 14000

doc1 vs doc3 = 1000

doc1 vs doc4 = 11000

The dot product is sensitive to vector magnitude, while cosine similarity emphasizes relative distribution of terms.

import numpy as np

def dot_product(vec1, vec2):
    return np.dot(vec1, vec2)

def cosine_similarity(vec1, vec2):
    return np.dot(vec1, vec2) / (np.linalg.norm(vec1) * np.linalg.norm(vec2))

# Results
# Dot Products:
# doc1 vs doc2: 14000
# doc1 vs doc3: 1000
# doc1 vs doc4: 11000
# Cosine Similarities:
# doc1 vs doc2: 1.0
# doc1 vs doc3: 0.7142857142857142
# doc1 vs doc4: 0.7857142857142857

L1 and L2 Distance

Manhattan (L1) distance sums absolute differences across dimensions, while Euclidean (L2) distance computes the straight‑line distance. For the same point pair, L1 ≥ L2, and as dimensionality grows, L1 often becomes more discriminative.

Practical Considerations

Vectors must have the same dimensionality to be comparable. If dimensions differ, pad the shorter vector with zeros in missing positions before computing similarity.

Angle Computation Example

Compute cosine similarity between two 2‑D vectors and convert the result to an angle in radians and degrees.

import torch
import torch.nn.functional as F
import math

A = torch.tensor([1.5, 1.5])
B = torch.tensor([2.0, 1.0])

cos = F.cosine_similarity(A, B, dim=0)
theta = math.acos(cos)            # radians
theta_degrees = math.degrees(theta)  # degrees

print("Cosine Similarity:", cos)
print("Angle in radians:", theta)
print("Angle in degrees:", theta_degrees)