Master Fuzzy String Matching in Python with FuzzyWuzzy: A Practical Guide

1. Introduction

When working with data you often need to match fields that have different representations, such as abbreviated province names versus full names. The FuzzyWuzzy library provides simple fuzzy string matching based on Levenshtein distance to solve this problem efficiently.

2. FuzzyWuzzy Overview

FuzzyWuzzy is a Python package that uses the Levenshtein Distance algorithm to compute similarity between two strings. Install it via pip.

pip install -i https://pypi.tuna.tsinghua.edu.cn/simple FuzzyWuzzy

2.1 fuzz module

The module offers four main functions:

Ratio – simple similarity score.

Partial Ratio – compares substrings.

Token Sort Ratio – ignores word order.

Token Set Ratio – removes duplicate tokens before comparison.

Example:

fuzz.ratio("河南省", "河南省")  # 100
fuzz.ratio("河南", "河南省")   # 80

2.2 process module

The process module is useful when you have a list of candidate strings and want to retrieve the best matches.

choices = ["河南省", "郑州市", "湖北省", "武汉市"]
process.extract("郑州", choices, limit=2)
# [('郑州市', 90), ('河南省', 0)]

process.extractOne("郑州", choices)
# ('郑州市', 90)

3. Real‑World Applications

3.1 Company name matching

A function fuzzy_merge merges two DataFrames by fuzzy matching a key column. Parameters include the left and right DataFrames, key column names, a similarity threshold (default 90), and a limit on returned matches.

def fuzzy_merge(df_1, df_2, key1, key2, threshold=90, limit=2):
    """Merge two tables using fuzzy string matching.
    :param df_1: left table
    :param df_2: right table
    :param key1: key column of left table
    :param key2: key column of right table
    :param threshold: minimum similarity (0‑100)
    :param limit: number of top matches to consider
    :return: DataFrame with a new 'matches' column
    """
    s = df_2[key2].tolist()
    m = df_1[key1].apply(lambda x: process.extract(x, s, limit=limit))
    df_1['matches'] = m
    m2 = df_1['matches'].apply(
        lambda x: [i[0] for i in x if i[1] >= threshold][0]
        if len([i[0] for i in x if i[1] >= threshold]) > 0 else ''
    )
    df_1['matches'] = m2
    return df_1

3.2 Province name matching

The same function can be used to match abbreviated province names to their full names, producing a new column with the best match when the similarity exceeds the threshold.

4. Full Function Code

# fuzzy matching

def fuzzy_merge(df_1, df_2, key1, key2, threshold=90, limit=2):
    """
    :param df_1: the left table to join
    :param df_2: the right table to join
    :param key1: key column of the left table
    :param key2: key column of the right table
    :param threshold: how close the matches should be to return a match, based on Levenshtein distance
    :param limit: the amount of matches that will get returned, these are sorted high to low
    :return: dataframe with both keys and matches
    """
    s = df_2[key2].tolist()
    m = df_1[key1].apply(lambda x: process.extract(x, s, limit=limit))
    df_1['matches'] = m
    m2 = df_1['matches'].apply(
        lambda x: [i[0] for i in x if i[1] >= threshold][0]
        if len([i[0] for i in x if i[1] >= threshold]) > 0 else ''
    )
    df_1['matches'] = m2
    return df_1

from fuzzywuzzy import fuzz
from fuzzywuzzy import process

df = fuzzy_merge(data, company, '公司名称', '公司名称', threshold=90)