Mastering pandas extract & extractall: Quick Tips for Precise Data Extraction

Rescue pandas Plan (22) – Revisiting extract and extractall

Many users avoid pandas for data manipulation, so this article demonstrates why pandas is worth using by focusing on the str.extract and str.extractall methods.

Data Requirement

Extract specific characters from a designated column.

import pandas as pd

data = {
    'num': ['1-3
-7', '2', '1-5', '3-5-8
', '0-4-5-10', '4-81-15'],
    'string': ['打水泥打灰，需要水泥打灰，搅拌下', '我不打灰谁打灰', '不想打灰了', '打点灰吧，要不去搬袋水泥', '搬水泥，打灰，搅拌均匀', '搅拌，搅拌……']
}
df = pd.DataFrame(data)

Method Parameters

Both methods accept a regular expression pat, optional flags, and expand (only for extract). extract returns a DataFrame by default; setting expand=False yields a Series.

extract vs. extractall

extract

returns the first match per row, similar to re.match, and adds a match index when multiple groups are captured. extractall returns all matches, using the match order as an index.

Examples

Extract the first digit from the num column:

df['num'].str.extract(r'(\d)')

Setting expand=False returns a Series.

df['num'].str.extractall(r'^(\d)')

Using ^ restricts the match to the start of the string.

Extract the first and last numeric characters from num:

import re
df['num'].str.extract(r'^(\d)?.*?(\d)$', re.S)

The re.S flag allows . to match newlines; optional groups handle single‑character cases.

Match rows in string containing either “水泥” or “打灰”:

df['string'].str.extract(r'(水泥|打灰)')
df['string'].str.extractall(r'(水泥|打灰)')

extract

returns only the first matched keyword, while extractall returns all occurrences.

Match rows where both keywords appear:

df['string'].str.extract(r'(?=.*打灰)(?=.*水泥)(.*)')

Using look‑ahead assertions ensures both words are present; re.S allows matching across line breaks.

keywords = ['水泥', '打灰']
pattern = '|'.join(keywords)  # 水泥|打灰
extract = df['string'].str.extractall(fr'({pattern})').groupby(level=0).nunique()
df.loc[extract[extract[0] == len(keywords)].index]

Summary

This guide illustrates the usage of extract and extractall in pandas, comparable to the re module’s findall, search, and match functions, enabling flexible data extraction through regular expressions.

Persistence, rather than grand gestures, often brings us closer to success.