Two Quick Pandas Tricks to Group and Concatenate Values

1. Introduction

In a Python community a user asked how to transform a DataFrame so that for each id and type the associated book values are concatenated into a single comma‑separated string.

Original data:

Creating the DataFrame:

df = pd.DataFrame({
    'id': ['A','A','A','A','A','A','B','B','B','B','B'],
    'type': [1,1,1,1,2,2,1,1,1,2,2],
    'book': ['Math','Math','English','Physics','Math','English','Physics','English','Physics','English','English']
})

res = df.groupby(['id','type']).book.apply(list).reset_index()
res['book'] = res.apply(lambda x: ','.join([str(i) for i in x['book']]), axis=1)

Expected result (illustrated below):

2. Implementation

Method 1

A concise solution using groupby with unique and str.join:

df.groupby(['id','type']).book.unique().str.join(',').reset_index()

The output matches the expected result:

Method 2

An alternative using set and a lambda to join strings:

res = df.groupby(['id','type']).book.apply(set).reset_index()
res['book'] = res['book'].apply(lambda x: ','.join(x))
print(res)

This also produces the desired output:

Performance Note

Comparing execution times shows a dramatic speedup when using vectorized operations and type‑casting:

普通apply处理需要18秒左右，
使用Swift进行加速提升到7秒左右，
函数向量化，时间缩短至0.4秒，
int64转为int16，用时缩短至0.1秒
尽可能转换为.values，再操作，仅用时0.07秒！！！
从18秒到0.07秒……

3. Conclusion

The article demonstrates two effective Pandas approaches to group by multiple columns and concatenate values, helping readers solve similar data‑processing challenges quickly and efficiently.