How to Remove Duplicate IDs While Keeping Comments Using Pandas GroupBy

1. Introduction

A fan asked how to delete duplicate IDs in a dataset while keeping the preceding approval comments. Simple set‑based deduplication fails because the comments must be retained.

2. Implementation

Method 1

This solution, suggested by a senior contributor, leverages pandas groupby to concatenate comments for each unique combination of process status and process number.

The original data contains blank cells that need to be handled before grouping.

data['审批意见'] = data['审批意见'] + ','

data = data.groupby(['流程状态', '流程编号'])['审批意见'].sum().reset_index()

data['审批意见'] = data['审批意见'].str.strip(',').str.replace(',+', ',', regex=True)

Method 2

An alternative approach, contributed by another expert, works when there are no blank cells. It demonstrates a different way to achieve the same goal, though it requires additional handling for empty values.

Conclusion

The two methods illustrate how pandas.groupby() can be used to batch‑process data, remove duplicate identifiers, and preserve important textual comments, deepening the reader's understanding of this powerful function.