Various Methods to Remove Duplicates from a Java List and Their Performance Comparison

The article presents multiple approaches to deduplicate a List<Integer> in Java, each accompanied by full source code and a brief explanation.

1. Two‑for‑loop deduplication (ordered)

/**使用两个for循环实现List去重(有序)
 * @param list
 */
public static List removeDuplicationBy2For(List<Integer> list) {
    for (int i = 0; i < list.size(); i++) {
        for (int j = i + 1; j < list.size(); j++) {
            if (list.get(i).equals(list.get(j))) {
                list.remove(j);
            }
        }
    }
    return list;
}

2. Using contains with a new list (ordered)

/**使用List集合contains方法循环遍历(有序)
 * @param list
 */
public static List removeDuplicationByContains(List<Integer> list) {
    List<Integer> newList = new ArrayList<>();
    for (int i = 0; i < list.size(); i++) {
        boolean isContains = newList.contains(list.get(i));
        if (!isContains) {
            newList.add(list.get(i));
        }
    }
    list.clear();
    list.addAll(newList);
    return list;
}

3. HashSet deduplication (unordered)

/**使用HashSet实现List去重(无序)
 * @param list
 */
public static List removeDuplicationByHashSet(List<Integer> list) {
    HashSet set = new HashSet(list);
    list.clear();
    list.addAll(set);
    return list;
}

4. TreeSet deduplication (ordered)

/**使用TreeSet实现List去重(有序)
 * @param list
 */
public static List removeDuplicationByTreeSet(List<Integer> list) {
    TreeSet set = new TreeSet(list);
    list.clear();
    list.addAll(set);
    return list;
}

5. Java 8 Stream deduplication (ordered)

/**使用java8新特性stream实现List去重(有序)
 * @param list
 */
public static List removeDuplicationByStream(List<Integer> list) {
    List newList = list.stream().distinct().collect(Collectors.toList());
    return newList;
}

The article also includes a benchmark program that generates 100 000 random integers (with values in different ranges) and measures the execution time of each method.

public static void main(String args[]) {
    List<Integer> list1 = new ArrayList<>();
    List<Integer> list2 = new ArrayList<>();
    List<Integer> list3 = new ArrayList<>();
    List<Integer> list4 = new ArrayList<>();
    List<Integer> list5 = new ArrayList<>();
    Random random = new Random();
    for (int i = 0; i < 100000; i++) {
        int value = random.nextInt(500);
        list1.add(value);
        list2.add(value);
        list3.add(value);
        list4.add(value);
        list5.add(value);
    }
    long startTime, endTime;
    startTime = System.currentTimeMillis();
    removeDuplicationByHashSet(list1);
    endTime = System.currentTimeMillis();
    System.out.println("使用HashSet实现List去重时间:" + (endTime - startTime) + "毫秒");
    // similar blocks for TreeSet, Stream, two‑for, and contains methods
}

Results show that HashSet and TreeSet are the fastest for unordered and ordered deduplication respectively, while the nested‑loop approach is significantly slower; performance varies with the range of random numbers.

Conclusion : Use HashSet for unordered deduplication and TreeSet for ordered deduplication to achieve the best efficiency.