How to Extract Dominant Colors for Product Search Using MeanShift & FloodFill

1. Introduction

In many search scenarios, users want to filter results by color, such as shoppers selecting clothing of a preferred hue or designers retrieving models of a specific color, e.g., all red sofas.

2. Problem Analysis

The issue is a typical Dominant Color problem. Google provides an API to extract dominant colors from an image, which are the main colors of the picture.

Traditional segmentation (e.g., K‑Means, Fuzzy C‑Means) can extract dominant colors, but human visual perception is affected by background, reflections, and regions of interest, making it difficult to obtain the perceived dominant color.

For example, a sofa image may be segmented into a large gray‑white background and reflections, yet a human instantly perceives it as a blue sofa . Simple color segmentation is insufficient.

3. Algorithm Design and Implementation

1. Algorithm Flow

Resize: uniformly scale images to speed up processing.

Segment: use MeanShift and FloodFill for image segmentation.

Merge: combine adjacent regions of the same color.

Gaussian Score: assign Gaussian‑weighted scores to each region and sort.

Select: choose the desired Dominant Colors .

Tag: map extracted colors to standard palette colors.

2. MeanShift

Basic Principle

MeanShift is a clustering algorithm used for image smoothing, segmentation, and tracking. It iteratively moves a window to the density maximum (centroid) until convergence.

Step1: In d‑dimensional space, pick a random point, draw a sphere of radius r, compute vectors from the center to points inside, sum them to obtain the MeanShift vector.

Step2: Use the vector endpoint as the new center and repeat.

Step3: Converge to the point of highest probability density.

MeanShift can be extended with kernel functions.

Engineering Implementation

Implemented with OpenCV’s pyrMeanShiftFiltering, which performs color‑space smoothing rather than strict segmentation, merging similar colors and eroding small regions. Parameters sp (spatial radius) and sr (color radius) control the smoothing strength.

3. FloodFill

Basic Principle

FloodFill fills a connected region with a specific color, using thresholds to control connectivity. It can generate masks to accelerate processing or isolate regions.

Engineering Implementation

Implemented with OpenCV’s floodFill. Key parameters: mask: prevents filling non‑zero mask pixels. loDiff: maximum lower brightness/color difference. upDiff: maximum upper brightness/color difference.

Example images illustrate how larger tolerances cause colors to “flow” over similar regions.

4. Merge

After FloodFill, connected color regions may still be split; merging combines regions within a defined color difference threshold.

5. Gaussian Score

Assuming human‑perceived colors lie near the image center, each region receives a Gaussian weight based on distance to the center; the average weight yields a score, allowing ranking of dominant colors.

6. Select

After scoring, the top‑ranked colors are selected, but challenges include varying numbers of dominant colors per image and avoiding background dominance.

Different images may have one or multiple dominant colors; simple Top‑N selection may miss some.

Background colors can dominate due to central position, obscuring other important colors.

Strategy: dynamically adjust color difference thresholds to ensure at least one dominant color survives after background removal and small region pruning.

7. Tag

Assigning color tags requires computing color differences in the perceptually uniform CIELab space (not RGB). The Pantone color card is used as the standard palette, ensuring equal spacing in CIELab and covering the full range of human‑recognizable colors.

4. Results

Examples show extracted dominant colors and their mapped standard colors for single‑color, two‑color, and multi‑color cases.

Search results using color tags are satisfactory.

5. Summary and Outlook

The workflow includes image preprocessing, MeanShift & FloodFill segmentation, region merging, Gaussian weighting, selection, tagging, and indexing. While traditional image processing yields good recall and precision, future work may explore semantic‑level color recognition using deep learning.

6. References

[0] Comaniciu, Dorin, and Peter Meer. "Mean shift: A robust approach toward feature space analysis." IEEE Transactions on Pattern Analysis and Machine Intelligence 24.5 (2002): 603‑619.

[1] http://www.cnblogs.com/mikewolf2002/archive/2013/08/20/3269883.html

[2] http://blog.csdn.net/dcrmg/article/details/52705087