Designing an Extensible Content Delivery Pipeline for E‑commerce

In the "content is king" era, every To‑C app needs a content delivery service that can handle products, images, videos, etc. The rapid change of business requirements makes it necessary to design an extensible, stable, low‑latency interface.

Key terminology : Content – the items delivered to the client; Data source completion – auxiliary information for a content ID; Content completion – fetching surrounding assets for display.

The article focuses on content‑type services within Alibaba’s ecosystem and outlines three main concerns: the source of content IDs, constraints (e.g., expiration of operational configs), and operation (flexibility for evolving filters and client updates).

Typical problems include fragmented field completion, varying validation requirements across scenarios, and constantly changing operational/product demands.

Solution : Build a modular pipeline where each node addresses a single concern. The pipeline consists of the following components:

Datasource – fetches raw data (e.g., from config, cache, DB).

Transfer – type conversion.

Filter – data filtering (black/white list, field constraints).

Sorter – ordering (top, shuffle).

Completer – enriches data (adds SKU, AR info).

Validator – validates completeness.

Factory – assembles final objects.

Iterator – abstracts traversal logic.

Example of a generic pipeline implementation (Java 7 compatible):

public class PipeLine<D, C> {
    private List<PipeLineFunction<D, C>> functionList = new ArrayList<>();
    public PipeLine<D, C> add(PipeLineFunction<D, C> pipeLineFunction) {
        functionList.add(pipeLineFunction);
        return this;
    }
    public D execute(D data, C context) throws Exception {
        for (PipeLineFunction function : functionList) {
            data = (D) function.execute(data, context);
        }
        return data;
    }
}

Initializing a SKU recommendation pipeline:

public void initSkuResultHotRecommendPipeLine() {
    PipeLine<SkuResultVO, SkuQuery> skuResultHotRecommendPipeLine = new PipeLine<>();
    skuResultHotRecommendPipeLine
        .add((data, ctx) -> skuResultDataSource.fromConfig(ctx))
        .add((data, ctx) -> skuResultSorter.shuffle(data))
        .add((data, ctx) -> skuResultSorter.topTheme(data, ctx))
        .add((data, ctx) -> skuResultSorter.page(data, ctx))
        .add((data, ctx) -> skuResultCompleter.addSkuInfo(data))
        .add((data, ctx) -> skuResultCompleter.addAREffect(data, ctx))
        .add((data, ctx) -> skuResultValidator.check(data));
}

Service method that runs the pipeline:

public ResultVO<SkuResultVO> getSkuList(SkuQuery skuQuery) {
    try {
        SkuResultVO skuResultVO = skuResultHotRecommendPipeLine.execute(new SkuResultVO(), skuQuery);
        // further processing ...
    } catch (Exception e) {
        log.error("", e);
        return ResultVO.failOf(e.getMessage());
    }
    return ResultVO.failOf(CameraArCause.No_Valid_Ar_Type.toMessage(skuQuery.toString()));
}

Iterator example using functional interfaces (Java 8+):

@FunctionalInterface
public interface FilterFunction<T> {
    boolean execute(T t) throws Exception;
}

@FunctionalInterface
public interface IterateFunction<T> {
    T execute(T t, FilterFunction<T> filterFunction);
}

private IterateFunction<SkuResultVO> skuVOFilterIterator = (skuResultVO, filter) -> {
    List<SkuFeedUnitVO> skuFeedUnitVOList = skuResultVO.getSkuFeedUnitVOList()
        .stream().filter(skuFeedUnitVO -> {
            List<SkuVO> filterSkuVOList = skuFeedUnitVO.getSkuVOList()
                .stream().filter(skuVO -> {
                    try { return filter.execute(skuVO); }
                    catch (Exception e) { log.warn("", e); return false; }
                }).collect(toList());
            if (filterSkuVOList.isEmpty()) return false;
            skuFeedUnitVO.setSkuVOList(filterSkuVOList);
            return true;
        }).collect(toList());
    if (skuFeedUnitVOList.isEmpty()) {
        log.warn(CameraArCause.No_Valid_Sku_Feed_Unit_List.toMessage(skuResultVO.toString()));
    }
    skuResultVO.setSkuFeedUnitVOList(skuFeedUnitVOList);
    return skuResultVO;
};

The overall system is organized into six layers: API & View, Controller, Manager (pipeline nodes & third‑party wrappers), Middleware, Model (POJOs), and Common (utility components).

For flexible filtering, a JSON‑based tab‑filter schema is proposed. Example of a filter definition:

{
  "tabList": [
    {
      "tabName": "tab1",
      "tabId": "xxx",
      "filterList": [
        {"filterName": "xxx", "filterId": "xxx", "filterItemList": [{"filterItemName": "filter1", "filterItemId": "xxx"}, {"filterItemName": "filter2", "filterItemId": "xxx"}]}
      ]
    },
    {
      "tabName": "tab2",
      "tabId": "xxx",
      "filterList": [
        {"filterName": "xxx", "filterId": "xxx", "filterItemList": [{"filterItemName": "filter1", "filterItemId": "xxx"}, {"filterItemName": "filter2", "filterItemId": "xxx"}]}
      ]
    }
  ]
}

The article concludes that a modular pipeline greatly reduces system complexity, improves extensibility, and simplifies operations for large‑scale content delivery in e‑commerce platforms.