Unveiling Java 8 Lambda: Inside the Stream Pipeline Mechanics

Preface

Java 8 lambda is familiar, this article explores its design and implementation from the source code perspective.

Basic Example and Analysis

First, consider the following example code:

static class A {
    @Getter
    private String a;
    @Getter
    private Integer b;
    public A(String a, Integer b) {
        this.a = a;
        this.b = b;
    }
}
public static void main(String[] args) {
    List<Integer> ret = Lists.newArrayList(
        new A("a", 1), new A("b", 2), new A("c", 3))
        .stream()
        .map(A::getB)
        .filter(b -> b >= 2)
        .collect(Collectors.toList());
    System.out.println(ret);
}

The code performs four steps: ArrayList.stream , .map , .filter , .collect .

ArrayList.stream calls Collector.stream, which creates a ReferencePipeline.Head object via StreamSupport.stream. The Head represents the pipeline’s entry point.

Calling .map invokes ReferencePipeline.map, which returns a StatelessOp that wraps the mapper function.

The .filter step also creates a StatelessOp with the predicate.

Finally, .collect triggers execution. Unlike the previous lazy operators, collect is a terminal action that forces the pipeline to process elements, illustrating Java’s lazy execution model.

Stream Internals

The pipeline execution proceeds through methods such as evaluate, wrapAndCopyInto, and copyInto. The copyInto method invokes the spliterator’s forEachRemaining, which for an ArrayList iterates over the underlying array and passes each element to the chained Sink objects representing the operators.

Each Sink implements accept to apply the mapper, then the filter, and finally the collector’s accumulator.

Dual‑Stream concat Example

A more complex scenario concatenates two IntStream instances and then applies additional operators:

static class Mapper1 implements IntUnaryOperator {
    public int applyAsInt(int operand) { return operand * operand; }
}
static class Filter1 implements IntPredicate {
    public boolean test(int value) { return value >= 2; }
}
static class Mapper2 implements IntUnaryOperator {
    public int applyAsInt(int operand) { return operand + operand; }
}
static class Filter2 implements IntPredicate {
    public boolean test(int value) { return value >= 10; }
}
static class Mapper3 implements IntUnaryOperator {
    public int applyAsInt(int operand) { return operand * operand; }
}
static class Filter3 implements IntPredicate {
    public boolean test(int value) { return value >= 10; }
}
public static void main(String[] args) {
    IntStream s1 = Arrays.stream(new int[]{1,2,3})
        .map(new Mapper1())
        .filter(new Filter1());
    IntStream s2 = Arrays.stream(new int[]{4,5,6})
        .map(new Mapper2())
        .filter(new Filter2());
    IntStream s3 = IntStream.concat(s1, s2)
        .map(new Mapper3())
        .filter(new Filter3());
    int sum = s3.sum();
}

The concatenated stream creates a Streams.ConcatSpliterator that sequentially processes the two source spliterators. The execution chain becomes:

Head (concatenated s1 + s2) → Mapper3 → Filter3 → ReduceOp(sum)

For each source stream, its own Head → MapperX → FilterX is linked before the common tail.

This demonstrates how the pipeline recursively wraps and copies sinks, ultimately invoking the terminal sum operation.

Class Hierarchy

The Stream API is built on several abstract classes: BaseStream defines core methods such as iterator(), spliterator(), isParallel(), etc. AbstractPipeline holds references to the source stage, previous stage, next stage, depth, and source spliterator.

Concrete subclasses include IntPipeline, LongPipeline, DoublePipeline, and ReferencePipeline, each providing type‑specific operators.

Each pipeline subclass contains three inner classes: Head, StatelessOp, and StatefulOp.

The design separates the user‑facing XXStream interfaces from the internal pipeline implementation, keeping the API clean while allowing powerful lazy processing.