Boost LLM App Speed with LangChain’s RunnableParallel: A Step‑by‑Step Guide

RunnableParallel Overview

LCEL provides native parallel execution via RunnableParallel, a Runnable that accepts multiple Runnable objects, forwards identical input to each, and aggregates outputs into a dictionary.

Why Parallel Execution Matters

In a Retrieval‑Augmented Generation (RAG) pipeline, step 2—querying several data sources (vector store, SQL database, documentation site) simultaneously—dominates latency. Serial queries sum latencies; parallel queries reduce total time to roughly the slowest query, often cutting overall latency by ~50 % for I/O‑bound calls.

Example Scenario

Two use‑cases are demonstrated:

Invoking an LLM multiple times in parallel to generate different content types (e.g., a joke and a poem).

Running several retrievers in parallel to fetch documents from distinct vector stores and merging results.

Example 1: Basic Parallel Chain ( example_1_parallel_chains.py )

Core Concepts

Parallel Branch Design : two independent chains— joke_chain (produces a joke) and poem_chain (produces a four‑line poem).

Parallel Execution Methods : two equivalent constructions.

# Method 1: dictionary literal (recommended)
parallel_chain = RunnableParallel({
    "joke": joke_chain,
    "poem": poem_chain,
})

# Method 2: explicit construction
explicit_parallel = RunnableParallel(
    joke=joke_chain,
    poem=poem_chain,
)

Performance Advantage

Serial Execution : runs chains sequentially; total time = time_joke + time_poem.

Parallel Execution : runs both concurrently; total time ≈ max(time_joke, time_poem).

Empirical runs on typical network‑I/O tasks show ~50 % latency reduction.

Input‑Output Pattern

Input : identical dictionary {"topic": "programmer"} passed to both branches.

Output : combined dictionary, e.g.

{
    "joke": "Generated joke...",
    "poem": "Generated poem...",
}

Applicable Scenarios

Analyzing the same content from multiple perspectives.

Generating several output formats (summary, keywords, sentiment).

Calling multiple API services to enrich content.

Any workload where independent tasks can be processed in parallel.

Implementation Details

Creating a RunnableParallel instance triggers the LCEL runtime to schedule each branch on asyncio. The runtime gathers each branch’s result into the final dictionary. Example of constructing a parallel runnable with mixed lambdas and RunnablePassthrough:

from langchain_core.runnables import RunnableParallel, RunnablePassthrough

runnable_1 = RunnableParallel(
    passed_through=RunnablePassthrough(),
    extra=RunnablePassthrough.assign(mult=lambda x: x["num"] * 3),
    modified=lambda x: x["num"] + 1,
)

runnable_2 = {
    "passed_through": RunnablePassthrough(),
    "extra": RunnablePassthrough.assign(mult=lambda x: x["num"] * 3),
    "modified": lambda x: x["num"] + 1,
}

When invoked, each key in the dictionary is executed concurrently using asyncio, making this pattern especially effective for I/O‑heavy operations such as multiple LLM API calls or multi‑source retrieval.

Reference

How to: invoke runnables in parallel – https://python.langchain.com/docs/how_to/invoke_runnables_in_parallel