gRPC-Rust Client API Evolution: Arena Support, Builder Patterns & Interceptor Design

Background: gRPC, Tonic, and Tower

gRPC is a high-performance RPC framework. Tonic is the widely used Rust implementation (12k+ GitHub stars) built to integrate with the Tower ecosystem, which aims to unify network client and server APIs for reusable middleware. The gRPC-Rust project was started to bring missing advanced gRPC features to Rust, such as built-in health checking, retries, zero-copy, and arena-based allocation optimizations.

Why Not Reuse Tonic's API?

The author analyzed Tonic's current client API and identified four fundamental problems:

Abstraction leak : Tonic exposes HTTP/2 primitives (e.g., GrpcService::ResponseBody typed as http_body::Body), preventing future transport switches like QUIC.

Unstable dependencies : Tonic relies on futures_stream /

tokio_stream

Stream

traits, which are not yet stable, blocking a stable gRPC-Rust release.

Performance bottleneck : Tonic owns protobuf messages internally, denying applications arena allocation — critical for high-QPS, high-concurrency systems.

Security risk : The ? operator propagates client call status including metadata (which may contain tokens/secrets) to server responses.

Why Not Tower?

Tower's timeout module races a timer against the wrapped Service, but this has two counter-intuitive behaviors for gRPC:

Timeout does not apply during poll_ready backpressure waits, allowing indefinite blocking.

Timeout only covers the call method return (near-instant for streaming RPCs), whereas gRPC timeout semantics cover the entire call from client initiation to final server status.

Additionally, Tower's timeout is invisible to the gRPC library, so the deadline cannot be propagated on the wire. Tower also makes memory allocation control awkward for unary RPCs — the very case where arena allocation matters most.

Two APIs: Generated vs. Channel

gRPC has two distinct APIs: a generated API (used via protobuf codegen, ergonomic) and a channel API (lower-level, streaming-only, for interceptors). Tonic's generated API was merely a generic specialization; gRPC-Rust aims to hide gRPC internals in the generated API, exposing only protobuf messages and essential primitives (like gRPC-Java). Cross-cutting concerns (metadata access, retry disabling, peer info) belong in interceptors.

Generated API Design

Arena Support

Arena allocation groups related memory operations in time and space. The target unary API lets the application control allocation:

async fn call(req: &MyRequestMessageView, resp: &mut MyResponseMessageView) -> Status;

Usage with a hypothetical arena API:

let req = MyRequestMessage::new_on_arena(arena).set_id(3);
let res = MyResponseMessage::new_on_arena(arena);
client.call(&req, &mut res).expect("RPC should succeed!");

Async Builder Pattern for Ergonomics

To avoid forcing manual response allocation for every call, the design adopts the async builder pattern ( IntoFuture). The call method returns an UnaryFutureBuilder that implements IntoFuture for simple owned-message usage, and provides with_response_message for pre-allocated arena responses. The request parameter uses AsView to accept both owned messages and views.

async fn call<Req>(req: Req) -> UnaryFutureBuilder<..>
where Req: AsView<Proxied=MyRequestMsg>;

impl IntoFuture for UnaryFutureBuilder {
    type Output = Result<MyResponseMessage, StatusError>;
}

impl UnaryFutureBuilder {
    async fn with_response_message<Res>(self, res: &mut Res) -> Status
    where Res: AsMut<MutProxied = MyResponseMessage>;
}

Simple usage:

let response = client.call(request).await.expect("RPC should succeed!");

Arena usage:

let req = MyRequestMessage::new_on_arena(arena).set_id(3);
let res = MyResponseMessage::new_on_arena(arena);
client.call(&req).with_response_message(&mut res).await.expect("RPC should succeed!");

The same pattern applies to streaming APIs. Bidirectional streaming returns a BidiCallBuilder yielding (GrpcStreamingRequest, GrpcStreamingResponse) via IntoFuture, with recv_into for arena-backed receives.

Channel API Design (Part 2)

Trait-Based for Zero-Cost Abstraction

The channel API is a trait ( Invoke) enabling interceptors via wrapping, with zero-cost abstraction (no dynamic dispatch between layers) for efficient retry, compression, etc.

Message Types: Dynamic References Over Generics

While generics avoid vtable lookups, heap allocation ( Box), and provide static type safety, they increase compile time and code bloat if propagated to serialization layers. The design instead uses dynamic trait references ( &mut dyn ResponseMessage) to avoid heap allocation while retaining flexibility. Static safety is already enforced by the generated API.

Splitting Send/Receive Streams

A single Stream object (like gRPC-Go) allows concurrent SendMsg / RecvMsg but cannot prevent multiple simultaneous sends or receives. Splitting into separate SendStream and RecvStream (like tokio::net::TcpStream::into_split) solves this and supports arena allocation.

RPITIT Problems and Associated Types Solution

Returning (impl SendStream, impl RecvStream) via RPITIT causes two issues:

Anonymous return types cannot be stored in wrapper structs for retry (cannot extract and re-store streams from a delegate).

Implicit &self lifetime capture prevents 'static bounds needed for cross-task usage; use<..> clauses (Rust 2024) add complexity.

The fix: declare associated types in the trait:

pub trait Invoke: Sync {
    type SendStream: SendStream + 'static;
    type RecvStream: RecvStream + 'static;
    async fn invoke(
        &self,
        headers: RequestHeaders,
        options: CallOptions,
    ) -> (Self::SendStream, Self::RecvStream);
}

Named types can be stored, updated during retry, and are 'static by declaration. The trait was renamed from Service::call (Tower's name) to Invoke to avoid semantic confusion.

Interceptor Complexity: Four Variants

Rust's ownership model requires special handling for metadata access via oneshot channels ( tokio::sync::oneshot). Interceptors must support both one-shot and reusable delegates, leading to four variants:

Basic (delegate once, reusable): logging, validation — implements Intercept<InvokeOnce> Once (delegate once, not reusable): metadata access — implements InterceptOnce<InvokeOnce> Retryable (delegate multiple times, reusable): retries — implements Intercept<Invoke + Clone + 'static> RetryableOnce (delegate multiple times, not reusable): (no current use case) — implements InterceptOnce<Invoke + Clone + 'static> Composition rules: retryable interceptors cannot wrap once interceptors, but basic can wrap once. The grpc::client::interceptor module provides InvokeExt / InvokeOnceExt with with_interceptor / with_once_interceptor for ergonomic chaining.

The API is released in preview; the team welcomes feedback for further adjustments.