Applying Node.js async_hooks API in Tubi’s Backend Services

In this article we introduce the new Node.js async_hooks API and how Tubi’s engineering team has applied it in practice. We first explain the motivations and specific scenarios that led us to try it, then discuss the problems we encountered, show code examples, and finally present the production solution currently running at Tubi.

Motivation

Besides using Elixir and Scala, many of Tubi’s services are built with Node.js because it allows rapid development and is well‑suited for Backends‑for‑Frontends. However, Node has its own challenges, some inherent to JavaScript and others specific to the runtime. To address these, we built a micro‑framework on top of node‑restify called UAPI (Unified API), which standardises API construction. A typical API follows the pattern shown in the diagram below:

As illustrated, required information is passed via function arguments, forming a clear call stack.

Through careful design and code review this pattern served us well for a long time. Over the past 18 months we have become enthusiastic about Lyft’s sidecar proxy Envoy and have been standardising our infrastructure around Envoy and gRPC. To stay DRY, we implemented RPC libraries in Node, Scala, and Elixir.

Our RPC libraries, built on Envoy, add features such as retries, circuit breaking, fallback, caching, and tracing (using Jaeger). The biggest pain point with Envoy for Node is propagating tracing headers through the call stack. Other languages solve this with thread‑local storage or built‑in Context objects, but Node’s earlier Continuation‑Local Storage library does not work well with async/await, a feature we have needed since Node v4.

async_hooks API

Fortunately, Node v8 introduced the experimental async_hooks API, which provides extensive execution context information. In short, it offers the following hooks (callback functions):

With asyncId and triggerAsyncId we can construct a tree representing the program’s execution flow and associate every function call with the request that triggered it. A typical API call stack is illustrated below:

The key requirement is to uniquely separate the stack created by each route (req, res). A simple way, if you are familiar with your framework, is to use monkey‑patching. Below is an example of patching a simple HTTP server:

This approach is easy to implement and automatically cleans up when the request/response cycle finishes. The downside is that it tightly couples to the specific framework; for example, we could not find a way to patch the node‑grpc library to work with gRPC calls.

In future posts we will explore more robust solutions that work across Express, node‑grpc, and other servers without monkey‑patching, and we will discuss handling Keep‑Alive connections in async hook lifecycles as well as memory and garbage‑collection management.

Note: When a new HTTP request reaches a Node HTTPServer , a new async execution resource is created, so we do not need additional logic to handle multiple requests sharing the same async ID.

About the Author Cheng Yingyu, Senior Backend Engineer on Tubi China team. After graduating, he worked at Microsoft for three years on Bing Ad Insights API and Middle Tier platform. Joined the newly formed Tubi China team in 2016, contributed to Adbreak Finder, Clip Pipeline, and now focuses on micro‑service infrastructure and Ad Server development.

Tubi Campus Recruitment Announcement:

November 5 (Monday) 19:00

Tsinghua University – East Main Building 10‑500

Follow the public account "比图科技" to stay updated on our future explorations or join us in exploring the new world together!