Operations 12 min read

Remote Execution Protocol: Solving New Challenges in Monorepo

The article introduces Remote Execution Protocol, a solution to address challenges in Monorepo environments by enabling distributed compilation and building on centralized infrastructure, improving build efficiency and reliability.

ByteDance Terminal Technology
ByteDance Terminal Technology
ByteDance Terminal Technology
Remote Execution Protocol: Solving New Challenges in Monorepo

本文介绍了Remote Execution协议,该协议由Bazel团队提出,旨在提升构建效率。协议的核心思想包括复用已有操作产生的结果(构建缓存)和投入更多计算资源加速执行(分布式编译)。在iOS Monorepo背景下,Monorepo方案导致源文件数量大幅增加,如果不采取措施,构建时间会增长数倍。通过抽象构建过程为原子操作(Action),并引入Content Addressable Storage(CAS)实现构建缓存,远程执行则将Action的输入发送到构建集群执行,返回结果。分布式编译的引入解决了缓存命中率较低情况下的构建效率问题,尤其在P90构建时间方面有显著提升。在Xcode体系下,通过hook编译器的方式实现,但存在依赖准确性问题。在Bazel体系下,依赖解析服务通过预处理指令分析头文件依赖,确保Action的依赖准确,从而提升缓存命中率。最终,Remote Execution方案在iOS Monorepo中显著提升了构建效率。

相关组件包括依赖解析服务(Sailfish)、构建缓存服务和远程任务执行服务(Tide)。依赖解析服务通过深度优先遍历预处理指令,准确解析头文件依赖。构建缓存服务优化了本地缓存命中率和网络下载效率。远程任务执行服务采用Rust编写,调度策略优化了任务分配,在集群资源充足时平均时间从422ms降至389ms。

build optimizationBazelBuild CacheRemote Executiondistributed compilationiOS MonorepoSailfish
ByteDance Terminal Technology
Written by

ByteDance Terminal Technology

Official account of ByteDance Terminal Technology, sharing technical insights and team updates.

0 followers
Reader feedback

How this landed with the community

login Sign in to like

Rate this article

Was this worth your time?

Sign in to rate
Discussion

0 Comments

Thoughtful readers leave field notes, pushback, and hard-won operational detail here.