Real‑Time Computing with Apache Flink at Beike Zhaofang: Hermes Platform Overview and Future Plans

Beike Zhaofang’s real‑time computing platform Hermes, built on Apache Flink, supports the company’s four core businesses (second‑hand housing, new housing, rentals, decoration) by processing billions of events daily.

The platform evolved from a DP data bus in 2018 to the Hermes platform, introducing a SQL V1 editor based on Spark Structured Streaming and later a Flink‑based SQL V2 editor with full Flink SQL support, custom functions, and source/sink integration.

Hermes now serves over 30 projects, runs more than 400 streaming jobs, handles up to 800 billion messages per day, and achieves sub‑40 ms latency for typical tasks.

Its architecture consists of four layers—engine (Flink/Table API), functional components (task, project, datasource management), resource isolation with dedicated and public queues, and monitoring/alerting that tracks latency, source write, and sink write metrics.

Rich built‑in functions (time, collection, JSON, string) and support for dimension table joins via async I/O, HBase/Redis storage, and LRU caching enable complex analytics such as real‑time user profiling, broker itinerary monitoring, and transaction dashboards.

Monitoring and alerting are powered by custom listeners and SDKs that feed metrics into Kafka for Hermes to trigger latency or heartbeat alarms.

Future work will focus on dynamic resource allocation, event‑driven rule engines, unified user data platforms, and Kappa‑style stream‑batch integration to improve state management and exactly‑once processing.