How JD’s JUST Engine Enables Real‑Time Urban Crowd Monitoring

After the 2014 New Year's Eve crowd disaster at Shanghai's Bund, public safety became a major societal concern. The key task is to monitor and quickly predict pedestrian flow in every city area so that authorities can take timely actions such as evacuation or traffic diversion when thresholds are exceeded.

To avoid similar hazards, a real‑time urban crowd monitoring and prediction system is essential. JD Urban, as the system integrator, identified several data sources that correlate with crowd density: mobile base‑station signaling, video surveillance, traffic flow, travel trajectories, weather conditions, point‑of‑interest (POI) capacity, and event information such as concerts.

Relying on a single data source is insufficient; effective monitoring requires the comprehensive integration of multiple streams.

JUST Engine Overview

JD Urban built the JUST (JD Urban Spatio‑Temporal) engine, which treats data with time, space, and location attributes as "spatio‑temporal data". The engine classifies data into six models, enabling unified storage and management.

Six Spatio‑Temporal Data Models

1. Static point data : fixed locations that do not change over time (e.g., train stations). 2. Static point with dynamic readings : locations remain constant while readings vary (e.g., video feeds, weather). 3. Dynamic point data : both location and readings change (e.g., event data, ride‑hailing orders). 4. Static network data : network structures that are immutable (e.g., road network). 5. Static network with dynamic readings : network edges generate periodic readings (e.g., traffic flow per road segment). 6. Dynamic network data : both network topology and readings evolve (e.g., trajectory data).

Indexing and Storage Challenges

Traditional relational databases (MySQL, Oracle, PostGIS) handle small‑scale spatio‑temporal queries but fail with massive data volumes. Distributed key‑value stores like HBase lack native multi‑dimensional indexing, leading to inefficient range queries and high storage overhead, especially for trajectory data where each GPS point occupies a separate row.

JUST creates multiple efficient spatio‑temporal indexes that encode three‑dimensional (longitude, latitude, time) information into a one‑dimensional key, enabling fast range queries. The indexing strategies are illustrated in Figure 6.

For trajectory storage, JUST compresses all GPS points of a segmented trajectory into a single record using GZip, reducing disk usage to one‑eighth of the original method (see Figure 7).

Performance Gains

Experiments show that JUST improves storage and indexing efficiency by over 7× and query speed by more than 100× compared with native HBase and other spatio‑temporal frameworks (Figure 9).

JUST also offers a SQL module, allowing users familiar with relational databases to execute all operations with zero learning curve, and a Notebook environment pre‑loaded with data preprocessing, analysis, and feature extraction tools for AI engineers.

Beyond the urban crowd monitoring project, JUST has been deployed in several large‑scale initiatives, including the Xiong’an New Area data platform, Guanghan National Agricultural Industry Park, and the Nantong city‑level governance modernization project (Figure 10).

JUST is now available as a public beta PaaS, providing high scalability, efficiency, and ease of use. Product homepage and portal are shown in Figures 11 and 12.

Compared with traditional GIS vendors, JUST offers richer spatio‑temporal data models, efficient indexing and storage, a full SQL engine, and ready‑to‑use analytical functions, making large‑scale spatio‑temporal data management more convenient and faster.

In summary, the JUST engine enables users to manage massive spatio‑temporal datasets more easily, monitor crowd flows in real time, and take swift actions to ensure public safety.