High‑Definition Maps and Localization for Autonomous Driving: Concepts, Pipeline, and Challenges

The talk, presented by Liang Jihong (Pony.ai Tech Lead), focuses on the application of high‑definition (HD) maps in autonomous driving, divided into two main parts: HD maps and localization.

1. High‑Definition Map

★ Topological Map / Road Graph

Traditional maps (e.g., Baidu maps) provide topological information, POIs, and meter‑level precision, which are insufficient for autonomous vehicles that require precise lane‑level data.

Typical driving scenario: a vehicle must know its lane, neighboring lanes, lane change feasibility, and semantic information (e.g., green belts) to make safe decisions.

3. HD Road Map

The HD Road Graph (topological map) describes road geometry at centimeter‑level accuracy, including lane types, connectivity, traffic signs, lights, crosswalks, and other semantic details.

Applications include precise traffic‑light localization and decision‑making based on semantic map information.

HD Map Production Pipeline

Data collection (sensor‑rich mapping vehicle)

Data cleaning/aggregation

Automated extraction of lane lines, traffic lights, intersections, etc.

Manual inspection and annotation to achieve near‑100% accuracy

Post‑processing and verification to ensure map reliability

Release and versioned management of validated map data

Challenges include high sensor cost, massive data storage/computation, expanding coverage, and maintaining accuracy.

★ 3D Grid Map

A 3D grid map discretizes the environment into voxels, storing the occupancy probability of each voxel. Input: LiDAR point clouds; Output: probabilistic 3D occupancy used for localization and static‑environment perception.

2. SLAM

Simultaneous Localization and Mapping (SLAM) builds a map while estimating the vehicle’s pose, using sensors such as LiDAR, cameras, and GNSS/IMU. Traditional SLAM is 2D and vision‑only, whereas autonomous‑driving SLAM is 3D, uses LiDAR, and incorporates GPS.

3. Pose Graph Optimization

Collected poses are discretized into vertices; relative constraints form edges. Optimization adjusts vertex positions to satisfy edge constraints, yielding a globally consistent map.

4. ICP (Iterative Closest Point)

ICP aligns two point clouds by iteratively finding closest point pairs and solving for the optimal rigid transformation.

5. Removing Dynamic Obstacles

Probabilistic models and machine‑learning methods filter out non‑static objects from the map to improve perception.

2. Localization

Localization requires centimeter‑level accuracy and millisecond latency. Methods include GNSS/RTK, IMU‑based inertial navigation, and point‑cloud matching against the HD map.

IMU provides high‑frequency acceleration and angular velocity but suffers from drift; inertial navigation mitigates this drift using sensor fusion.

Point‑cloud localization is robust to GNSS outages but depends on the prior map’s accuracy and timeliness.

Multi‑sensor fusion, typically based on Kalman filtering, combines GNSS, IMU, and map‑based measurements to achieve high‑precision, low‑latency vehicle pose estimates.

Overall, the system aims for reliable, low‑cost, and robust positioning even in challenging environments such as traffic jams or GPS‑denied areas.