How PiLoT Enables Monocular Drones to Navigate 10 km Drift‑Free and Lock onto Targets

In GNSS‑denied or jammed environments, a UAV equipped only with a monocular RGB camera must obtain precise, drift‑free self‑localization and target positioning.

Traditional pipelines combine visual‑inertial odometry (VIO) with GNSS for pose and use a heavy lidar for target coordinates, which introduces external‑signal dependence and payload constraints.

PiLoT Overview

PiLoT (Neural Pixel‑to‑3D Registration for UAV‑based Ego and Target Geo‑localization) directly registers each video frame to a pre‑built 3D geographic model, providing global, drift‑free 6‑DoF pose estimation and real‑time 3D target localization using only the camera feed.

Technical Challenges

Efficient allocation of compute resources for dense 2D‑to‑3D correspondence on an embedded platform.

Learning geometry‑aware features that bridge the domain gap between ground‑based training data and aerial visual conditions.

Core Innovations

Render‑Pose dual‑thread decoupling : Two high‑concurrency threads run independently. The rendering thread continuously generates geo‑referenced synthetic views; the pose thread rapidly registers the live video stream to these views, breaking the linear serial bottleneck.

Million‑scale global synthetic dataset : Constructed with AirSim, Cesium and Unreal Engine, the dataset supplies absolute pixel‑depth and precise 6‑DoF ground truth across diverse weather, illumination and viewpoint variations, enforcing learning of underlying 3D geometry.

Efficient pixel‑to‑3D registration framework : Uses a lightweight MobileOne‑Unet backbone trained on the synthetic data. A “one‑to‑many” registration mode renders a single reference view as a geographic anchor and matches multiple pose hypotheses within a shared feature space, drastically reducing rendering cost.

Rotation‑aware anisotropic sampling : Expands the search space for yaw and pitch, handling up to 10 m translation and 10° rotation between frames.

Multi‑scale feature‑pyramid Levenberg‑Marquardt optimization : Custom CUDA kernels accelerate LM iterations, delivering a 30× speedup and smooth convergence to the global optimum.

Experimental Evaluation

PiLoT was benchmarked against hybrid methods (Render2ORB, Render2RAFT) and absolute‑pose baselines (PixLoc, Render2Loc equipped with LoFTR, EfficientLoFTR, RoMaV2, Aerial‑MASt3R). On SynthCity‑6, UAVScenes and UAVD4L‑2yr datasets, PiLoT runs at 28 FPS, achieves meter‑level accuracy, and outperforms all competitors in both self‑localization and dynamic target tracking.

In a 13‑minute, >10 km flight test, PiLoT maintained 25‑30 FPS inference and recorded a mean positioning error of 1.374 m. Dynamic 3D target localization was obtained via ray tracing using the accurate UAV pose.

Conclusions and Future Work

Direct pixel‑to‑3D alignment eliminates cumulative drift and enables GNSS‑independent navigation. High‑quality synthetic data with strict geometric supervision provides zero‑sample generalization to unseen real‑world scenes. Future work will explore lighter map representations such as digital orthophoto images (DOM) and digital elevation models (DEM) to further reduce map acquisition constraints.

Paper: https://arxiv.org/abs/2603.20778 (CVPR 2026 Highlight)

Project page: https://nudt-sawlab.github.io/PiLoT/