How Nvidia’s Open‑Source LocateAnything‑3B Enables Image & Video Target Pointing and Open‑Vocabulary Grounding

As visual‑language models evolve from merely recognizing images to performing precise visual grounding, tasks such as open‑vocabulary detection, GUI agent interaction, document understanding, and autonomous‑driving perception demand higher accuracy in locating targets.

Most existing models generate bounding‑box coordinates token by token, which harms the internal geometric consistency of the box and limits inference speed because of the strict sequential decoding order.

To address this bottleneck, Nvidia recently open‑sourced LocateAnything‑3B, a 3 billion‑parameter visual‑language grounding model that supports open‑vocabulary object detection, pointer‑based expression grounding, OCR text localization, GUI element localization, and target pointing in both images and videos, aiming to provide a unified grounding framework.

The model’s core innovation is Parallel Box Decoding (PBD), which predicts entire bounding boxes, keypoints, and other geometric elements in a single parallel step. PBD preserves the geometric structure of the target region and dramatically increases decoding throughput, enabling faster inference without sacrificing precision.

Nvidia also built a large‑scale training pipeline, releasing the LocateAnything‑Data collection containing over 138 million samples spanning natural scenes, robotics, autonomous driving, GUI interactions, document understanding, and OCR. This extensive dataset improves the model’s generalization across diverse real‑world scenarios.

Experimental results on multiple visual‑grounding benchmarks show that LocateAnything‑3B simultaneously achieves higher localization quality and faster decoding speed, breaking the traditional speed‑accuracy trade‑off and providing a foundational capability for emerging GUI agents, auto‑annotation systems, and next‑generation multimodal AI agents.

HyperAI hosts a notebook‑based tutorial for the model. Users can navigate to the HyperAI tutorial page, clone the notebook repository, select an NVIDIA RTX 5090 + PyTorch container, launch the workspace, run the notebook, and view the demo results, making deployment straightforward.

For experimentation, HyperAI also offers free compute resources such as RTX 5090 and PRO 6000 GPUs through a giveaway program.