Amap's 16 NeurIPS 2026 Papers: Multimodal LLMs, Spatial Intelligence & Visual Reasoning

NeurIPS 2026 received 30,709 submissions and accepted 7,900 papers (25.7% acceptance rate). Amap contributed 16 accepted papers spanning multimodal large models, spatial intelligence, visual reasoning, reinforcement learning, and related areas.

Paper 1: OmniVerifier-C1: Decomposed Graded Verifier for Rule-Governed Generative World Modeling

Domain: image generation reinforcement learning. Traditional verifiers only judge overall correctness, failing to reward partially correct outputs. The proposed decomposed graded verifier breaks complex requirements into sub-questions, scores each, and aggregates. The verifier is trained for per-item judgment and overall scoring; for tasks with reference images it cross-checks against ground truth. Results: SD3.5 and FLUX GenEval scores rise from 0.66/0.68 to 0.77/0.79; sub-question judgment accuracy reaches 94.0%. Limitation: mainly effective for text-aligned and reference-based visual tasks; open-world generalization needs further validation.

Paper 2: Concentrated Gradients Amplify Forgetting: Dominant-direction Projection for Continual Multimodal Learning (DiGPro)

Domain: multimodal LLMs, continual learning. Catastrophic forgetting persists even with LoRA because new tasks concentrate updates on high-sensitivity parameters shared across tasks. DiGPro uses recent gradients to adaptively extract a dominant subspace and suppresses updates aligned with that subspace based on their overlap, preserving learning signals in other directions without storing old data or subspaces. On an 8-task benchmark (image, audio, video) DiGPro achieves the lowest average forgetting across three task orders. On MLLM-DCL benchmark it reaches 65.34% average accuracy and 2.27% average forgetting, outperforming all compared methods.

Paper 3: TransitLM: A Large-Scale Dataset and Benchmark for Map-Free Transit Route Generation

Domain: route generation, evaluation benchmark. Can LLMs plan transit routes without map databases? TransitLM builds a dataset from 13M real transit planning records, enabling models to learn network topology, spatial relations, and routing patterns. Benchmarks cover optimal route, preference-aware, and multi-route generation. A 4B model achieves 97.9% route connectivity, 98.9% stop attachment, and 73.7% exact match without any map access. Using only GPS coordinates (no place names) exact match remains 72.9%. Performance is comparable to a tool-augmented agent calling the Amap transit API, with higher stop-sequence overlap.

Paper 4: SaaSBench: Exploring the Boundaries of Coding Agents in Long-Horizon Enterprise SaaS Engineering

Domain: coding agents. Existing benchmarks focus on local edits or simple single-stack projects, not heterogeneous enterprise SaaS environments. SaaSBench provides 30 tasks across 6 domains, 5,370 verification nodes, covering 8 languages, 6 databases, 13 frameworks. A DAG-based dependency-aware evaluation combines rule-based and model-based judging. Best Pass@1 is only 20.68%; bottlenecks lie in environment setup, deployment stability, and multi-component integration. Agents also exhibit premature termination and ineffective debugging loops.

Paper 5: Masked Generative Pretraining Improves Cross-Dataset Transfer in Pixel-Space Diffusion (MPG)

Domain: image generation, diffusion models, self-supervised pretraining. Pixel-space diffusion models lag latent-space models in training efficiency. Contrastive pretraining (e.g., EPG) injects invariance priors that erase fine-grained details when transferring to out-of-distribution datasets. MPG uses masked image modeling without augmentations: the encoder reconstructs masked regions and maintains prediction consistency across adjacent diffusion trajectory samples with different noise levels. This dual objective captures low-level details and representation consistency along the diffusion ODE. Fine-tuning discards the lightweight decoder and attaches a higher-capacity generative decoder. A new metric NC-CKNNA measures semantic neighborhood consistency across noise levels. Across 7 downstream datasets (CelebA-HQ, CIFAR-10, Flowers, LSUN-Bedroom/Church, Places, ADE20K) MPG outperforms contrastive baselines, with average FID relative improvement of 15.1% (up to 26.0% on Places) and a competitive FID of 2.21 on ImageNet-256.

Paper 6: ALAM: Algebraically Consistent Latent Action Model for Vision-Language-Action Models

Domain: embodied intelligence, VLA. Robot action data is scarce; existing latent action models learn from video reconstruction but representations may not suit policy generation. ALAM imposes algebraic consistency on frame triplets: composition consistency (sum of two consecutive changes approximates the full change) and inverse consistency (reverse change approximates the negative of forward change). Joint flow matching in downstream training generates latent action trajectories and robot actions simultaneously. Compared to π₀ baseline, MetaWorld MT50 success rate jumps from 47.9% to 85.0%, LIBERO from 94.1% to 98.1%, with consistent gains on real robot tasks.

Paper 7: DriveStreamBench: Evaluating User-Conditioned Watch-and-Notify in Streaming Driving Video

Domain: VLMs, streaming video understanding, proactive interaction evaluation. Current driving VLM benchmarks focus on offline perception and short-term reasoning, lacking systematic tests of continuous monitoring and user-conditioned notification. DriveStreamBench organizes the same driving events into four cognitive levels, supporting hierarchical analysis and gated evaluation under a unified streaming protocol. Multiple VLMs and a supervised fine-tuned reference model (DriveStream-SFT) are evaluated. Results show all tested models lack this basic but necessary capability; current VLMs are not ready for reliable user-conditioned reminders in driving scenarios.

Paper 8: Urbex: Agentic Spatial Grounding for City-Scale 3D Scenes

Domain: spatial intelligence, 3D visual grounding, multimodal agents, RL. City-scale 3D object grounding typically uses candidate ranking, requiring pre-generated candidates and suffering inefficiency in large outdoor scenes. Urbex reframes the task as active spatial search: an agent interacts with a multi-view environment, queries landmarks, zooms local regions, and renders oblique views to locate the target, then submits a bounding box. RL on CityRefer uses grounding rewards and auxiliary exploration rewards. Urbex achieves the best Hit@1 and IoU on CityRefer, enables candidate-free grounding at inference, is significantly faster than the strongest candidate-ranking baseline, and shows improved cross-scene robustness in zero-shot tests on STPLS3D-Refer.

Paper 9: PRPO: Perception-Reinforced Policy Optimization via Token-Level Dynamic Advantage Reshaping

Domain: multimodal LLMs, visual reasoning, RL. Multimodal LLMs may fail due to perception errors rather than reasoning deficits; key visual tokens (digits, coordinates, geometric relations) are few but affect the entire reasoning chain. Existing RL assigns uniform advantage across all tokens. PRPO introduces Robust Visual Dependency (RVD) by comparing predictions on original, strongly perturbed, and weakly perturbed images to identify key visual tokens that rely on visual evidence and are stable to mild noise. Perception Advantage Reshaping (PAR) uses a bounded S-shaped mapping to dynamically adjust token-level learning signals, reinforcing key visual information while preserving language reasoning learning. On 7 multimodal reasoning benchmarks, 3B and 7B models improve average scores from 30.7%/41.4% to 54.0%/62.5% (gains of 23.3/21.1 percentage points), outperforming VPPO by 1.9/1.7 pp.

Paper 10: What if Agents Could Imagine? Reinforcing Open-Vocabulary HOI Comprehension through Generation (ImagineAgent)

Domain: multimodal agents, HOI. MLLMs struggle with open-vocabulary human-object interaction due to cross-modal hallucinations and limited image perspectives. ImagineAgent combines cognitive mapping, tool-augmented RL, and generative world modeling. Innovations: hicodet-6K CoT dataset organizing perceived entities into interaction pairs; a multimodal tool library (online retrieval, image cropping, generative modeling) for dynamic tool use; generative models reconstruct images from different perspectives for "imagination"; a composite reward optimizing prediction accuracy and tool-use efficiency. Using only 36.7% of the training data of existing methods, ImagineAgent achieves state-of-the-art on SWIG-HOI and HICO-DET.

Paper 11: Stable3R: Streaming 3D Reconstruction with Stable Geometric References

Domain: streaming feed-forward reconstruction, spatial intelligence. Streaming 3D reconstruction over-relies on the first frame with limited information; early errors are uncorrected and propagate, causing unstable scene structure and camera trajectories. Stable3R makes geometric references more reliable as observations accumulate. Forward-looking enhanced attention fuses new observations into a history cache for richer geometric cues while maintaining causal reasoning. A frame-equivariant architecture with relative pose supervision between historical frames reduces first-frame dependence. Results: strong reconstruction in sparse views, dynamic scenes, and long sequences. On TUM dynamic scenes, absolute trajectory error drops 50% versus StreamVGGT; trained on max 24-frame sequences with bounded memory, it generalizes to 300-frame inference.

Paper 12: Breaking the Static: Dynamic Text Conditioning for Diverse Image Generation

Domain: image generation, diffusion models, diversity. Text-to-image diffusion models lack diversity: similar layouts, poses, and styles for the same prompt due to high-probability mode preference and post-training alignment. The proposed training-free dynamic semantic interpolation uses CLIP to evaluate each word's semantic contribution, constructs a low-frequency prompt of dominant semantics, and dynamically interpolates low-frequency and full prompt embeddings during early denoising, letting the model explore diverse modes before introducing full semantics; later stages use the full prompt for alignment and quality. Comprehensive experiments show diversity improvement across general, RL-aligned, and distilled models while maintaining text alignment and aesthetic quality; the method combines additively with existing diversity techniques; ablation confirms component effectiveness, synergy, and lightweight nature.

Paper 13: Hybrid Reinforcement Learning for One-step Degraded Infrared and Visible Image Fusion (OneHRL)

Domain: infrared-visible image fusion, diffusion models, RL. Complex degradations (fog, rain, noise, stripes) affect both modalities; diffusion models restore well but multi-step sampling is costly; one-step distillation oversmooths textures. OneHRL: Stage 1 distills the diffusion prior, treating the fused degraded bimodal representation as an intermediate noise state to achieve restoration and fusion in one forward pass. Stage 2 introduces Flow-GRPO, converting deterministic generation into stochastic exploration with a coarse-to-fine hybrid reward: Qwen2.5-VL evaluates global semantics and naturalness; SAM instance segmentation reinforces target details across five dimensions (thermal target saliency, local sharpness, information entropy, infrared information preservation, structural similarity). On M³FD and MSRS composite degradation tests, OneHRL leads in fusion metrics and downstream detection/segmentation; single-image inference takes 0.4 seconds versus DRMF's 25 seconds (62.5× speedup).

Paper 14: PLAN4D: Generative Plannable 4D Worlds

Domain: dynamic 4D scene reconstruction, generative world models, video generation. Existing world models for future prediction focus on visual aesthetics, lack explicit 4D modeling, and have weak physical grounding and limited cross-time geometric consistency. Plan4D predicts future RGB video and dense 4D geometry (depth, camera parameters, point clouds, scene flow) given observed context and text-specified actions. It uses a model-stitching architecture combining Depth Anything 3 and LongCat-Video, plus an auxiliary DPT head for scene flow, leveraging complementary priors from video generation and 3D reconstruction foundation models due to scarcity of paired action-annotated 4D data. Experiments on synthetic and real-world datasets show state-of-the-art performance in environment-responsive generative 4D modeling, producing complex, long-horizon, fine-grained actions with improved visual fidelity and geometric accuracy.

Paper 15: Learning Agentic Policy from Action Guidance (ActGuide-RL)

Domain: agentic RL, LLMs. LLM agent RL relies on base model exploration; when the model cannot independently complete complex tasks, no success reward is obtained for learning. Existing methods need supervised fine-tuning with high-quality interaction trajectories, which is costly. ActGuide-RL uses action data from daily human-computer interactions as plan-style reference guidance to help agents overcome reachability barriers to success states. It follows minimal intervention: guidance is introduced adaptively only when autonomous exploration stalls. Hybrid policy training jointly optimizes guided and unguided trajectories, internalizing exploration gains into independent task-solving ability. Theoretical and experimental analysis examines the trade-off between guidance benefit and off-policy risk. On search agent benchmarks, Qwen3-4B based experiments show GAIA and XBench success rates improve by 10.7 and 19 percentage points over direct RL, matching SFT+RL performance without SFT cold-start.

Paper 16: VeriTrip: A Verifiable Benchmark for Travel Planning Agents over Unstructured Web Corpora

Domain: travel planning, evaluation benchmark, multimodal agents. As LLM agents gain autonomous retrieval and complex planning, benchmarks still rely on structured data or preset APIs, failing to measure information acquisition, cross-modal understanding, and fact verification in real unstructured web environments. VeriTrip provides a multimodal retrieval corpus with 8,210 documents and 4,146 images, plus a verifiable knowledge base inaccessible to agents for fact-checking. Agents must autonomously retrieve heterogeneous information, recognize destinations via images, and synthesize transport, accommodation, attractions, and user preferences into a travel plan. A fine-grained fact verification mechanism decomposes the plan into verifiable units and compares them against the knowledge base. Experiments with mainstream MLLMs reveal challenges balancing factual reliability and instruction following. For example, Claude-4.5-Sonnet achieves 68.60% factual reliability on complex tasks but only ~50% of plans satisfy all user preferences.