ECCV 2026 Paper Showcase: Breaking Vision Perception & Prediction Boundaries

Paper 1: From One-to-One to Many-to-Many — Dynamic Cross-Layer Injection for Deep Vision-Language Fusion

Problem

Existing vision-language models (VLMs) typically connect only the final layer of the visual encoder to the large language model (LLM) input, creating a severe visual feature bottleneck. This static, coarse-grained architecture limits the LLM's ability to align with hierarchical visual knowledge, making it difficult to combine local details and global semantics during coherent reasoning.

Method: CLI Framework

The paper proposes the Cross-Layer Injection (CLI) framework, establishing a dynamic "many-to-many" bridge between vision and language modalities. CLI consists of two parameter-efficient modules:

Adaptive Multi-Projection (AMP) — uses per-layer LoRA to align features from different visual layers.

Adaptive Gated Fusion (AGF) — selectively injects the most relevant visual information based on the LLM's real-time decoding context.

Experiments & Results

CLI is integrated into LLaVA-OneVision and LLaVA-1.5 and evaluated on 28 diverse benchmarks . Results show that granting the LLM on-demand access to the full visual hierarchy consistently improves fine-grained perception, visual grounding, and compositional reasoning, providing a scalable paradigm for deeper multimodal fusion.

Paper 2: E-VLA — Event-Augmented Vision-Language-Action Model for Dark and Blurred Scenes

Problem

Vision-Language-Action (VLA) models excel in well-lit lab environments but fail catastrophically in low-light, under-exposed, or fast-motion-blurred conditions — the robot literally "cannot see." Frame-based perception suffers from physical degradation: short exposures yield black images; long exposures introduce severe motion blur.

Method: E-VLA

Researchers from Zhejiang University and Ant Group introduce E-VLA , the first integration of an event camera into a VLA framework. Unlike prior work that reconstructs RGB frames from events, E-VLA directly injects motion and structural cues from the event stream into a pre-trained VLA architecture, preserving its original semantic understanding while adding complementary perception.

Two plug-and-play fusion strategies are designed:

Zero-parameter overlay fusion

Lightweight hierarchical event adapter

Dataset & Results

A teleoperation platform was built using the open-source SO100 robot arm and a DAVIS346 event camera , yielding the first multi-task, multi-illumination synchronized RGB-event-action manipulation dataset.

At 20 lux (extreme darkness), Pick-Place success rate jumps from 0% (visual-only baseline) to 90% .

Under 1000 ms severe motion blur , Sorting success rate rises from 5% to 32.5% .

E-VLA generalizes to unseen lighting conditions without any additional low-light training data.

Paper 3: Learning Video Dynamics with Predictive Differentiable Rendering (PDR)

Problem

Deterministic video prediction models operating in discrete pixel space with pixel-wise MSE optimization tend to produce over-smoothed results and lose fine details.

Method: PDR & PredGS

The paper proposes Predictive Differentiable Rendering (PDR) , unifying discrete pixel prediction with continuous 2D Gaussian representation in an end-to-end framework. The core component, PredGS , is a lightweight, plug-and-play adapter that predicts per-Gaussian parameters — position, scale, rotation, and arbitrary channel amplitudes — and performs real-time differentiable rendering via a CUDA-accelerated predgsplat module. Training combines L1 and SSIM losses.

Experiments & Results

PDR is evaluated on TaxiBJ, WeatherBench, KTH, and Human3.6M benchmarks. It consistently improves both prediction accuracy and visual fidelity while introducing only minimal additional parameters and computational overhead.

Paper 4: Pushing the Limits of High-Resolution Weather Forecasting through Data Scaling (BaguanHR)

Problem

Development of 0.1° global weather forecasting models has been hindered by insufficient high-resolution training data: long-term reanalysis data (ERA5) is mainly available at 0.25°, while operational 0.1° analysis data covers only the past ~10 years.

Method: BaguanHR — Data Scaling via Super-Resolution

BaguanHR reframes high-resolution forecasting as a data scaling problem rather than a model transfer problem. The key insight: super-resolution (single-time-step spatial reconstruction) has lower conditional entropy and input perturbation amplification than multi-step weather forecasting. BaguanHR applies per-variable super-resolution to convert long-term ERA5 data into 0.1° pseudo-labels, then jointly trains the forecasting model with these pseudo-labels and the real 0.1° analysis data.

Results & Scaling Law

On over 85% of lead times within 72 hours , BaguanHR outperforms all listed baselines.

Overall RMSE is reduced by 4.0% compared to IFS-HRES .

Extending training data from 7 years to 18 years lowers 72-hour and 120-hour RMSE by 4.6% and 4.9% respectively, demonstrating a clear data scaling law .