SurgMotion: Billion‑Parameter Model Pushes Video AI to Motion Prediction

Paradigm Shift: From Pixel Reconstruction to Spatiotemporal Motion Prediction

Traditional medical vision models rely on single‑frame perception or pixel‑level reconstruction, which struggles with surgical smoke, glare, and bleeding. SurgMotion replaces pixel decoding with latent‑space motion prediction using a video‑joint embedding predictor (V‑JEPA), introducing three core innovations: Motion‑Guided Latent Masked Prediction, Spatiotemporal Affinity Self‑Distillation, and Spatiotemporal Feature Diversity Regularization (SFDR).

Largest Surgical Video Pre‑training Dataset

The SurgMotion‑15M dataset contains 3,658 hours of real surgical footage, totaling 15 million frames from 50 sources and covering 13 anatomical regions across multiple specialties (laparoscopy, thoracoscopy, neurosurgery, ophthalmology, ENT, etc.). This scale provides unprecedented generalisation and cross‑department applicability.

Billion‑Parameter Scale

With 1 billion parameters, SurgMotion integrates video feature space to capture instrument‑tissue interaction, tissue deformation, and procedural rhythm, surpassing previous models in representing dynamic and spatiotemporal information.

Comprehensive Evaluation on 17 Core Tasks

In systematic benchmarks covering workflow recognition, action‑triplet recognition, depth estimation, lesion segmentation, and skill assessment, SurgMotion achieves leading results: workflow F1 improves by 14.6 % on EgoSurgery and 10.3 % on PitVis; action‑triplet mAP‑IVT reaches 39.54 % (world record); depth error drops 2.95 %; lesion segmentation accuracy rises ~1.0 %.

Rapid Adoption and Open Ecosystem

Within three months the model became the most‑downloaded surgical video foundation model on Hugging Face, with usage requests from 5 medical‑technology companies (e.g., Intuitive Surgical, Karl Storz, ZEISS), 22 universities and research institutes, and 7 hospitals. All weights, fine‑tuning code, evaluation framework, and environment interfaces are openly released on GitHub and Hugging Face.