ModelScope CV Model Overview: Visual Detection and Keypoint Applications

ModelScope is a model‑as‑a‑service platform that supports end‑to‑end workflows such as model management, download, fine‑tuning, and inference deployment for a wide range of AI models, including state‑of‑the‑art computer‑vision (CV) models.

The CV model catalog currently offers around 100 models covering image understanding, generation, editing, and video tasks. Users can explore model cards, download models for local installation, or run them directly in cloud notebooks with a single line of SDK code. Deployment options span cloud, on‑premise, and edge devices.

Visual Detection Models are organized by modality (image, video, 3D) and further divided into general‑purpose detectors (e.g., VitDet, YOLOX) and domain‑specific high‑performance detectors (e.g., human, face, vehicle, smoke, mask, safety‑helmet detection). Notable models include:

VitDet – a ViT‑backbone detector pretrained with MAE, achieving strong COCO results without an FPN.

YOLOX – a real‑time detector with automatic GT assignment, decoupled classification‑regression heads, and extensive data augmentation.

res2net – a camouflage‑color detector addressing low visual contrast and limited annotation data.

FasterRCNN with dynamic head – a specialized human detector optimized for low‑light outdoor surveillance.

MogFace – a face detector that won six championships on the Wider Face benchmark.

YOLOX‑PAI – an enhanced vehicle detector surpassing YOLOv5/v6, suitable for occluded and small‑target scenarios.

Stream YOLO – a real‑time video detector that predicts future frames by leveraging temporal context.

OSTrack – a SOTA single‑object tracker robust to occlusion and similar distractors.

These models are deployed in various products, ranging from edge devices to autonomous‑driving platforms.

Keypoint Models include 2D and 3D solutions for human, face, hand, and full‑body pose estimation. Highlights are:

HRNet‑based 2D human pose model with multi‑scale feature fusion, optimized for fitness and sports scenarios.

Lightweight face keypoint model built on MobileNet, enabling real‑time deployment on mobile devices.

litehrnet‑w18 hand keypoint model (HRNetv2 + DarkPose) using heat‑map derivatives for precise decoding.

Full‑body 133‑point model covering face, hand, skeleton, and foot keypoints, supporting downstream 3D reconstruction.

VideoPose3D‑style 3D human pose model that predicts 3D joint locations from multi‑frame video input.

These keypoint solutions power applications such as fitness mirrors, exercise counting apps, beauty filters, gesture recognition, and virtual avatar control.

The roadmap includes releasing higher‑performance detection and keypoint models, end‑to‑end fine‑tuning toolkits, deployment suites, and model‑composition capabilities, while continuing to recruit research interns for detection, keypoint, and visual‑editing algorithms.