How Alibaba Sports Built AI‑Powered Home Exercise with Real‑Time Pose Detection

Background

Over the past year, Alibaba Sports' technology team explored "endpoint intelligence" and launched an AI‑exercise project that digitizes sports, enabling users to exercise at home with only a phone and a small space.

Endpoint Intelligence Practice

After a year of development, the team built a systematic client‑side intelligent sport platform that runs inference on the phone using Alibaba's deep‑learning engine, detects human pose and actions, analyzes trajectories and angles, and provides real‑time feedback and correction for more than ten exercise types.

Technical Support

The core technique uses the MNN inference engine for pose detection:

Real‑time detection of 14 key skeletal points from images or video.

Connecting points to form lines creates action representations for posture, angle and trajectory analysis.

Action‑pose matching enables timing, counting, standardization feedback and interactive correction.

Improving Recognition Accuracy

Accuracy depends on the matching algorithm; improvements focus on stabilizing skeletal points, selecting representative actions for state machines, and ensuring frame rates cover all states. Examples show how mis‑detected arm points or low frame rates lead to incorrect action matching and how historical motion data can be used for correction.

Reducing Performance Consumption

Mobile devices have limited CPU, memory and power, so the three inference stages—pre‑processing, inference, and post‑processing—are optimized. Pre‑processing minimizes format conversions, inference relies on MNN, and post‑processing uses platform‑specific rendering (e.g., Metal on iOS) to lower energy use and heat.

Improving Testing Efficiency

Traditional manual testing is costly and inconsistent. The team built an AI‑exercise automated testing tool that batch‑processes video samples, extracts skeletal data, runs business‑logic tests, and generates reports, dramatically reducing labor, improving coverage, and enabling quantitative evaluation of model accuracy, algorithm efficiency, and performance.

Business Results

The AI‑exercise system now supports dozens of motions, offers AI training courses, and runs continuous "celebrity coaching" sessions to boost user engagement. Ongoing development will expand motion libraries and enrich product features, establishing a distinctive intelligent‑sports brand.