How a 29‑Second Video Enables a Robot to Learn a New Skill with the Open‑Source HOST Framework

01 Why HOST Can Learn Skills from Video

Previous attempts to teach robots by video failed because human motion timing and robot actuation timing differ, and human limbs and six‑axis robot arms have incompatible kinematics. Those methods compressed video frames into features and directly mapped them to robot actions, resulting in low success rates.

HOST adopts a different perspective: it ignores absolute time and instead aligns progress. A task such as pouring water is abstracted into a sequence of progress states (start → pick cup → tilt → water flows → stop → return). Both human and robot share this progress scale.

HOST trains a self‑supervised alignment module that maps human video frames and robot trajectories onto a shared task‑progress manifold. The video becomes a guide rather than a passive observer: at any moment the robot identifies its current progress, finds the next video frame that corresponds to the next progress step, and follows that cue.

The approach is broken into three steps, called self‑localizing cascade prediction:

Locate the robot's current progress position within the demonstration video.

Imagine, from the robot's perspective, what the next video frame would look like after the robot performs the next action (a perspective and embodiment transformation).

Infer the concrete robot motions required to produce that imagined view.

This mirrors human observational learning: rather than copying motions, we infer which actions would lead to the observed outcome.

02 Three Numbers That Outperform Conventional Fine‑Tuning

HOST was evaluated on 50 completely unseen manipulation tasks. The results highlight three striking figures:

62% success rate when learning from a single 29‑second video, compared with 19% for the strongest non‑fine‑tuned baseline (AWDA), a 43‑percentage‑point gain.

507× speedup : the mainstream Pi‑0.5+SFT pipeline needs 50 tele‑operation demonstrations and four hours of fine‑tuning to reach 38% success, whereas HOST learns from one short video without any fine‑tuning.

Data efficiency improves 50× and time efficiency improves 507×, meaning the same learning budget could teach roughly 500 skills.

99% retention of prior skills after acquiring a new one, whereas the best fine‑tuned baseline retains only about 40% of previously learned abilities.

The key reason is that HOST never updates model parameters during skill acquisition; all learning happens at inference time, treating the video itself as a portable skill storage medium. This avoids catastrophic forgetting and enables continual skill accumulation.

03 Why “Learn‑by‑Demo” Is Crucial for Home Robots

For household robots, the most natural way to acquire new abilities is through a single human demonstration, just as people teach each other. Users can simply show the robot how to fold clothes, set a table, or tidy toys without programming or tele‑operation equipment. HOST makes this interaction feasible by turning a brief video into an instantly usable skill.

Conclusion

HOST represents a paradigm shift from the traditional train‑then‑fine‑tune loop to inference‑time skill acquisition. The open‑source project provides the paper, training code, and offline data‑processing components, inviting the community to explore embodied intelligence that learns on the fly.

Project homepage: https://host-site.host-robotics.workers.dev
Paper link: https://arxiv.org/abs/2607.20033
GitHub: https://github.com/CGuangyan-BIT/HOST
HuggingFace: https://huggingface.co/papers/2607.20033