AR Fundamentals and Shoe Try‑On Implementation

This talk introduces Augmented Reality (AR), distinguishing it from AI and VR, and explains how AR seamlessly integrates real‑world and virtual information to enhance perception.

Applications of AR include showrooms, games, interior design, advertising, and virtual try‑on; at 得物 the focus is on shoe try‑on, supporting over 1500 models.

The AR shoe‑try‑on pipeline captures video at 30 fps, processes each frame with AI‑based key‑point detection, solves pose to obtain a transformation matrix, and overlays a 3D shoe model rendered via a graphics pipeline.

3D modeling methods discussed are manual digital modeling, laser/depth/structured‑light scanning, and multi‑view photogrammetry, optionally enhanced with light‑field capture for realistic lighting and material properties.

Pose estimation relies on the camera imaging formula and coordinate systems (world, camera, image); the pose matrix is split into translation and rotation, followed by perspective projection to map 3D points to the 2D image plane.

Rendering uses the standard GPU pipeline (application, geometry, rasterization) and can be improved with physically‑based rendering (PBR) that incorporates micro‑facet theory, energy conservation, and BRDF.

Additional techniques such as green‑screen (绿魔) plane and object recognition help with occlusion and shadow effects, concluding with the audience’s positive feedback on the AR overview.