How Frontend Engineers Turn 360° Panorama Videos into Click‑Boosting Cover Clips

Panorama Cover Video on Hand Taobao

During the "Double Promotion" period, Hand Taobao produced many high‑quality 360° panorama videos and generated cover videos for them, which doubled the click‑through rate of the modules that displayed these covers.

Application of Panorama Video

Panorama video is captured with a 3D camera and allows users to freely rotate the view, providing an immersive experience without spatial limits. It is mainly used in real‑estate and home‑decoration scenarios to showcase products in a way that static images cannot.

Business Value

Improves consumer shopping experience with 720° rotation and reduces decision cost.

For the platform, low shooting and production costs unlock a large supply of high‑ticket home‑decoration content, enriching the content ecosystem.

For merchants, flexible shooting and enhanced public‑traffic acquisition.

Why a Cover Video?

Because panorama videos cannot be directly displayed on list pages, a cover video is needed to expose the content on the homepage, search results, and other list pages. Three feasible solutions were evaluated: a static image, a static frame from the panorama, and a dynamic cover video. The dynamic cover video was chosen for its higher conversion potential.

Cover Video Production Process

The cover video must meet three requirements: high consumer experience, low generation cost (preferably automatic), and short duration.

Generation Schemes

Algorithm‑driven: automatically detect product segments, low merchant cost, high development cost, unpredictable quality.

Manual: merchants select view and content, high merchant cost, controllable quality.

Rule‑based: generate based on configurable rules, moderate cost and quality.

Considering the early stage of the product, the rule‑based approach with backend configuration and merchant confirmation was adopted.

Default rule:

Take the first 8 seconds of the video.

Center the view on the middle of the video.

During playback, move the view 20° left and right.

Technical Research

Two directions were explored: server‑side rendering with FFmpeg and front‑end rendering with WebGL.

Server‑Side Scheme

$ ffmpeg -ss 00:00:00 -t 00:00:05 -i panorama.mp4 -vcodec copy -acodec copy output.mp4

$ ffmpeg -i output.mp4 -vf v360=e:c3x2:cubic:out_pad=0.01 output2.mp4

After mapping to a sphere or cube, a specific view is extracted.

Front‑End Scheme

The front‑end renders the panorama with Three.js, drives view changes according to rules, captures the canvas stream, and uploads the resulting video.

Render panorama in the browser via WebGL.

Control view based on configurable rules.

Capture canvas media stream with captureStream.

Convert stream to a video file using MediaRecorder.

Upload the file to the content platform.

Three.js Implementation

Initialize scene, camera, and renderer, then load six cube‑map textures or a video texture for ERP/EAC projections.

const scene = new THREE.Scene();
const width = window.innerWidth > 640 ? 640 : window.innerWidth;
const height = window.innerHeight;
const camera = new THREE.PerspectiveCamera(90, width / height, 0.1, 100);
const renderer = new THREE.WebGLRenderer();
renderer.setPixelRatio(window.devicePixelRatio);
renderer.setSize(width, height);
document.getElementById('container').appendChild(renderer.domElement);

For cube‑map rendering, load six textures and create a box geometry with inverted scale so the camera is inside the cube.

const geometry = new THREE.BoxGeometry(1, 1, 1);
const textures = [/* six URLs */];
const materials = textures.map(url => new THREE.MeshBasicMaterial({ map: new THREE.TextureLoader().load(url) }));
const box = new THREE.Mesh(geometry, materials);
box.geometry.scale(1, 1, -1);
scene.add(box);

For ERP video rendering, replace the geometry with a sphere and use a VideoTexture:

const geometry = new THREE.SphereGeometry(1, 32, 16);
const video = document.createElement('video');
video.muted = true;
video.src = 'https://example.com/panorama.mp4';
const texture = new THREE.VideoTexture(video);
const material = new THREE.MeshBasicMaterial({ map: texture });
const sphere = new THREE.Mesh(geometry, material);
scene.add(sphere);
video.play();

View Control API

A helper function locate(longitude, latitude, fov) converts spherical coordinates to Cartesian coordinates and orients the camera.

function locate(longitude = 0, latitude = 0, fov = 90) {
  const modifiedLat = Math.max(-90, Math.min(90, latitude));
  const phi = THREE.MathUtils.degToRad(90 - modifiedLat);
  const theta = THREE.MathUtils.degToRad(longitude + 180);
  const distance = 1;
  const x = distance * Math.sin(phi) * Math.cos(theta);
  const y = distance * Math.cos(phi);
  const z = distance * Math.sin(phi) * Math.sin(theta);
  camera.lookAt(x, y, z);
  camera.fov = fov;
}

Animating the view according to the default rule:

const duration = 2000;
setTimeout(() => locate(-20), duration);
setTimeout(() => locate(0), duration * 2);
setTimeout(() => locate(20), duration * 3);
setTimeout(() => locate(0), duration * 4);

For smoother motion, Anime.js can drive a numeric value and call locate on each frame.

function runAnimate(update) {
  const anim = { x: 0 };
  anime.timeline({
    targets: anim,
    duration: 2000,
    easing: 'linear',
    update: () => update(anim)
  })
    .add({ x: -20 })
    .add({ x: 0 })
    .add({ x: 20 })
    .add({ x: 0 });
}
video.addEventListener('play', () => {
  runAnimate(({ x }) => locate(x));
});

Media Capture and Upload

Capture the renderer canvas stream and record it:

let recordedBlobs = [];
let mediaRecorder;
function startRecording() {
  mediaRecorder = new MediaRecorder(renderer.domElement.captureStream());
  mediaRecorder.ondataavailable = event => {
    if (event.data && event.data.size > 0) recordedBlobs.push(event.data);
  };
  mediaRecorder.start();
}
function stopRecording() {
  mediaRecorder.stop();
}

Upload the resulting Blob as a file using the internal uploader library:

import { createUploader } from '@ali/speedster-media-upload';
const file = new File([new Blob(recordedBlobs)], 'cover.webm');
const uploader = await createUploader();
const fileInfo = await uploader.startUpload(file);

Future Outlook

Support multiple projection formats to reduce server‑side transcoding.

Add audio playback for review scenarios.

Provide interactive controls (progress bar, fullscreen, speed) for merchant review.

The front‑end solution demonstrates how 360° panorama video can be turned into engaging cover clips that boost exposure during large‑scale promotions while keeping production costs low.