Turn Your Smart Speaker into a Pet‑Friendly VLog Camera with AI

Product Overview

The Tmall Genie CC/CCL smart speaker can recognize household pets and automatically capture their most interesting moments, thanks to AI Labs' new smart photography algorithm called Pet Secret , which brings an intelligent camera into millions of Chinese homes.

Feature Introduction

Users set a time window on the speaker; even when they are away, the device records cats and dogs playing, stitches the footage into short clips, and saves them, ensuring no memorable moment is missed.

VLog Market

Short‑form video platforms have surpassed 500 million monthly active users, with daily active peaks of 160 million, making VLog—a video diary of daily life—a highly popular content format.

Why Smart Photography?

Traditional pet photography suffers from three main pain points: lack of time, repetitive results, and the difficulty of capturing unpredictable animals. Early attempts like Google Clips demonstrated automatic capture but required massive professional labeling and lacked true perception.

Our Solution: Pet Secret

Pet Secret addresses these challenges with three core functions:

Content Understanding : AI perceives each video frame, labeling objects such as cats, dogs, and people.

Highlight Extraction : Edge‑cloud algorithms score interest for every frame and extract the most engaging segments.

Smart Editing : Automatic speed adjustment, montage creation, and music matching produce polished short videos.

How It Works

The algorithm relies on interest prediction , a research area that assigns an interest score to video segments, similar to recommendation systems.

Challenges include the need for large‑scale professional annotations and the lack of semantic perception. Pet Secret overcomes these by using a lightweight edge model for coarse filtering and a powerful cloud model for fine‑grained analysis.

System Architecture

The design combines edge and cloud processing:

Edge filtering discards irrelevant footage before upload, ensuring privacy and reducing bandwidth.

Uploaded clips are anonymized and processed in the cloud with user authorization.

Final videos are stored in the user's private cloud space.

The edge runs AI Labs' proprietary ACE engine , capable of 24‑hour continuous operation, while the cloud service performs high‑resolution interest prediction and video fusion within an hour.

Interest Prediction Pipeline

Data Labeling

Large crowdsourced pet video collections are annotated with absolute interest scores, requiring at least ten annotators per video to mitigate subjectivity.

Device‑Side Model Training

The on‑device model receives fixed‑length video clips and predicts a binary outcome: whether the clip contains a pet and is interesting enough to upload.

A perception module tags each frame with object categories and locations, feeding these features into the interest predictor.

Cloud‑Side Model Training

Cloud models have greater compute and memory, enabling richer perception and higher‑FLOP architectures. They also perform video fusion across adjacent clips and generate a continuous interest curve for each video, which guides precise segment cutting and automatic editing.

Privacy Guarantees

Three safeguards protect user data: edge‑side coarse filtering, cloud‑side anonymization with user consent, and storage of final videos in a private cloud.

Current Capabilities

Pet Secret currently supports cats and dogs, with plans to expand to other pets. All generated videos are algorithm‑produced, not promotional content.

Illustrations