Self-Service Security System for a Social Karaoke App: Design and Implementation

Background

The social karaoke app faces severe security threats such as harassment, advertising, fake followers, pornographic, and political content, which jeopardize healthy operation. Existing automatic and manual review mechanisms cannot keep up with the volume and variety of malicious content, and manual review is limited by manpower.

High‑level users (e.g., wealthy or influential players) are targeted by covert ads that are few in number, making them hard for staff to detect and for users to report promptly. The previous workflow required users to either tolerate the harassment or report it late, leading to low efficiency and high labor cost.

Principle

To address manpower shortage and low processing efficiency, the solution leverages trusted external community users to assist with content moderation.

These external users have high activity, contribution, and loyalty, making them ideal for detecting malicious behavior. By granting them limited, non‑sensitive management permissions, the system supplements internal staff, accelerates response, and improves user experience.

A dedicated security team composed of high‑trust external users handles malicious/illegal content quickly and at scale. Reports from internal staff or trusted external users trigger automatic actions (e.g., delete image, remove work, ban account) based on fine‑grained permission settings. High‑trust internal users can handle all content; external users can handle low‑risk, high‑frequency issues.

Automatic analysis prioritizes reports from high‑level users or multiple reporters for manual review or auto‑processing, while protecting VIP, high‑level, and whitelist users from erroneous actions.

Implementation

Legacy reporting workflow required full manual review, multiple hand‑offs, and PC‑only management tools, resulting in slow response and limited coverage.

The optimized solution reuses the existing reporting channel and introduces a refined permission management platform:

Permission management grants specific users automatic processing rights with quota limits (e.g., 1‑3 automatic actions). External high‑trust users start with limited scope; successful reports lead to increased permissions.

A reporting content management platform records all reports, enabling operators to query, filter, and confirm issues efficiently.

Protection mechanisms for VIP, high‑level, and whitelist users prevent automatic actions unless multiple trusted reports are received.

Automatic analysis and ranking of reports surface high‑priority items (multiple reports, high‑level reporters, newly created accounts) for rapid operator attention, complemented by behavior‑analysis monitoring.

Effect

After deployment, the complex communication and processing steps were reduced to a single click on the mobile client. Average response time for urgent security incidents dropped from 20 minutes to sub‑second levels. Daily missed security issues fell from ~15 to only a few per week.

Conclusion

The karaoke app serves over 35 million daily active users, generating diverse UGC (text, image, audio, video) and facing a wide range of security challenges (ads, scams, porn, politics, fake interactions). Development and operations manpower is limited, and off‑hours handling required VPN‑based PC tools, causing delays.

The self‑service security system requires minimal development effort, works without client modifications, reuses the existing reporting channel, and automatically processes high‑trust user reports. Fine‑grained permission delegation expands the review capacity by involving trusted external members. Since launch, the system has virtually eliminated external security incidents.