Content Risk Control Industry Overview and Evaluation System

Guest Speaker: Chen Wentao, Engineer at China Academy of Information and Communications Technology (CAICT), DataFunTalk editorial team.

Introduction: In the context of the digital economy, internet content regulation faces new challenges. China is building an ecosystem for internet content governance, but content review technologies encounter difficulties, prompting the emergence of third‑party organizations for standardization and capability assessment.

01. Development Background of the Content Risk Control Industry

1. The digital economy is a major driver of global growth, characterized by digital industrialization, industrial digitization, digital governance, and data valorization. The surge of online activities has also led to a proliferation of illegal and harmful information, making content risk control a global priority.

2. Internet content regulation has become a worldwide challenge. Since around 2011, regions such as the EU and the US have enacted laws (e.g., GDPR) that shift responsibility for content governance from platform self‑regulation to legal accountability.

3. China is constructing an internet content governance ecosystem, with increasing policy emphasis and detailed regulations covering multiple domains (e.g., cybersecurity law, minors protection, deep‑fake detection).

02. Current State and Challenges of Content Review Technology

1. Core technologies include NLP, OCR, facial recognition, voiceprint, and logo detection. Models are trained offline and then applied to streaming text, audio, video, and images to filter violations. Human reviewers handle ambiguous cases, and feedback loops continuously improve models.

2. Major challenges:

Changing review focus: Public opinion shifts (e.g., sudden events like the Shanghai COVID‑19 outbreak) require rapid model adaptation, while adversarial tactics, black‑gray markets, and content variants increase detection difficulty.

Technical difficulties: Text may contain emojis or pinyin; images may embed subtle information that humans recognize but machines struggle with; deep‑fake detection techniques exist but are not yet mature for production use.

03. Establishment of the Content Technology Promotion Alliance

In December 2021, the "Content Technology Promotion Alliance" was founded to provide an open platform for third‑party technical exchange, covering content generation, risk control, and dissemination technologies, as well as standard formulation and government support.

The alliance officially launched in January 2022 with over 60 participating enterprises.

04. Main Work of the Alliance

1. Content Generation Technology Research : Digital human standards submitted to ITU, virtual identity registration platform, and ongoing evaluation of digital‑human technologies.

2. Content Distribution and Intelligent Marketing Research :

Intelligent recommendation: Revised standards based on the 2021 Cyberspace Administration guidelines, focusing on reliability, transparency, data protection, responsibility, and inclusiveness.

Intelligent marketing: Development of standards for CRM, social CRM, marketing automation, and user data platforms.

3. Content Review Technology Research :

Evaluation system for technology and service providers, covering functional and performance testing of machine review, with plans for human‑review assessment.

Third‑party services: Training for human reviewers, monitoring platforms for illegal content, and shared databases of discredited individuals.

05. Content Review Standards and Evaluation Work

Key activities include:

Machine Review Evaluation: Functional and performance tests using over 400,000 multi‑modal samples (text, images, audio, video). Certifications have been issued to more than ten cloud service providers.

Platform Governance Maturity Assessment: Evaluates review制度, process execution, and capability across platforms, with sector‑specific assessments planned.

Human Review Capability Assessment: Defines requirements for system security, data protection, reviewer training, and operational procedures.

Q&A Session

Q1: How does the alliance support the Cyberspace Administration’s 2022 algorithm governance action? A1: The alliance’s work on content safety, recommendation algorithms, and deep‑fake governance aligns with the action’s focus, and we aim to assist enterprises in self‑inspection.

Q2: Are the 400,000 test samples all text? A2: No, they include images, audio, and video as well.

Q3: Is content review limited to the internet sector? A3: Our standards are industry‑agnostic; future assessments will target specific sectors such as finance and e‑commerce.

Q4: How is the evaluation data collected? A4: Data are gathered from the internet based on regulatory‑defined violation categories, supplemented by samples provided by participating enterprises.

Q5: Is there a formal mechanism with regulators? A5: While no formal mechanism exists, our standards and evaluations overlap with regulator priorities, helping enterprises mitigate compliance risks.

Thank you for attending the session.