Design and Architecture of the ANNO AI Data Annotation Platform

Background AI algorithms are applied across iQIYI’s production lines, greatly improving efficiency and user experience. AI data serves as the foundation for algorithm development, and the demand for annotated data continues to grow with increasing manpower and algorithmic needs.

Previously, each algorithm team performed its own data annotation, leading to duplicated tool development, fragmented manpower management, and inconsistent quality control. To address these challenges, the ANNO platform was created to provide a unified solution for internal AI researchers, annotation managers, annotators, and external partners.

Challenges The platform must flexibly support a wide variety of annotation scenarios while remaining extensible. Without careful design, each new scenario would require ad‑hoc interface and workflow development, incurring high costs.

Key challenges include:

Diverse data types (text, image, audio, video, or mixed)

Complex data structures (e.g., n:n relationships between primary and reference materials)

Rich auxiliary information (metadata, per‑item label options)

Multiple annotation modes (classification, QA, segmentation, etc.)

Various business workflows (single upload, incremental batches, real‑time streams)

Model Design After studying algorithm teams’ needs and surveying existing products (e.g., JD Micro‑Work, Alibaba Data Annotation, Amazon Mechanical Turk), the following abstract data model was defined.

Annotation Media Record – the basic unit representing a multimedia file (text, image, audio, video). Example:

{
    "metadata": {"postId": "p123456"},
    "type": "text",
    "content": {
        "评论": "知道爱豆恋爱了，而且对方我十分不喜欢，你们都是怎么做的？想脱粉了。",
        "ocr文本": "竟成了一场笑话纯妃(纯惠皇)"
    }
}

Annotation Data (HIT) – a complete annotation task consisting of one or more media records. Example:

{
  "metadata": {"location": "chengdu", "device": "iPhone6"},
  "workSet": [
    {"type": "image", "url": "http://somePath/images/xxx.jpg", "height": 405, "width": 720},
    {"metadata": {"postId": "p123456"}, "type": "text", "content": {"评论": "...", "ocr文本": "..."}}
  ]
}

Partition – a collection of HITs, serving as the basic data source for an annotation task. Example:

{
  "description": "2019 Q1 Partition",
  "hits": [$hit1, $hit2, $hitN]
}

Hitset – a set of partitions sharing common attributes, analogous to a dataset.

Annotation Task Design The platform defines a Label Point as the atomic operation on data. Two fundamental actions are identified:

Select a subset P of a data set H (P ⊆ H).

Perform classification or QA on P.

Examples of label points include:

Classification label point (classify the whole HIT):

{
  "target": ".",
  "type": "question_choice",
  "question": "图片中人物是否与参照人物图一致？",
  "options": [{"name": "一致", "value": "yes"}, {"name": "不一致", "value": "no"}]
}

Image bounding‑box label point (draw a rectangle or polygon around a face and answer sub‑questions):

{
  "target": ".",
  "type": "lasso",
  "question": "请将图中的人脸框出来",
  "subLabelPoints": [
    {"type": "question_choice", "question": "是男是女？", "options": [{"name": "男", "value": "man"}, {"name": "女", "value": "female"}]},
    {"type": "question_text", "question": "颜值打几分（满分10分）？"}
  ]
}

Assignment records a complete annotation operation, including operator, timestamps, task IDs, and generated label answers. Example:

{
    "jobId": "xxxxxx",
    "hitId": "xxxxxx",
    "assignType": "LABEL",
    "operator": "christian",
    "labelAnswers": [{"target": "*", "type": "question_choice", "question": "图片类别", "values": ["porn"]}]
}

Frontend Modularity The abstract data model drives a modular front‑end architecture built with Vue.js. Components for media records, HITs, label points, and answers are composed dynamically to support any annotation scenario without ad‑hoc development.

AI Intelligent Annotation The platform integrates AI models that generate pre‑annotation answers, which annotators can adjust. This creates a feedback loop where corrected answers improve the models. An OCR‑based image annotation model reduces average labeling time from 13 s per image to 7 s (46 % efficiency gain) in a subtitle‑annotation use case.

Conclusion ANNO now supports a wide range of algorithmic annotation scenarios, provides robust manpower management, performance statistics, and data‑security controls. Future plans include opening the platform to more annotation teams, extending data collection, storage, and management services to further accelerate AI algorithm development.