Tagged articles
offline processing
12 articles · Page 1 of 1
This article provides a comprehensive guide to offline document parsing for Retrieval‑Augmented Generation, covering multi‑format extraction, layout analysis, OCR pitfalls, chunking strategies, hierarchical metadata tagging, and how these steps directly affect retrieval accuracy and overall RAG performance.
RAGdocument parsingmetadata
0 likes · 14 min read

This article details the design of a community‑search pipeline that leverages long‑term user interaction sequences for CTR/CVR prediction, describes the global, online and offline architectures, enumerates the major performance and consistency challenges encountered, and presents the practical optimizations and future directions adopted to achieve reliable, high‑throughput sequence modeling.
Data ConsistencySequence Modelingai-optimization
0 likes · 12 min read

Taobao’s AI virtual‑try‑on system pre‑computes fitting results offline, writes them into the Item Center via scalable ScheduleX tasks, optimizes pagination, locking and flow‑control, and thereby processes millions of apparel items in under thirty minutes with 99.9% success and reliable checkpoint‑resume monitoring.
AITask SchedulingVirtual Try-On
0 likes · 16 min read

The paper presents Baidu’s Mobile Operations Promotion Platform redesign, introducing an online‑offline integrated task‑scheduling architecture that partitions settlement fields to the data‑warehouse, records all jobs in a unified MySQL operation table, orchestrates them via Turing Data Studio, and manages dependencies to achieve consistent, auditable, billion‑scale settlement processing.
BaiduTask Schedulingdata warehouse
0 likes · 14 min read

This article presents iQIYI’s comprehensive data link governance practice, covering the motivations, offline and real‑time pipeline governance strategies, monitoring mechanisms, data lineage, and exploratory work such as intelligent attribution and field‑level lineage to improve data accuracy, timeliness, and reliability.
data governancedata lineageiQIYI
0 likes · 11 min read

Youzan Data Report Center monitors offline batch and real‑time data pipelines using accuracy and timeliness rules, cross‑table checks, upstream‑downstream comparisons, and scheduled alerts to detect anomalies early; since 2021 it has generated over 25 alerts, and plans a unified data‑quality dashboard.
FlinkHivebig data
0 likes · 12 min read


58 TechMay 31, 2021 · Artificial Intelligence
This article presents the design, training, deployment, and evaluation of a self‑developed Voice Activity Detection system used in both real‑time streaming dialogues and offline audio analysis at 58.com, detailing algorithm choices, smoothing strategies, engineering challenges, and future research directions.
AISpeech RecognitionVAD
0 likes · 18 min read

Within three weeks, the team built an interest‑based live‑stream recommendation platform for Xianyu that combined operational insights, BI analysis, and offline algorithms to generate user‑anchor interest tags, sync them to an online graph, and dramatically boost top‑room UV and click‑through rates.
big datagraph databaseinterest tagging
0 likes · 8 min read


Didi TechJul 24, 2020 · Artificial Intelligence
DLFlow, an end‑to‑end framework from Didi’s user‑profile team, merges Spark and TensorFlow to automate feature preprocessing, large‑scale distributed training, and massive prediction for big‑data offline tasks, offering configuration‑driven pipelines, task scheduling, and easy deployment that dramatically speeds model development.
SparkTensorFlowdeep learning
0 likes · 9 min read

This article presents a comprehensive offline big‑data processing framework that leverages scalable object storage for PB‑level data, details storage and compute engine requirements, compares cost options, describes data pipeline design, and showcases an e‑commerce case study with Spark‑driven analytics.
Cost OptimizationData EngineeringObject Storage
0 likes · 19 min read

This article details how Alibaba migrated its massive Taobao‑Tmall search workload to the search offline platform, tackling challenges of massive data volume, one‑to‑many joins, and hotspot sellers through a series of performance optimizations—including local joins, salt‑based data sharding, dynamic aggregation jobs, and asynchronous processing—to achieve high‑throughput full loads and low‑latency incremental updates.
AlibabaFlinkbig data
0 likes · 15 min read

To achieve millisecond‑level counting on billion‑scale data, the Xianyu team replaced slow MySQL count queries with an offline batch that snapshots relational tables and computes totals, then uses KV‑store incremental statistics for online updates, delivering sub‑10 ms responses with near‑100 % success.
big datadatabaseincremental counting
0 likes · 7 min read
