large-scale retrieval

Alimama Tech

Dec 14, 2023 · Artificial Intelligence

AI-Driven Content Risk Control: System Evolution and Optimization at Alibaba

Alibaba Mom’s AI‑driven content risk platform has evolved from simple rule‑matching to a data‑centric, serverless architecture that integrates large‑model acceleration, decision‑tree compilation, high‑throughput vector retrieval and elastic word‑matching, delivering sub‑100 ms text and sub‑1 s image moderation while remaining stable during peak promotional traffic.

AIDevOpscontent moderation

0 likes · 25 min read

AI-Driven Content Risk Control: System Evolution and Optimization at Alibaba

Baidu Geek Talk

Oct 31, 2023 · Artificial Intelligence

Interview on Baidu's Open‑Source Large‑Scale Vector Search Engine Puck

Baidu has open‑sourced its high‑performance, trillion‑scale vector search engine Puck—originally built for ultra‑large image‑search workloads, winner of multiple BIGANN categories, now supporting diverse embeddings alongside the medium‑size Tinker algorithm—to accelerate community innovation, improve code quality, and broaden AI retrieval applications across search, recommendation and cloud services.

AIANNBaidu

0 likes · 12 min read

Interview on Baidu's Open‑Source Large‑Scale Vector Search Engine Puck

High Availability Architecture

Apr 27, 2023 · Artificial Intelligence

Design and Optimization of Bilibili's Large‑Scale Video Duplicate Detection System

This article describes the design, algorithmic improvements, and engineering performance optimizations of Bilibili's massive video duplicate detection (collision) system, covering challenges of low‑edit‑degree reposts, two‑stage retrieval, self‑supervised feature extraction, GPU‑accelerated preprocessing, and the resulting gains in accuracy and throughput.

BilibiliFeature ExtractionVector Search

0 likes · 17 min read

Design and Optimization of Bilibili's Large‑Scale Video Duplicate Detection System

Bilibili Tech

Apr 21, 2023 · Artificial Intelligence

Design and Optimization of Bilibili's Large-Scale Video Duplicate Detection System

Bilibili built a massive video‑duplicate detection platform that trains a self‑supervised ResNet‑50 feature extractor, removes black borders, and uses a two‑stage ANN‑plus‑segment‑level matching pipeline accelerated by custom GPU decoding and inference, boosting duplicate rejection 7.5×, recall 3.75×, and cutting manual misses from 65 to 5 per day.

Feature ExtractionGPU Accelerationdeep learning

0 likes · 19 min read

Alimama Tech

Mar 14, 2023 · Artificial Intelligence

Neural Approximate Nearest Neighbor (NANN): Open‑Source Large‑Scale Retrieval with Arbitrary Complex Models

Alibaba’s open‑source Neural Approximate Nearest Neighbor (NANN) library decouples index learning from model training, enabling any TensorFlow‑based deep model to perform high‑throughput, high‑accuracy HNSW‑based retrieval with GPU multi‑streaming, XLA acceleration, graph optimizations, and adversarial training that mitigates L2‑distance mismatch, all supported by ready‑to‑use benchmarks and demos.

Neural ANNTensorFlowadversarial training

0 likes · 7 min read

Neural Approximate Nearest Neighbor (NANN): Open‑Source Large‑Scale Retrieval with Arbitrary Complex Models

Alimama Tech

Mar 23, 2022 · Artificial Intelligence

Advancements in Keyword Recall for Search Advertising: From Binary Retrieval to Hierarchical Bidding Graph

The paper reports a year‑long evolution of Alibaba’s search‑advertising keyword recall, replacing the traditional two‑stage rewrite‑and‑score pipeline with a low‑storage binary retrieval model and then a joint recall framework built on a hierarchical bidding graph, delivering near‑full‑precision recall, 16× memory savings, and quota‑free global ranking.

AISearch Advertisingbinary retrieval

0 likes · 22 min read

Advancements in Keyword Recall for Search Advertising: From Binary Retrieval to Hierarchical Bidding Graph

Alimama Tech

Dec 8, 2021 · Artificial Intelligence

Dual Vector Foil (DVF): Decoupled Index and Model Retrieval for Large-Scale Recall

The Dual Vector Foil (DVF) system decouples index construction from model training by building a post‑training HNSW graph, enabling any complex model to score candidates, which yields a 5.7 % recall boost, cuts latency from ~40 ms to 6.5 ms, and raises QPS over tenfold while simplifying maintenance.

IndexingOnline Inferencedeep learning

0 likes · 27 min read

Dual Vector Foil (DVF): Decoupled Index and Model Retrieval for Large-Scale Recall

Architecture Digest

Dec 8, 2021 · Cloud Native

Implementing Compute-Storage Separation for Large-Scale Retrieval Systems Using Fluid

This article describes the challenges of operating massive, TB‑scale retrieval clusters at Zuoyebang, and presents a Fluid‑based compute‑storage separation architecture that improves data distribution, update efficiency, scalability, and stability, enabling containerized search services to be managed like regular stateless workloads.

Data OrchestrationFluidKubernetes

0 likes · 13 min read

Implementing Compute-Storage Separation for Large-Scale Retrieval Systems Using Fluid

58 Tech

Mar 3, 2021 · Artificial Intelligence

Design and Implementation of a Faiss‑Based Vector Search Platform

The article describes the design, architecture, and key components of a vector search platform built on Faiss that supports full‑index construction, incremental and distributed indexing, online retrieval, city‑level search, and vector update/delete operations to meet large‑scale AI application needs.

AIKubernetesVector Search

0 likes · 10 min read

Design and Implementation of a Faiss‑Based Vector Search Platform

DataFunTalk

Dec 9, 2020 · Artificial Intelligence

WeChat Identify: From Object Detection to Large‑Scale Image Search – Technical Overview

This article details the evolution of WeChat’s Identify product, explaining its end‑to‑end image recognition pipeline—including object detection, multi‑label classification, mobile‑side detection, large‑scale retrieval, unsupervised clustering, and system architecture—while showcasing various application scenarios such as product, plant, and landmark recognition.

Image RecognitionWeChatcomputer vision

0 likes · 12 min read