This article examines how large pretrained BERT models can be compressed via knowledge distillation into a lightweight TextCNN classifier for efficient garbage opinion detection, detailing traditional distillation methods, several practical schemes, experimental results, and the advantages of the approach.
This article presents a comprehensive overview of automatic speech recognition (ASR) error correction techniques employed by Xiaomi's Xiao‑Ai team, detailing problem definition, related work on BERT and ELECTRA, a custom generator‑discriminator architecture with a fuzzy‑phoneme simulator, experimental results, and prospective research directions.
The Meituan Search team boosted ranking relevance by training a domain‑specific BERT, applying data augmentation, brand‑sample optimization, knowledge‑graph fusion, multi‑task and pairwise fine‑tuning, joint end‑to‑end training with LambdaLoss ranking models, and compressing the model for low‑latency inference, delivering up to +925 BP offline accuracy gains and measurable CTR and NDCG improvements in production.
This article shares the Xiaomi AI Lab NLP team's experience in the NLPCC lightweight language model competition, discusses efficiency challenges of large pretrained models like BERT, and details practical inference optimizations—including model distillation, batching, FP16 quantization, and FasterTransformer integration—that dramatically reduce latency and hardware costs in production.
This article investigates the practical performance of Google’s 2019 Unsupervised Data Augmentation (UDA) framework on real‑world financial text classification tasks, detailing experiments with limited labeled data, domain‑out‑of‑distribution samples, noisy labels, and comparisons between BERT and lightweight TextCNN models.
The article explains how the GEAR model uses graph neural networks and BERT representations to aggregate multiple pieces of evidence for fact verification, improving accuracy on datasets like FEVER and offering applications in misinformation detection, knowledge‑graph completion, and advertising analytics.
This article introduces FashionBERT, a multimodal BERT‑based model that replaces ROI‑based image tokens with uniform image patches to overcome e‑commerce specific challenges, details its architecture, adaptive loss balancing, deployment in Alibaba search, and reports significant performance gains on public and internal datasets.
This article describes the design, model selection, experimental evaluation, and engineering optimization of an AI‑driven post‑sale question‑answering system for hotel services, covering FAQ construction, intent detection, deep‑learning matching models such as DSSM, ESIM, BERT, and their performance and latency trade‑offs.
This article presents a comprehensive overview of automated techniques for generating high‑quality search engine advertising keywords, covering background, traditional manual methods, intelligent keyword expansion using NLP, segmentation, POS tagging, BILSTM‑CRF, BERT classification, semantic matching with DSSM, and additional approaches such as query suggestion and synonym rewriting.
Meituan’s Search & NLP team, together with two universities, won the WSDM Cup 2020 Citation Intent Recognition task by building a multimodal retrieval‑ranking pipeline that merges semantic, spatial and axiomatic recall models with pairwise BERT and LightGBM ranking, achieving the highest MAP@3 and now powering Meituan’s QA, FAQ and core search systems.
This article describes iQIYI's short‑video content tagging system, outlining the challenges of extracting type and abstract tags from multimodal data, detailing the evolution from text‑only models to image‑fusion, BERT‑enhanced, and video‑frame models, and discussing their applications and future directions.
This article proposes a method to identify low-quality text in business data by training a large-scale unsupervised language model to compute sentence perplexity, converting the detection problem into a threshold decision, and details model design, challenges, optimizations, and online performance results.
This article examines the challenges of short‑video content tagging and describes iQIYI's multi‑stage evolution from simple text‑only models to sophisticated multimodal architectures that fuse cover images, BERT embeddings, and video frames to improve tag generation accuracy.
iQIYI’s short‑video tagging system tackles multimodal fusion, open‑set and abstract tags by evolving from a text‑only model through cover‑image, BERT‑vector, and video‑frame fusion architectures, enabling automated labeling, personalized recommendation, and semantic search while planning to add OCR, audio, and knowledge‑graph enhancements.
This article describes how a BERT‑based model is fine‑tuned to compute sentence‑pair similarity for improving recommendation accuracy in an online school, detailing the architecture, training mechanisms, code implementation, experimental results, and future extensions such as sentiment analysis.
This article reviews the specific NLP difficulties encountered in Sina Weibo's feed—such as short text, informal language, and ambiguous user behavior—and details the multi‑stage tagging system, material library, multimodal modeling, multi‑task learning, and large‑scale pre‑training techniques used to address them.
The article discusses how to discover hot topics using algorithms such as TextRank, BERT embeddings, and BM25, outlines the lifecycle of a hot topic, and proposes practical methods for evaluating article deduplication accuracy and recall when no labeled test set is available.
This article reviews the evolution of language models from bag‑of‑words to BERT, compares ELMo, GPT, and BERT architectures, and details how NetEase Yanxuan applies pre‑trained models to classification, text matching, sequence labeling, and generative tasks in production.
Amap's POI naming automation uses a two-stage cascade model: Stage 1 extracts token and sentence features with POS tags and domain-adapted BERT‑POI; Stage 2 employs a Bi‑LSTM to model line relationships, achieving over 95% semantic accuracy and 3‑6% recall improvements, thereby enhancing automatic signboard‑based POI name generation.
This article describes the development of MT‑BERT, a BERT‑based language model pre‑trained on Meituan‑Dianping business data, its distributed mixed‑precision training pipeline, domain adaptation, knowledge‑graph integration, model compression techniques, and the wide range of downstream NLP applications achieved in the platform.
The article reviews the CCL “Chinese Law Research Cup” similarity competition, describing the legal text dataset, converting the triple‑sample task to a binary similarity problem, outlining challenges such as long documents, and summarizing the BERT‑based Siamese, InferSent, and triplet‑loss models that achieved top‑10 results.
MT‑BERT at Meituan‑Dianping combines mixed‑precision, domain‑adapted continual pre‑training, knowledge‑graph‑aware masking, and extensive compression techniques to produce fast, accurate BERT models that power fine‑grained sentiment analysis, intent classification, recommendation reasoning, and other NLP tasks across the platform.
This article examines how sequence labeling methods such as pattern matching, CRF, and deep‑learning models like BiLSTM‑CRF and BERT are applied to automotive search tasks—including car‑series, model, and location/entity recognition—detailing their development, implementation challenges, and performance results.
This article presents a comprehensive solution to a competition that requires matching description paragraphs with the three most relevant papers from a 200,000‑paper corpus, detailing background, task definition, evaluation metrics, modeling strategy, and core algorithms such as SIF, InferSent, Bi‑LSTM, and BERT.
This article describes the design and implementation of an AI‑powered intelligent customer service system for airline ticket operations, covering data‑driven problem analysis, dialogue architecture, intent recognition using BERT and fastText, knowledge‑base QA, and future development plans.
This article presents a comprehensive overview of Alibaba's brand advertising business model, its technical architecture, and the practical application of NLP algorithms—specifically brand intent recognition and short‑text relevance—detailing model evolution, evaluation results, and future research directions.
The article surveys the evolution and core components of knowledge graphs—from early Linked Data concepts to modern semantic networks—detailing the end‑to‑end pipeline of data acquisition, cleaning, extraction, and fusion, and showcases Tencent Cloud’s Merak framework and encyclopedia KG, highlighting model choices, performance benchmarks, and real‑world applications such as recommendation and intelligent Q&A.
This article presents a comprehensive solution to the absolute semantic recognition competition, detailing the problem background, dataset, evaluation metrics, feature engineering, model architecture—including Attention, Capsule, Bi‑GRU, and BERT—and analysis of results and lessons learned.
The talk presents Didi AI Labs' research on intelligent dialogue robots for ride‑hailing, covering single‑turn QA, multi‑turn conversation, multi‑task learning architectures, model experiments, active learning pipelines, and the overall system design that integrates intent detection, slot extraction, dialogue management, and response generation.
The article details a multi‑turn dialogue intent classification pipeline that extracts and expands labeled utterances, preprocesses text with custom tokenization, trains a two‑layer CNN‑Highway and a multi‑head self‑attention model, analyzes errors, and achieves up to 98.7% accuracy on a large, balanced dataset.
The JDATA Absolute Semantic Recognition Challenge concluded on July 15 with eight finalist teams presenting their models, the champion "Label Team" using BERT, judges highlighting data cleaning, model fusion, and the distinction between absolute and contextual words, and a detailed ranking of all participating groups.
This article details Alibaba's technical challenges with BERT's high resource consumption in online search, analyzes memory and compute bottlenecks using TensorFlow profiling, and presents both TensorFlow‑based tweaks and a custom CUDA implementation that together double throughput and cut latency by about 50%.
This article explains the architecture of 360's search advertising system, detailing its ad recall, ranking, and display modules, illustrates exact‑match and semantic recall methods with a case study, and reviews the evolution from feature‑engineered GBDT models to deep learning approaches such as DSSM, ESIM, and BERT, including data preparation, training, and performance evaluation.
Alibaba's AI model topped the MS MARCO reading comprehension challenge, achieving the highest scores in document ranking and open‑domain question answering, even surpassing human performance, thanks to its deep‑cascade BERT‑based architecture that mimics human reading and is already deployed in e‑commerce applications.
This article examines XLNet, contrasting it with BERT by detailing its novel permutation language modeling, dual‑stream attention, and larger pre‑training data, and analyzes experimental results that show XLNet’s superior performance on reading‑comprehension, GLUE, and other NLP tasks, especially for long documents.
This article surveys the rapid proliferation of BERT-based research over the past six months, analyzing its impact on various NLP tasks such as question answering, information retrieval, dialog systems, summarization, data augmentation, classification, and sequence labeling, while also discussing the model's strengths, limitations, and future research opportunities.
This article reviews the historical development of natural language processing language models, covering expert rule‑based systems, statistical n‑grams, smoothing techniques, neural network models such as NNLM, RNN, word2vec, GloVe, ELMo, and the transformer‑based breakthroughs of GPT, BERT and GPT‑2, and summarizes their impact on modern NLP tasks.
This article reviews modern deep‑learning methods for Chinese word segmentation, comparing traditional CRF‑based approaches with BiLSTM‑CRF and BERT models, describing their architectures, training procedures, experimental results, and practical considerations for deployment.
This article reviews the taxonomy of transfer learning in NLP, summarizes representative pre‑training models such as ELMo, ULMFiT, BERT, GPT, MASS and UNILM, discusses their strengths and limitations, and provides practical recommendations for applying these techniques in real‑world projects.
This article walks through the development of deep learning for natural language processing, starting with basic neural cells and shallow networks, then exploring CNNs, RNNs, LSTMs, TextCNN, ESIM, ELMo, and culminating with the Transformer‑based BERT model, its training objectives, fine‑tuning strategies, and performance comparisons.
The article details the Travel team's award‑winning solution for the WSDM Cup 2019 fake‑news detection task, describing data analysis, preprocessing, label‑propagation augmentation, a BERT‑based baseline, a three‑stage multi‑level model‑fusion framework, experimental results, and future directions.
Natural language processing, a cornerstone of AI, relies on language models to capture linguistic features; this article reviews classic pre‑training models—ELMo, GPT, and BERT—explaining their architectures, training objectives, and how they boost downstream NLP tasks despite data‑scarcity challenges.
This article surveys the evolution of object detection models—comparing one‑stage and two‑stage approaches, their performance trade‑offs, and recent state‑of‑the‑art methods—while also outlining key concepts and breakthroughs in natural language processing, highlighting the impact of deep‑learning models such as BERT.
This article provides an in‑depth overview of BERT, covering its NLP background, GLUE benchmark achievements, Transformer‑based architecture, pre‑training strategies (MLM and NSP), downstream fine‑tuning methods, and includes detailed PyTorch code implementations of its core components.
This article recounts a Hulu NLP research engineer's experience, reviewing key milestones such as NNLM, Word2vec, Transformer and BERT, and then contrasting academic research with product development while illustrating real-world projects like news personalization and content embedding, and describing the supporting AI platform architecture.
In the WSDM Cup 2019 fake-news detection challenge, the Meituan Travel team secured second place by combining extensive data analysis, Chinese-English BERT fine-tuning, label-propagation augmentation, and a three-level fusion framework—blending, stacking, and linear regression—that lifted weighted accuracy to 0.88156.
Cheng Huige’s winning AI Challenger 2018 solution treated fine‑grained Chinese review sentiment as a 20‑aspect multi‑class task, combining a high‑capacity LSTM encoder with self‑attention, word‑and‑character embeddings, simplified ELMo pre‑training, diverse tokenizations and a weighted seven‑model ensemble (including BERT), which together delivered the competition’s top F1 performance.
This article explains BERT’s architecture—from its bidirectional Transformer encoder and attention mechanisms to its pre‑training tasks—and presents extensive experiments showing its superior performance on various Chinese and English text‑classification benchmarks across multiple datasets.
This article presents an overview of machine reading comprehension, covering the evolution from modular question‑answering systems to end‑to‑end neural models, discusses key datasets such as SQuAD and MS MARCO, and explores voice interaction technologies and future industry trends.
This article provides a step‑by‑step guide for fine‑tuning Google’s open‑source BERT on Chinese datasets, covering model download, processor customization, code examples, training commands, and insights into the underlying TensorFlow estimator architecture and deployment considerations.
