This article explains an end‑to‑end semi‑supervised training pipeline for Transformer‑based NLP models, detailing the unsupervised language‑model pre‑training, supervised fine‑tuning, and the internal architecture of embeddings, encoder layers, and downstream tasks such as text classification and NER.
This article describes the motivation, micro‑service architecture, code implementation, V1.0 browser‑based NLP prototype, V2.0 AI‑enhanced version with BM25 and BERT, integration with ChatUI, DingTalk bot, command‑based automation, and future plans for the internal intelligent QA chatbot named Jarvis.
This article explains how JD's intelligent recommendation selling point system leverages NLP, BERT, Transformer and pointer‑generator models to automatically create short, personalized product highlights, describing the technical background, system architecture, model training pipeline, online/offline monitoring, and the resulting business benefits.
The article reviews the history and current state of NLP, compares symbolic deep‑parsing and neural pre‑trained approaches, discusses the knowledge‑bottleneck and low‑code trend, and illustrates semi‑automated, low‑code NLP deployment in the financial domain while pondering future integration of symbolic and neural methods.
This talk introduces graph deep learning techniques for natural language processing, covering the motivation for graph representations, traditional graph-based NLP methods, fundamentals of graph neural networks, static and dynamic graph construction, representation learning, and showcases the open‑source Graph4NLP Python library with example applications.
Tencent’s Hunyuan AI large model shattered records by scoring 80.888 to claim first place on the CLUE benchmark, showcasing its advanced natural language processing, multimodal abilities, curriculum‑learning training approach, and real‑world deployments in WeChat Search and advertising.
This article explains how TSDAE, a transformer‑based denoising auto‑encoder, converts noisy job description sentences into robust vector embeddings, details its training process, loss function, dataset preparation, and demonstrates using FAISS for similarity search on the resulting Job2Vec representations.
In this interview, Zuoyebang’s NLP lead explains how the company built self‑developed platforms like IQC and FTP to automate text quality inspection and intelligent labeling, outlines their architecture, shares practical deep‑learning applications such as translation and grammar correction, and discusses future research directions in large‑scale multi‑label classification, few‑shot learning, and multimodal models.
This article proposes a dual‑mode detection solution for call‑recording quality inspection that combines rule‑based semantic similarity matching with BERT‑based sentence segmentation and RoBERTa multi‑label classification to achieve high accuracy, fast task adaptation, and strong generalization for customer‑service scenarios.
This article describes the end‑to‑end design, architecture, code implementation, and deployment steps for an internal intelligent QA chatbot named “Jarvis”, covering its V1.0 browser‑based prototype, V2.0 AI‑enhanced version, DingTalk integration, automation features, and future roadmap.
ArcCSE introduces an angular‑margin contrastive loss and both pairwise (dropout‑augmented) and triple‑wise (span‑masked) relationship modeling to self‑supervise text embeddings, yielding tighter decision boundaries, higher alignment and uniformity, and superior performance on unsupervised STS, SentEval, and Alibaba’s retrieval and recommendation systems.
This article surveys relation extraction, defining the task, categorizing its five main forms, and detailing key approaches such as entity position encoding, dependency‑tree methods like shortest dependency path and BRCNN, as well as distant supervision with multi‑instance learning and selective attention.
This article details the design, architecture, key technologies, and deployment of Xiaosi, an AI‑powered intelligent customer service chatbot built on the internal IM platform, highlighting its problem‑diagnosis accuracy, manpower savings, adapter mechanism, high‑availability backend, and practical use cases.
At SIGIR 2022, the authors present a constrained Seq2Tree model that transforms hierarchical label taxonomies into preorder sequences and applies dynamic‑dictionary decoding to ensure label consistency, achieving superior hierarchical text classification performance on benchmark datasets and real‑world deployment within Alibaba Entertainment’s AI Brain.
This article explores how to automatically discover new words in Chinese web novels by combining n‑gram statistics, pointwise mutual information, information entropy, and TF‑IDF filtering, presenting a practical, unsupervised pipeline that improves tokenization and search recall without manual labeling.
The 2021 Taobao Push project introduced an AI‑driven copy‑generation platform that combines template extraction and fine‑tuned Unilm models with diverse beam search, creating diverse, high‑quality push messages, cutting manual costs, and delivering a 10 % click‑through lift and higher material adoption.
This article explains how product researchers can tap into massive user comment data using NLP tools to identify pain points, sentiment, and opportunities, offering practical scenarios and tool recommendations for e‑commerce, offline retail, and app platforms.
The article explains how rapid internet growth created information overload, leading to concise summary services, and how recent AI advances—especially large language models like GPT‑3—enable platforms such as JD.com to automatically generate high‑quality, multimodal product copy that drives sales and supports diverse creative tasks.
The Tsinghua‑Meituan Digital Life Joint Research Institute’s Academic Salon on March 23 featured Associate Professor Liu Zhiyuan presenting the latest advances and ten key challenges in large‑model technologies, aiming to foster industry‑academia collaboration and drive innovation in representation learning, knowledge graphs, and social computing.
This article reviews recent developments in text classification and intent recognition, comparing industrial practices such as business‑coupled feature engineering with research trends like pretrained language models, and provides references and practical insights for building effective NLP solutions.
This article introduces the open‑source WantWords reverse‑dictionary project, explains its token‑based processing pipeline, walks through Python implementation and model invocation with Hugging Face’s Transformers, reviews NLP’s historical evolution, and shows how front‑end developers can quickly integrate NLP models into products.
The article analyzes a recent Kaggle essay‑segmentation competition, explains why standard BERT‑based models plateau, and shows how a two‑stage pipeline that combines coarse BERT filtering with a feature‑rich tree model and post‑processing scaling can push scores well beyond the 70‑point barrier.
This article reviews the seven most notable Python libraries and packages of 2018 for data scientists and AI practitioners, including AdaNet, TPOT, SHAP, Optimus, spaCy, Jupytext, and Chartify, with descriptions, installation commands, and usage examples.
This article breaks down the evolution of Kaggle’s Jigsaw toxic comment competitions and presents a three‑step solution—training on historic data, using a genetic algorithm to weight multi‑label predictions, and ensembling fifteen models—to achieve high‑accuracy zero‑shot text classification.
This article traces the history of deep‑learning pre‑training techniques, comparing the parallel developments in natural‑language processing and computer vision—from early word2vec and bag‑of‑words models through ELMo and BERT to recent transformer‑based vision models like iGPT, ViT, BEiT and MAE—highlighting key innovations, challenges, and the convergence of the two fields.
This article presents a comprehensive overview of OPPO's XiaoBu Assistant, detailing its research background, chat skill architecture, evolution from retrieval and rule‑based methods to generative models, industry model comparisons, decoding and ranking strategies, safety mechanisms, performance optimizations, and evaluation results.
This article shows how the compress‑fasttext Python library can shrink a 7 GB fastText word‑embedding model to about 21 MB—a 300‑fold reduction—while preserving almost the same accuracy on downstream NLP tasks, and explains the underlying compression techniques, usage examples, and evaluation results.
The paper proposes a simple yet effective method that fuses label embeddings into BERT, enhancing text‑classification performance without increasing computational cost, and validates the approach across six benchmark datasets, also exploring tf‑idf‑based label augmentation and the impact of using [SEP] versus no‑[SEP] inputs.
This guide introduces BertViz, an interactive visualization tool for transformer models such as BERT, GPT‑2 and T5, explains how to install it via pip along with required dependencies, and demonstrates head, model, and neuron view visualizations with code examples in Jupyter.
The Beike Real Estate NLP team secured first place in the CCIR Cup 2021 Intelligent Human‑Computer Interaction track by applying semi‑supervised and transfer learning techniques to small‑sample intent recognition and slot filling, and also presented the large‑scale Mandarin dialect speech benchmark KeSpeech at NeurIPS 2021.
JD Explore’s Vega v1 model topped the GLUE benchmark with a 91.3 average score, outperforming Microsoft, Facebook, and Stanford across multiple NLP tasks, including first‑ever human‑level results on sentiment analysis and coreference, showcasing JD’s leading position in deep‑learning research.
This article reviews Baidu's knowledge‑enhanced semantic understanding models, detailing the evolution from early semantic techniques to ERNIE 1.0, 2.0 and the large‑scale ERNIE 3.0, its architecture, training strategies, performance benchmarks, and real‑world applications across industry.
This article presents Ctrip's intelligent customer service system, detailing the evolution of semantic matching methods from traditional lexical models to deep learning approaches such as BERT and ESIM, and describing multi‑stage retrieval, multilingual transfer learning, and KBQA techniques for improving query understanding and response accuracy.
This tutorial walks through Chinese text tokenization with Jieba, optimizes the token list using stop‑words and part‑of‑speech filtering, visualises word frequencies, and applies SnowNLP to perform sentiment analysis on Weibo comments, complete with code examples and result charts.
This article traces the evolution of deep‑learning pre‑training techniques, starting with word2vec in NLP, moving through ELMo and BERT, then shifting to computer‑vision models such as iGPT, ViT, BEiT, and MAE, highlighting key innovations, challenges, and the convergence of NLP and CV paradigms.
This talk presents Alibaba DAMO Academy’s recent work on compressing large pretrained language models, covering task‑adaptive AdaBERT, data‑augmented L2A, and meta‑knowledge distillation Meta‑KD, describing their motivations, architectures, NAS‑based search, loss designs, and experimental results across multiple NLP tasks.
This article explains how to perform Chinese part-of-speech tagging using the jieba.posseg library in Python, including loading stop words, extracting article content via Newspaper3k, applying precise mode segmentation, filtering, and presenting results in a pandas DataFrame.
This article explains practical techniques for improving NLP model accuracy on massive corpora, covering challenges of multi‑field text, word‑embedding choices, a fasttext‑based regression demo with book‑review data, feature engineering tricks, and a comparison with tf‑idf + LASSO.
This article details Laiye Technology’s AI R&D team’s end‑to‑end approach to designing and optimizing a custom entity‑relationship annotation tool, covering data‑labeling challenges, shortcomings of Excel and off‑the‑shelf solutions, architectural requirements, line‑breaking and mark‑position algorithms, performance improvements, and real‑world results.
To enhance decision‑making for users and quality monitoring for merchants, Meituan’s to‑restaurant platform implements fine‑grained aspect‑based sentiment analysis that extracts dish, attribute, opinion and polarity tuples from reviews, employing both a BERT‑CRF pipeline and a joint Dual‑MRC model which raise F1 scores from 0.61 to 0.68, and are deployed in dashboards and review‑management tools, with future work targeting efficiency and broader four‑tuple extraction.
This article walks through a complete Python workflow that loads the 20 Newsgroups dataset, preprocesses the documents, vectorizes them with TF‑IDF, groups them using KMeans, reduces dimensions with PCA, and visualizes the resulting clusters, illustrating each step with code and plots.
This article explains how Shannon's information entropy and NLP are combined in Alibaba Cloud's ARMS intelligent noise reduction to quantify alert uncertainty, filter redundant notifications, and automatically prioritize critical incidents, offering a practical, self‑learning solution for modern monitoring environments.
The presentation explains how JD.com leverages large‑scale text mining and NLP techniques—including data cleaning, multi‑level labeling, sentiment classification with models such as TextCNN, RoBERTa, and USE—to transform unstructured customer feedback into actionable product insights across various e‑commerce scenarios.
This article presents a step‑by‑step methodology for diagnosing and fixing model prediction errors—especially bad cases—in NLP and search systems, covering sample bias, threshold selection, preprocessing, post‑processing, validation cycles, and guidance on when to replace the model.
This article introduces several pandemic‑related open‑source resources—including a nationwide COVID‑19 vaccine record lookup, a tracker for ineffective vaccine distribution, and Baidu's DDParser NLP tool—detailing their purpose, usage, and installation to help developers build better vaccine‑related applications.
Baidu’s Fast Editing tool automates video trimming by using NLP to recognize subtitles, tone markers and duplicate sentences, then aligns them with the timeline for one‑click removal, employing character, Levenshtein and cosine similarity algorithms within a three‑module architecture (Plugin, Window, Caption) and planning on‑device PaddlePaddle analysis to cut latency and cost.
The KuaiSearch research team introduced PERKS, a large‑scale Chinese pre‑trained language model that achieved an 80.618 score on the CLUE 1.1 language classification task, narrowing the gap to human annotation and demonstrating significant advances in multi‑stage training, model optimization, and real‑world search applications.
This article introduces the importance of pre‑training in natural language processing, reviews classic pre‑training models such as Skip‑thoughts, BERT, GPT‑2 and T5, presents the modular UER‑py framework and its Chinese resources, compares it with Huggingface Transformers, and outlines practical deployment steps in industry.
This article presents a comprehensive overview of Xiaomi's Xiao AI intelligent QA system, detailing its background, three core answering modules—knowledge‑graph QA, retrieval‑based FAQ, and reading‑comprehension—and the underlying methods such as template matching, cross‑domain semantic parsing, path‑based reasoning, semantic retrieval, and neural matching, while also discussing performance results and practical trade‑offs.
This article explores how table‑based question answering can be integrated into B2B intelligent customer service by converting table queries into entity‑attribute recognition and multi‑turn dialogue, comparing end‑to‑end NL2SQL and slot‑filling approaches, and presenting NetEase Qiyu's practical implementation with its benefits and use cases.
Meituan’s knowledge‑based question answering system tackles diverse, constraint‑rich, multi‑hop queries across pre‑sale, in‑sale and post‑sale scenarios by integrating fine‑grained query understanding, relation recognition, sub‑graph retrieval and answer ranking, employing optimized BERT models, pre‑training tasks, and domain‑specific enhancements to boost response speed, conversion rates, and benchmark performance, while acknowledging remaining challenges in long‑tail and complex queries.
Meituan’s NLP Center introduced the ASAP dataset—the largest real‑world Chinese attribute‑level sentiment corpus—to date, and the article traces the progression from document‑level regression models upgraded with MT‑BERT, through multi‑task attribute‑level ABSA and opinion‑triplet extraction, to scalable real‑time and batch services, while outlining future transfer‑learning and few‑shot research.
This article presents a comprehensive overview of constructing an industry‑chain knowledge graph for the financial sector, covering its theoretical background, architectural design, automated building pipeline, key NLP techniques, and practical applications such as visualization, IPO review, and investment analysis.
This article explains how Yuedu's TTS synthesis platform tackles the booming audiobook market by using AI‑driven text preprocessing, role graph construction, content structuring, emotion and effect recognition, and a streamlined post‑processing workflow to efficiently generate multi‑character, emotionally rich audio books at scale.
The 9th CCF Big Data & Computing Intelligence Contest partners with Gaode Map to launch a POI Name Generation challenge, requiring participants to fuse image, signboard detection, and OCR data to automatically produce accurate, fluent place names, with a ¥50,000 prize pool, weekly vouchers, and recruitment opportunities for global teams.
This presentation by Tencent senior researcher Li Chao outlines the background, typical challenges, and multi‑layer technical solutions for intelligent grading in education, covering AI‑driven classroom, homework, review, and exam scenarios, multimodal spell‑checking, essay evaluation, and adaptive learning pipelines.
PaddleNLP v2.1 introduces an industrial‑grade Taskflow for eight NLP scenarios, a three‑line few‑shot learning paradigm that boosts small‑sample performance, and a FasterTransformer‑based inference engine that delivers up to 28‑fold speedup for text generation, all backed by extensive model and algorithm integrations.
This article presents a comprehensive overview of AI‑driven intelligent grading technologies, covering background, typical educational challenges, multimodal NLP solutions for essay, spelling and grammar correction, adaptive learning, and related research, illustrating how deep learning and multimodal models improve automated assessment across K‑12 scenarios.
This article presents a practical study of extracting phone numbers from two‑speaker voice dialogues using Seq2Seq models—including LSTM, GRU with attention and feature fusion, and Transformer—detailing data characteristics, model architectures, training strategies, experimental results, and comparative analysis showing the GRU‑Attention approach achieving the best performance.
This presentation reviews the importance of entity‑relation extraction for knowledge‑graph construction, compares sentence‑level and complex contexts, and surveys joint extraction techniques—including sequence labeling, table filling, and seq2seq models—as well as document‑level graph‑based methods and future research directions.
This article explains the fundamentals of contrastive learning, reviews typical models such as SimCLR, MoCo, SwAV, BYOL, SimSiam and Barlow Twins, and demonstrates how these methods are applied to Weibo text and multimodal (text‑image) representation tasks like hashtag generation and image‑text matching.
This article reviews the background and evolution of pre‑training models in NLP, introduces classic models such as Skip‑thoughts, BERT, and T5, and details the modular UER‑py framework, its comparison with HuggingFace Transformers, available Chinese pre‑trained weights, and practical deployment workflows.
This article details the authors' approach to the 2021 Sohu Text Matching competition, describing the task definition, data splits, model architectures (cross‑encoder and bi‑encoder), pretrained language models used, various training tricks, ensemble strategies, and the resulting evaluation scores.
The article explains how Ctrip’s hotel customer‑service team uses the Snorkel weak‑supervision framework to generate large‑scale labeled data for training models that automatically produce structured event summaries, detailing the workflow, labeling functions, generative and discriminative model training, and performance improvements.
This article recounts the author's six‑month Text2SQL research experience, detailing how systematic paper reading, leveraging existing engineering solutions, and fully utilizing academic, human, and hardware resources led to a successful thesis, a patent, a paper, and a second‑place finish in Yale's global Text2SQL competition.
Meituan’s NLP team leverages few‑shot learning—using data‑augmentation, semi‑supervised, ensemble/self‑training, and domain‑adaptation techniques—to cut annotation costs, achieving 1–2 percentage‑point accuracy gains on internal benchmarks and deploying high‑performing models for tasks such as topic classification, fake‑review detection, and sentiment analysis, while planning broader platform and model extensions.
Meituan’s 2021 ACL contributions comprise seven accepted papers—six long and one short—introducing novel approaches to event argument decoding, cross‑domain slot transfer, contrastive out‑of‑domain detection, novel slot discovery, self‑supervised sentence representation, unsupervised semantic parsing, and pseudo‑query‑enhanced dense retrieval, inviting further research and collaboration.
This article presents a comprehensive technical summary of the 2021 Sohu Campus Text Matching Algorithm Competition, detailing data characteristics, preprocessing strategies, tokenization choices, positional encoding methods, model architectures using relative encodings such as WoBERT and RoFormer, experimental results, and reflections on future improvements.
The article explains how Tencent addresses the limitations of traditional content understanding methods in personalized recommendation by introducing an interest‑graph framework that combines classification, concept, entity, and event layers, and details the associated mining, matching, and online evaluation techniques.
This article provides a comprehensive overview of natural language processing fundamentals, covering the challenges of language modeling, N‑gram and Markov assumptions, smoothing techniques such as discounting and add‑one, evaluation via perplexity, basic tasks like Chinese word segmentation, subword tokenization, POS tagging, syntactic and semantic parsing, and a range of downstream applications including information extraction, sentiment analysis, question answering, machine translation, and dialogue systems.
At iQIYI’s iTech Conference, Zhang Zhigang detailed a full‑stack search ranking system that combines NLP‑driven query analysis, hierarchical indexing, multi‑stage coarse‑to‑fine ranking, Transformer‑based re‑ranking, sparse‑feature DNN enhancements and LIME/SE‑Block explainability, delivering measurable gains in CTR and NDCG for the platform’s video search.
This article presents the background, problem analysis, data preprocessing, modeling approaches and optimization results of applying various NLP methods—including statistical models, word embeddings, attention mechanisms and pretrained language models such as BERT—to improve the accuracy of classifying Ctrip ticket customer service dialogues.
This article reviews the fundamentals of entity‑relation extraction, surveys joint extraction techniques such as sequence labeling, table‑filling and seq2seq models, discusses document‑level graph‑based methods, highlights experimental findings, and outlines future research directions in knowledge‑graph construction.
The Local Life Comprehensive Demand Knowledge Graph (GENE) reorients Meituan’s supply‑demand matching by building a multi‑layer, user‑centric graph that captures intent and consideration, employing BERT, Word2Vec, ELECTRA, and reinforcement‑learning models to generate concrete and scene‑based demand nodes, now powering parent‑child, leisure, medical‑beauty, and education services.
This article describes how Douyin built an AI‑powered feedback management platform that uses NLP to automatically tag and cluster user comments, maps them to business scenarios, defines quantitative experience metrics, and creates a closed‑loop system for rapid problem discovery and product improvement.
Using Python, the author collected and visualized Douban and Weibo comments on the film “Chinese Doctor,” showing rating distributions, keyword word clouds, and fan sentiment, and compares the 2021 movie with the 2019 documentary to illustrate audience perception of the pandemic drama.
Baidu's ERNIE 3.0 introduces a 100‑billion‑parameter, knowledge‑graph‑augmented language model that breaks 54 Chinese NLP benchmarks, achieves human‑level performance on SuperGLUE, and demonstrates strong generation and zero‑shot capabilities, now available for public demo and research.
This article describes how Dingxiangyuan's algorithm team adopted Milvus for distributed vector indexing to improve semantic search in their community forum, detailing the background, retrieval workflow, various embedding models—including Bi‑Encoder, Spherical Embedding, and Knowledge Embedding—and summarizing the benefits and future applications.
This comprehensive survey reviews the latest research on graph neural networks applied to natural language processing, covering graph construction methods, graph representation learning techniques, encoder‑decoder models, static and dynamic graph building, and discusses challenges, benchmarks, and future directions in the field.
This article describes how 58.com built an end‑to‑end bidding service that crawls tender documents, classifies them into multiple categories with BERT‑based models (including softmax, sigmoid and ensemble approaches) and extracts key entities using BERT‑CRF and reading‑comprehension techniques, achieving over 90% overall accuracy and dramatically improving recall and precision.
This tutorial walks you through installing the jieba Python library, explains its three segmentation modes—precise, full, and search—demonstrates how to add or delete words, manage custom dictionaries, handle stop words, perform weight analysis, adjust word frequencies, and retrieve token positions, all with clear code examples and visual output.
This article presents seven practical Python scripts—from a concise web scraper for Zhihu images and a chatbot conversation loop to a Naive Bayes poem author classifier, a lottery number generator, an automated essay writer, a screen‑capture tool, and a GIF creator—demonstrating how to avoid reinventing the wheel while exploring diverse automation tasks.
This article details the design, iterative development, and performance optimizations of 58's AI‑driven intelligent Q&A chatbot, covering its overall three‑layer architecture, the QABot, TaskBot, and answer‑recommendation modules, as well as dynamic strategy adjustment, caching mechanisms, and real‑world deployment results.
ConSERT is a contrastive self‑supervised framework that fine‑tunes BERT with augmented sentence views and NT‑Xent loss to overcome embedding collapse, delivering roughly 8% higher STS performance than prior methods, remaining robust in few‑shot and supervised scenarios, and now deployed in Meituan’s NLP pipelines.
This article explains the Transformer architecture—its self‑attention core, multi‑head attention, positional encoding, encoder‑decoder structure, and how it overcomes RNN limitations, providing a foundation for its use in NLP, image detection, and OCR.
The paper outlines an end‑to‑end pipeline that standardizes highly personalized food‑delivery dish names by combining rule‑based and BERT‑DSSM text synonym detection with EfficientNet image classification, constructing a multi‑level taxonomy that improves aggregation, supply‑demand analysis, recall ranking and merchant tagging.
This article introduces a curated selection of lesser‑known Python libraries for data‑science tasks—including data acquisition, date‑time handling, imbalanced‑learning, fast keyword extraction, fuzzy string matching, time‑series modeling, 3‑D visualization, web‑app building, and reinforcement‑learning—providing installation commands and concise usage examples.
Automatic homework grading leverages AI to transform captured images into graded results through preprocessing, layout analysis, OCR, answer matching, and strategy modules, while addressing three question categories—logical, text‑rich, and graphic—each presenting distinct technical challenges and future research directions.
The CKBQA task introduced at CCKS2021 challenges participants to build Chinese knowledge‑graph question answering systems that combine Meituan’s life‑services KG with open‑domain data from PKUBASE, requiring accurate, efficient and explainable handling of both domain‑specific and open‑domain queries to advance real‑world applications such as smart customer service and e‑commerce.
This article explores the landscape of text matching in NLP, covering problem types, three model interaction levels, sentence embedding techniques, supervised and unsupervised approaches, and the role of contrastive learning with alignment and uniformity metrics.
This article presents a comprehensive overview of Meituan's query understanding system, detailing how entity recognition, entity linking, query rewriting, and intent detection are designed and deployed to improve O2O search relevance across diverse local services.
This article presents a collection of Python 3.6.4 scripts that demonstrate how to scrape Zhihu images, converse with chatbots, classify Tang poems using NLP, generate random lottery numbers, automatically compose apology letters, capture screen images, and create animated GIFs, providing practical code examples for each task.
This article explains the BERT model by first reviewing the Encoder-Decoder framework, then detailing the attention mechanism—including self-attention and multi-head attention—before describing the full Transformer architecture and finally outlining BERT’s encoder-only design, training stages, and fine-tuning applications.
The article describes how NetEase adapts a pre‑trained BERT model to classify news articles into ultra‑short, short, or long timeliness categories by combining rule‑based strong and weak time cues, key‑sentence extraction, domain‑embedding fusion and multi‑layer semantic aggregation, achieving accurate and interpretable predictions for its platform.
This article reviews recent advances in abstractive summarization, covering pointer‑generator networks with coverage loss, integration of entity knowledge, strategies to mitigate repetition such as unlikelihood training and nucleus sampling, and emerging non‑autoregressive approaches like the Levenshtein Transformer.
The article reviews the Chinese NLP pre‑training model generalization competition, detailing data preprocessing, augmentation, external data usage, model scaling and architecture tweaks, loss functions, learning‑rate and adversarial training strategies, regularization techniques, post‑processing optimizations, and ineffective methods, highlighting their impact on performance metrics.
Youku’s new text‑video alignment system automatically generates short video summaries by extracting multimodal video and linguistic features, matching sentences to clips through embedding and tag‑level models, and enabling AI‑driven auto‑editing that cuts production time from days to minutes.
This article describes how a team at Longquan Temple leverages deep‑learning, NLP, and OCR technologies to automatically punctuate, recognize, and translate the massive Buddhist canon, achieving near‑human accuracy and dramatically accelerating scholarly work on ancient texts.
This article provides an in‑depth introduction to multi‑task learning (MTL), explaining its core concepts, why it is widely used in recommendation systems, NLP, CV and reinforcement learning, and offering guidance on model architectures, loss design, auxiliary tasks, and practical deployment tips.
This article presents a comprehensive study on generating recruitment job titles by combining a rule‑based pipeline with advanced seq2seq models—including BiLSTM‑Attention, Pointer‑Generator, and a Field‑Gate Dual‑Attention architecture—demonstrating significant performance gains on real‑world hiring data.
