BestHub
Sign in
Tagged articles

contrastive learning

87 articles · Page 1 of 1
AntTech
AntTech
Jun 30, 2026 · Artificial Intelligence

FinixDoc Tackles Hard Financial Document Parsing with a 4B Model and Open‑Source Benchmark

FinixDoc is an end‑to‑end financial document parsing system built on a 4‑billion‑parameter Qwen3‑VL model that outperforms open‑source baselines on the newly released FinixDocBench, handling low‑quality, complex and ultra‑large documents through a specialized training pipeline and evaluation matrix.

FinixDocBenchQwen3-VL-4Bcontrastive learning
0 likes · 11 min read
FinixDoc Tackles Hard Financial Document Parsing with a 4B Model and Open‑Source Benchmark
JD Retail Technology
JD Retail Technology
May 19, 2026 · Artificial Intelligence

Spectral Disentanglement and Enhancement: Teaching Multimodal Models to Denoise and Purify

The paper introduces the Spectral Disentanglement and Enhancement (SDE) framework, which uses singular value decomposition to separate strong semantic signals, weak auxiliary signals, and noise, applies curriculum‑based spectral enhancement, and jointly optimizes a dual‑domain contrastive loss, achieving markedly improved robustness and generalization on large‑scale multimodal benchmarks.

Multimodal Learningcontrastive learningdual-domain loss
0 likes · 14 min read
Spectral Disentanglement and Enhancement: Teaching Multimodal Models to Denoise and Purify
Data Party THU
Data Party THU
May 9, 2026 · Artificial Intelligence

NOSE: Enabling AI to Smell with a Unified Molecule‑Receptor‑Semantic Tri‑modal Representation

NOSE introduces a neural olfactory‑semantic embedding that unifies molecular structure, receptor sequences, and natural‑language odor descriptions into a continuous space, achieving state‑of‑the‑art results on eleven tasks and strong zero‑shot generalization for odor and receptor retrieval.

Deep Learningcontrastive learningmolecular design
0 likes · 8 min read
NOSE: Enabling AI to Smell with a Unified Molecule‑Receptor‑Semantic Tri‑modal Representation
AI Algorithm Path
AI Algorithm Path
Feb 17, 2026 · Artificial Intelligence

Why Contrastive Learning Is the Core Foundation of Visual Language Models

The article explains how contrastive learning replaces fixed‑category visual training with a relationship‑based approach, detailing the dual‑encoder architecture, cosine similarity loss, batch scaling, temperature control, zero‑shot capabilities, scalability from web data, and the method's strengths and limitations in modern multimodal AI.

CLIPMultimodal AIcontrastive learning
0 likes · 25 min read
Why Contrastive Learning Is the Core Foundation of Visual Language Models
AntTech
AntTech
Feb 5, 2026 · Artificial Intelligence

How Triple Alignment and Rationale Generation Supercharge Knowledge‑Based VQA

This paper presents a lightweight, high‑efficiency framework called Triple Alignment with Rationale Generation (TAG) that transforms knowledge‑based visual question answering into a contrastive learning task, dramatically reducing trainable parameters while achieving state‑of‑the‑art performance on major KVQA benchmarks.

CLIPMultimodalVQA
0 likes · 7 min read
How Triple Alignment and Rationale Generation Supercharge Knowledge‑Based VQA
PaperAgent
PaperAgent
Feb 3, 2026 · Artificial Intelligence

Relink: Turning GraphRAG into a Dynamic, Query‑Driven Knowledge Graph

Relink introduces a ‘reason‑and‑construct’ paradigm that builds knowledge‑graph paths during inference, combining a high‑precision factual graph with a high‑recall potential‑relation pool, using query‑driven dynamic path expansion and contrastive alignment to markedly improve multi‑hop QA performance and robustness to sparse knowledge.

Dynamic RetrievalGraphRAGLLM
0 likes · 8 min read
Relink: Turning GraphRAG into a Dynamic, Query‑Driven Knowledge Graph
PaperAgent
PaperAgent
Jan 27, 2026 · Artificial Intelligence

How Agentic‑R Boosts Multi‑Turn Retrieval for LLMs by 2–3 EM Points

This article analyzes the Agentic‑R framework, which upgrades traditional single‑hop Retrieval‑Augmented Generation by introducing dual‑perspective scoring and a bidirectional flywheel, resulting in 2–3 absolute EM improvements across seven QA datasets and a 10–15% reduction in search rounds.

Agentic SearchLLMRAG
0 likes · 6 min read
How Agentic‑R Boosts Multi‑Turn Retrieval for LLMs by 2–3 EM Points
DeWu Technology
DeWu Technology
Jan 21, 2026 · Artificial Intelligence

Breaking the Recommendation Feedback Loop with LLM‑Powered Dynamic User Knowledge Graphs

By integrating large language models to dynamically construct user knowledge graphs and applying two‑hop reasoning, the authors enhance serendipity in a large‑scale e‑commerce community recommendation system, achieving significant online gains in diversity, novelty, and user engagement metrics.

Industrial DeploymentLLMSerendipity
0 likes · 17 min read
Breaking the Recommendation Feedback Loop with LLM‑Powered Dynamic User Knowledge Graphs
PaperAgent
PaperAgent
Jan 13, 2026 · Artificial Intelligence

How C2LLM Redefines Code Retrieval with Attention‑Based Pooling

Introducing C2LLM, a contrastive code LLM series that replaces mean and EOS pooling with a multi‑head attention pooling module, achieving top scores on the MTEB‑Code benchmark across 12 tasks and demonstrating cost‑effective, high‑precision code retrieval for both production and AI agent applications.

Code EmbeddingMTEB-CodeRetrieval-Augmented Generation
0 likes · 8 min read
How C2LLM Redefines Code Retrieval with Attention‑Based Pooling
Kuaishou Tech
Kuaishou Tech
Dec 18, 2025 · Artificial Intelligence

How SSR Turns Multimodal Recommendation into an Interpretable Frequency‑Domain Reasoning Problem

The paper introduces SSR, a novel multimodal recommendation framework that leverages graph Fourier transforms, energy‑balanced frequency bands, structured regularization, and low‑rank tensor decomposition to replace black‑box fusion with explainable, adaptive reasoning, achieving state‑of‑the‑art results on Amazon datasets and strong cold‑start performance.

Graph Neural Networkscold-startcontrastive learning
0 likes · 15 min read
How SSR Turns Multimodal Recommendation into an Interpretable Frequency‑Domain Reasoning Problem
Data Party THU
Data Party THU
Nov 22, 2025 · Artificial Intelligence

How Frequency‑Refined Augmentation Boosts Contrastive Learning for Time‑Series Classification

FreRA introduces a lightweight, plug‑in frequency‑refined augmentation that adaptively refines spectral components to preserve global semantics while injecting variance, dramatically improving contrastive learning performance on time‑series classification, anomaly detection, and transfer learning across multiple benchmark datasets.

contrastive learningdata augmentationfrequency domain
0 likes · 13 min read
How Frequency‑Refined Augmentation Boosts Contrastive Learning for Time‑Series Classification
DataFunSummit
DataFunSummit
Oct 12, 2025 · Artificial Intelligence

How Baidu’s Generative Recall System (COBRA) Revolutionizes Ad Recommendations

This article details Baidu's generative recommendation ad recall framework, introducing the COBRA system and its three development stages—dense representation compression, sparse quantization with ID generation, and dense‑sparse cascading—highlighting coarse‑to‑fine inference, performance gains, long‑sequence extensions, online deployment, and future research directions.

COBRAGenerative AIRecommendation Systems
0 likes · 18 min read
How Baidu’s Generative Recall System (COBRA) Revolutionizes Ad Recommendations
AI Frontier Lectures
AI Frontier Lectures
Jul 10, 2025 · Artificial Intelligence

Can Dispersive Loss Supercharge Diffusion Models Without Extra Pre‑training?

Dispersive Loss is a plug‑and‑play regularization technique that enhances diffusion‑based generative models by encouraging dispersed internal representations, requiring no additional pre‑training, parameters, or data, and consistently improves performance across various model sizes and configurations, as demonstrated through extensive experiments.

Dispersive LossRegularizationcontrastive learning
0 likes · 18 min read
Can Dispersive Loss Supercharge Diffusion Models Without Extra Pre‑training?
AI Algorithm Path
AI Algorithm Path
Jul 5, 2025 · Artificial Intelligence

Beginner’s Guide to Vision‑Language Models Day 7: How CLIP Achieves Joint Visual‑Language Understanding

This article explains CLIP’s dual‑encoder architecture—using a Vision Transformer for images and a Transformer for text—how both encoders map inputs into a shared embedding space, the role of cosine similarity, and the InfoNCE contrastive loss that drives joint visual‑language learning.

CLIPInfoNCEMulti-modal Embedding
0 likes · 8 min read
Beginner’s Guide to Vision‑Language Models Day 7: How CLIP Achieves Joint Visual‑Language Understanding
AI Algorithm Path
AI Algorithm Path
Jun 23, 2025 · Artificial Intelligence

Visual Language Model Beginner’s Guide Day 4: Major Contrastive Learning Frameworks

This article surveys six leading contrastive learning frameworks—SimCLR, MoCo, BYOL, SwAV, Barlow Twins, and NNCLR—detailing their loss functions, data‑augmentation pipelines, encoder architectures, and unique mechanisms such as momentum queues, twin networks, clustering swaps, and redundancy reduction, while highlighting their advantages and impact on self‑supervised vision research.

BYOLBarlow TwinsMoCo
0 likes · 14 min read
Visual Language Model Beginner’s Guide Day 4: Major Contrastive Learning Frameworks
AI Algorithm Path
AI Algorithm Path
Jun 22, 2025 · Artificial Intelligence

Beginner’s Guide to Visual Language Models – Day 3: Contrastive Learning Loss Functions

This article systematically introduces the most common contrastive learning loss functions—including Contrastive Loss, Triplet Loss, N‑pair Loss, InfoNCE, and Cross‑Entropy—explaining their mathematical formulations, advantages, challenges, and typical applications in visual, textual, and multimodal representation learning.

InfoNCELoss Functionscontrastive learning
0 likes · 10 min read
Beginner’s Guide to Visual Language Models – Day 3: Contrastive Learning Loss Functions
AI Algorithm Path
AI Algorithm Path
Jun 20, 2025 · Artificial Intelligence

Beginner’s Guide to Visual Language Models – Day 2: Understanding Contrastive Learning

This article explains contrastive learning for visual language models, covering its definition, four‑step workflow, how to choose positive and negative pairs, the difference between supervised and self‑supervised variants, and why the technique is essential for zero‑shot and cross‑modal capabilities.

contrastive learningdata augmentationrepresentation learning
0 likes · 6 min read
Beginner’s Guide to Visual Language Models – Day 2: Understanding Contrastive Learning
AI Algorithm Path
AI Algorithm Path
Jun 20, 2025 · Artificial Intelligence

Beginner’s Guide to Visual Language Models – Day 1: What They Are and Why They Matter

This article introduces visual‑language models (VLMs), explaining how they combine large language models with visual encoders, why they overcome the rigidity of traditional computer‑vision systems, their key advantages, modular architecture, training methods, and practical applications such as image captioning and visual question answering.

AI ApplicationsLarge Language ModelsMultimodal AI
0 likes · 8 min read
Beginner’s Guide to Visual Language Models – Day 1: What They Are and Why They Matter
AI Algorithm Path
AI Algorithm Path
Jun 15, 2025 · Artificial Intelligence

Fine‑Tuning Text Embeddings for Domain‑Specific Search: A Complete Walkthrough

This article explains why generic text‑embedding models often fail in specialized retrieval tasks, then demonstrates how to fine‑tune such models using contrastive learning, curated job‑listing data, and the Sentence‑Transformers library, achieving near‑perfect accuracy on a job‑matching benchmark.

HuggingFaceSentence-Transformerscontrastive learning
0 likes · 11 min read
Fine‑Tuning Text Embeddings for Domain‑Specific Search: A Complete Walkthrough
Network Intelligence Research Center (NIRC)
Network Intelligence Research Center (NIRC)
May 14, 2025 · Artificial Intelligence

Hands‑On CLIP: Implementing Multimodal Vision‑Language Understanding

This article introduces OpenAI’s CLIP multimodal model, explains its architecture and contrastive training, details hardware and installation steps, and demonstrates a hands‑on zero‑shot image classification workflow that achieves 97% confidence on a cat image without any task‑specific fine‑tuning.

CLIPMultimodalPython
0 likes · 6 min read
Hands‑On CLIP: Implementing Multimodal Vision‑Language Understanding
Baidu Tech Salon
Baidu Tech Salon
Mar 21, 2025 · Artificial Intelligence

Semantic Embedding with Large Language Models: A Comprehensive Survey

This survey reviews the evolution of semantic embedding—from Word2vec and GloVe to BERT, Sentence‑BERT, and recent contrastive methods—then examines how large language models improve embeddings via synthetic data generation and backbone architectures, detailing techniques such as contrastive prompting, in‑context learning, knowledge distillation, and discussing resource, privacy, and interpretability challenges.

In-Context LearningInformation RetrievalNLP
0 likes · 27 min read
Semantic Embedding with Large Language Models: A Comprehensive Survey
Baidu Geek Talk
Baidu Geek Talk
Mar 12, 2025 · Artificial Intelligence

How LLMs Are Revolutionizing Semantic Embeddings: Models, Methods, and Trends

This article reviews how large language models (LLMs) enhance semantic text embeddings by comparing traditional methods with LLM‑based approaches, detailing synthetic data generation, backbone model designs, key model families, experimental results on the MTEB benchmark, and future research challenges.

LLMMultilingualcontrastive learning
0 likes · 30 min read
How LLMs Are Revolutionizing Semantic Embeddings: Models, Methods, and Trends
DeWu Technology
DeWu Technology
Feb 19, 2025 · Artificial Intelligence

Scenario-aware Multi-Scenario Recommendation Models: SACN, SAINet, and DSWIN

The paper presents a comprehensive multi‑scenario recommendation study introducing three models—SACN, SAINet, and DSWIN—that integrate scene‑aware attention, attribute‑level preferences, and contrastive disentanglement to capture distinct user interests, achieving consistent AUC gains and online CTR improvements across real‑world datasets.

CTR PredictionDeep Learningcontrastive learning
0 likes · 43 min read
Scenario-aware Multi-Scenario Recommendation Models: SACN, SAINet, and DSWIN
Xiaohongshu Tech REDtech
Xiaohongshu Tech REDtech
Dec 26, 2024 · Artificial Intelligence

Instruction Embedding: Latent Representations of Instructions for Task Identification

The paper introduces Instruction Embedding—a task‑focused text representation learned on the new Instruction Embedding Benchmark—and shows that Prompt‑based Instruction Embedding (PIE) outperforms standard embeddings in clustering, similarity, and downstream tasks such as data selection, in‑context example retrieval, test‑set compression, and task‑correlation analysis.

Large Language Modelscontrastive learningfine-tuning
0 likes · 15 min read
Instruction Embedding: Latent Representations of Instructions for Task Identification
Baobao Algorithm Notes
Baobao Algorithm Notes
Oct 24, 2024 · Artificial Intelligence

How NoteLLM-2 Boosts Multimodal Recommendations with In-Content Learning

NoteLLM-2 introduces multimodal In-Content Learning and Late Fusion to overcome visual‑modality bias in end‑to‑end fine‑tuned large representation models, delivering significant gains over baseline multimodal LLMs and traditional retrieval methods in recommendation tasks.

AI researchRecommendation Systemscontrastive learning
0 likes · 11 min read
How NoteLLM-2 Boosts Multimodal Recommendations with In-Content Learning
DataFunTalk
DataFunTalk
Aug 5, 2024 · Artificial Intelligence

Enhancing Taobao Display Advertising with Multimodal Representations: Challenges, Approaches, and Insights

This article presents a comprehensive study on integrating multimodal image‑text representations into large‑scale e‑commerce advertising CTR models, introducing a semantic‑aware contrastive pre‑training (SCL) method and two application algorithms (SimTier and MAKE) that together achieve over 1 % GAUC improvement and significant online gains.

CTR PredictionRecommendation Systemscontrastive learning
0 likes · 21 min read
Enhancing Taobao Display Advertising with Multimodal Representations: Challenges, Approaches, and Insights
Alimama Tech
Alimama Tech
Aug 2, 2024 · Artificial Intelligence

Multimodal Representations Boost Taobao Display Advertising CTR

Alibaba’s advertising team introduces semantic‑aware contrastive learning to pre‑train multimodal image‑text embeddings, integrates them via SimTier and MAKE into ID‑based CTR models, achieving up to 6.9% lift in Taobao display ad click‑through rates and improving long‑tail item performance.

CTR PredictionMultimodal LearningRecommendation Systems
0 likes · 21 min read
Multimodal Representations Boost Taobao Display Advertising CTR
NewBeeNLP
NewBeeNLP
Jun 20, 2024 · Artificial Intelligence

How LLMs Transform Recommendation Systems: Insights from Kuaishou’s LERAN Paper

This article analyzes Kuaishou’s May 2024 paper on LLM‑driven recommendation, detailing its dual‑tower architecture, contrastive learning of user and item embeddings, and a CVR‑auxiliary task that together improve cold‑start handling and boost both offline and online AUC metrics.

Industrial ApplicationItem EmbeddingLLM
0 likes · 10 min read
How LLMs Transform Recommendation Systems: Insights from Kuaishou’s LERAN Paper
Alibaba Cloud Big Data AI Platform
Alibaba Cloud Big Data AI Platform
May 29, 2024 · Artificial Intelligence

ContraLSP: Contrastive Sparse Perturbations Transform Time‑Series Explanation

Recent collaboration between Alibaba Cloud’s big‑data team and leading universities introduced ContraLSP, a novel contrastive and locally sparse perturbation framework that outperforms state‑of‑the‑art methods in explaining time‑series models, offering improved interpretability for both white‑box forecasting and black‑box classification tasks.

contrastive learninginterpretabilitymachine learning
0 likes · 8 min read
ContraLSP: Contrastive Sparse Perturbations Transform Time‑Series Explanation
Alibaba Cloud Big Data AI Platform
Alibaba Cloud Big Data AI Platform
Dec 11, 2023 · Artificial Intelligence

How Hyperbolic Space and Contrastive Learning Boost Domain-Specific Language Models

This article introduces the KANGAROO model, which injects hierarchical semantic information via hyperbolic space and leverages contrastive learning on dense subgraph structures to overcome global sparsity in vertical‑domain knowledge‑enhanced pre‑trained language models, and evaluates its performance on finance and medical tasks.

Domain AdaptationNLPcontrastive learning
0 likes · 10 min read
How Hyperbolic Space and Contrastive Learning Boost Domain-Specific Language Models
DataFunSummit
DataFunSummit
Dec 8, 2023 · Artificial Intelligence

Multimodal Cold‑Start Techniques for Music Recommendation at NetEase Cloud Music

This article presents NetEase Cloud Music's multimodal cold‑start solution, detailing the problem background, feature selection using CLIP, two modeling approaches (I2I2U indirect and U2I DSSM direct), contrastive learning enhancements, interest‑boundary modeling, and evaluation results showing significant gains in user engagement.

.aiMultimodalcold-start
0 likes · 15 min read
Multimodal Cold‑Start Techniques for Music Recommendation at NetEase Cloud Music
Alimama Tech
Alimama Tech
Nov 15, 2023 · Artificial Intelligence

Hybrid Contrastive Constraints for Multi-Scenario Ad Ranking (HC²)

The HC² framework enhances multi‑scenario ad ranking by jointly applying a generalized contrastive loss on shared representations and an individual contrastive loss on scenario‑specific layers, using label‑aware positive sampling, diffusion‑noise negative sampling, and inverse‑similarity weighting, achieving consistent offline gains and up to 2.5% CVR and 3.7% GMV improvements in Alibaba’s live system.

Recommendation Systemsad rankingcontrastive learning
0 likes · 16 min read
Hybrid Contrastive Constraints for Multi-Scenario Ad Ranking (HC²)
DataFunTalk
DataFunTalk
Nov 10, 2023 · Artificial Intelligence

Multimodal Cold-Start Techniques for Music Recommendation at NetEase Cloud Music

This article presents NetEase Cloud Music's multimodal cold-start recommendation approach, detailing the problem's significance, feature extraction using CLIP, I2I2U indirect modeling, U2I DSSM direct modeling with contrastive learning and interest‑boundary mechanisms, deployment pipeline, evaluation results, and future optimization directions.

Multimodal Learningcold-startcontrastive learning
0 likes · 14 min read
Multimodal Cold-Start Techniques for Music Recommendation at NetEase Cloud Music
NetEase Media Technology Team
NetEase Media Technology Team
Nov 6, 2023 · Artificial Intelligence

Overview of Sequential Recommendation Models

The article surveys sequential recommendation models from early non-deep approaches like FPMC, through RNN-based GRU4Rec and CNN-based Caser, to Transformer-based methods such as SASRec, BERT4Rec, TiSASRec, and recent contrastive-learning techniques, recommending SASRec or its variants for production use.

Deep LearningTransformercontrastive learning
0 likes · 17 min read
Overview of Sequential Recommendation Models
Alimama Tech
Alimama Tech
Nov 1, 2023 · Artificial Intelligence

BOMGraph: Boosting Multi-Scenario E-commerce Search with a Unified Graph Neural Network

BOMGraph introduces a unified heterogeneous graph neural network that jointly models text, image, and similar‑item search across multiple e‑commerce scenarios, using meta‑path‑guided attention, disentangled scenario‑specific and shared embeddings, and contrastive learning to alleviate sample sparsity, achieving consistent offline and online performance gains.

Graph Neural Networkcontrastive learninge-commerce
0 likes · 13 min read
BOMGraph: Boosting Multi-Scenario E-commerce Search with a Unified Graph Neural Network
Kuaishou Tech
Kuaishou Tech
Sep 26, 2023 · Artificial Intelligence

Cross-Domain Product Representation (COPE): A Large-Scale Dataset and Baseline Model for Rich‑Content E‑Commerce

The paper introduces ROPE, the first large‑scale cross‑domain product recognition dataset covering detail pages, short videos and live streams, and proposes COPE, a dual‑tower multimodal model that learns unified product embeddings using contrastive and classification losses, achieving superior retrieval and few‑shot classification performance across domains.

Deep Learningcontrastive learningcross-domain
0 likes · 13 min read
Cross-Domain Product Representation (COPE): A Large-Scale Dataset and Baseline Model for Rich‑Content E‑Commerce
Alimama Tech
Alimama Tech
Sep 12, 2023 · Artificial Intelligence

Content Collaborative Graph Neural Network for Large‑Scale E‑commerce Search

CC‑GNN addresses three drawbacks of existing graph‑neural retrieval for e‑commerce by adding content phrase nodes, scalable meta‑path message passing, and difficulty‑aware noisy contrastive learning with counterfactual augmentation, achieving up to 16 % recall improvement and notably larger gains on long‑tail queries and cold‑start items.

E-commerce SearchGraph Neural NetworksLong Tail
0 likes · 19 min read
Content Collaborative Graph Neural Network for Large‑Scale E‑commerce Search
Huolala Tech
Huolala Tech
Jul 28, 2023 · Artificial Intelligence

How HuoLala Leverages AI to Revolutionize Service Quality Inspection

This article details HuoLala's AI‑driven intelligent quality inspection system, covering its NLP‑based semantic understanding pipeline, data denoising, confidence learning, contrastive learning, model acceleration techniques such as pruning, knowledge distillation, quantization, and interpretability methods to improve coverage, recall and risk detection.

NLPcontrastive learningdata denoising
0 likes · 23 min read
How HuoLala Leverages AI to Revolutionize Service Quality Inspection
DataFunTalk
DataFunTalk
Jun 21, 2023 · Artificial Intelligence

Low‑Resource NLP Pretraining: Methodology, Experiments, and Zero‑Shot Applications

This article presents a low‑resource NLP pretraining approach that combines transformer‑based language modeling with contrastive vector learning, details the unsupervised sample‑pair construction, introduces a camel‑shaped masking distribution, and demonstrates through extensive experiments that the resulting model achieves strong zero‑shot NLU, NLG, and retrieval performance while requiring minimal compute and data.

Language Modelingcontrastive learninglow-resource
0 likes · 10 min read
Low‑Resource NLP Pretraining: Methodology, Experiments, and Zero‑Shot Applications
Network Intelligence Research Center (NIRC)
Network Intelligence Research Center (NIRC)
May 24, 2023 · Artificial Intelligence

COPNER: Contrastive Learning with Prompt Guidance for Few‑Shot Named Entity Recognition

The article introduces COPNER, a contrastive‑learning framework that uses class‑specific prompt words to guide sentence encoders, addressing the limited semantic capture of existing few‑shot NER methods and demonstrating superior performance across multiple benchmark datasets and K‑shot settings.

COPNERNLPcontrastive learning
0 likes · 4 min read
COPNER: Contrastive Learning with Prompt Guidance for Few‑Shot Named Entity Recognition
DataFunSummit
DataFunSummit
May 23, 2023 · Artificial Intelligence

Continuous Semantic Enhancement for Neural Machine Translation: Methodology, Experiments, and Community Deployment

This article introduces a continuous semantic enhancement approach for neural machine translation that overcomes the limitations of discrete data‑augmentation techniques, details the neighbor risk minimization training objective, presents benchmark improvements on ACL‑2022 datasets, and describes practical deployment and fine‑tuning workflows in the Modu community.

Neural Machine Translationcontinuous semantic augmentationcontrastive learning
0 likes · 19 min read
Continuous Semantic Enhancement for Neural Machine Translation: Methodology, Experiments, and Community Deployment
AntTech
AntTech
May 10, 2023 · Artificial Intelligence

Brainwave and Behavior Recognition: Multi‑Modal Biometric Authentication with Adversarial Contrastive Transfer Learning

This article presents Ant Security's research on novel biometric methods—brainwave (脑纹) and behavior recognition—detailing their scientific background, data collection, multi‑modal deep‑learning algorithms, adversarial and contrastive training strategies, experimental results, and practical applications for inclusive, secure identity verification.

Multimodal AIaccessibilityadversarial learning
0 likes · 17 min read
Brainwave and Behavior Recognition: Multi‑Modal Biometric Authentication with Adversarial Contrastive Transfer Learning
Kuaishou Tech
Kuaishou Tech
Apr 25, 2023 · Artificial Intelligence

DCCL: A Contrastive Learning Framework for Causal Representation Decoupling in Recommendation Systems

The paper introduces DCCL, a model‑agnostic contrastive learning framework that decouples user interest and conformity representations to address popularity bias and out‑of‑distribution challenges in recommendation systems, demonstrating significant offline and online performance gains on real‑world datasets.

OOD robustnesscausal inferencecontrastive learning
0 likes · 8 min read
DCCL: A Contrastive Learning Framework for Causal Representation Decoupling in Recommendation Systems
Alimama Tech
Alimama Tech
Dec 14, 2022 · Artificial Intelligence

Contrastive Image Representation Learning with Debiasing for CTR Prediction

The article proposes a three-stage contrastive learning framework—pre‑training, fine‑tuning, and debiasing—to generate unbiased, fine‑grained image embeddings for mobile Taobao CTR prediction, achieving higher accuracy, fairness, and a 4‑5% CTR lift in large‑scale offline and online evaluations.

CTR PredictionDeep Learningbias mitigation
0 likes · 14 min read
Contrastive Image Representation Learning with Debiasing for CTR Prediction
Alibaba Cloud Big Data AI Platform
Alibaba Cloud Big Data AI Platform
Dec 8, 2022 · Artificial Intelligence

KECP: Enhancing Few-Shot Machine Reading Comprehension via Knowledge-Driven Prompt Tuning

KECP, a Knowledge‑Enhanced Contrastive Prompt‑tuning model, achieves strong few‑shot extractive question answering by converting questions to masked statements, injecting external knowledge via gated fusion, and leveraging contrastive learning alongside masked language modeling, as demonstrated on EMNLP‑2022 benchmarks.

NLPcontrastive learningknowledge injection
0 likes · 9 min read
KECP: Enhancing Few-Shot Machine Reading Comprehension via Knowledge-Driven Prompt Tuning
Baidu Geek Talk
Baidu Geek Talk
Nov 16, 2022 · Artificial Intelligence

How Baidu’s Ernie‑SimCSE Uses Contrastive Learning to Crush Spam Promotion

This article explains how Baidu's anti‑spam team tackled large‑scale promotional spam on Baidu Zhidao by combining the Ernie pretrained model with SimCSE contrastive learning, detailing the problem background, traditional methods, text‑representation stages, the SimCSE approach, training pipeline, optimizations, and experimental results.

ERNIENLPPretrained Models
0 likes · 15 min read
How Baidu’s Ernie‑SimCSE Uses Contrastive Learning to Crush Spam Promotion
Alimama Tech
Alimama Tech
Nov 9, 2022 · Artificial Intelligence

Graph-based Weakly Supervised Framework for Semantic Relevance Learning in E-commerce

The paper introduces a graph‑based weakly supervised contrastive learning framework that uses heterogeneous user‑behavior graphs, e‑commerce‑specific augmentations, and a hybrid fine‑tuning/transfer learning strategy to improve semantic relevance matching between queries and product titles, achieving significant gains on a large‑scale Taobao dataset.

Graph Neural NetworksInformation RetrievalWeak Supervision
0 likes · 12 min read
Graph-based Weakly Supervised Framework for Semantic Relevance Learning in E-commerce
Zhuanzhuan Tech
Zhuanzhuan Tech
Oct 28, 2022 · Artificial Intelligence

Contrastive Learning: Definitions, Principles, Classic Algorithms, and Applications in Recommendation Systems

This article introduces contrastive learning, explains its definition, principles, and classic algorithms such as SimCLR and MoCo, and details its practical applications in recommendation systems, including a case study of its deployment at Zhuanzhuan that boosted order rates by over 10%.

.aicontrastive learningself-supervised learning
0 likes · 12 min read
Contrastive Learning: Definitions, Principles, Classic Algorithms, and Applications in Recommendation Systems
Youzan Coder
Youzan Coder
Oct 24, 2022 · Artificial Intelligence

Knowledge Base Retrieval Matching: Algorithm and Engineering Service Practice

The article outlines a comprehensive knowledge‑base retrieval matching solution—combining PageRank‑enhanced DSL rewriting, keyword and dual‑tower vector recall, contrastive fine‑ranking, and optimized vector‑based ranking—implemented via offline DP training and Sunfish online inference on Milvus, with applications in enterprise search and recommendations and future plans for graph‑neural embeddings.

InfoNCEMilvusNLP
0 likes · 12 min read
Knowledge Base Retrieval Matching: Algorithm and Engineering Service Practice
DataFunTalk
DataFunTalk
Oct 6, 2022 · Information Security

Graph Machine Learning for Security Risk Control: Architecture, Models, and Future Directions

This article presents a comprehensive overview of applying graph machine learning to security risk control, covering background cases, system architecture, dynamic heterogeneous graph modeling with HGT and DDGCL, experimental results, and future research directions for fraud, money‑laundering, and gambling detection.

Graph Neural Networkscontrastive learningdynamic heterogeneous graph
0 likes · 10 min read
Graph Machine Learning for Security Risk Control: Architecture, Models, and Future Directions
DataFunTalk
DataFunTalk
Sep 27, 2022 · Artificial Intelligence

Contrastive Learning for Text Generation: Motivation, Methodology, Experiments, and Discussion (CoNT Framework)

This article reviews the integration of contrastive learning into text generation, explains why it helps mitigate exposure bias, introduces the CoNT framework with three key improvements, presents extensive experiments on translation, summarization, code comment and data‑to‑text tasks, and discusses practical deployment considerations.

.aiCoNTMachine Translation
0 likes · 21 min read
Contrastive Learning for Text Generation: Motivation, Methodology, Experiments, and Discussion (CoNT Framework)
DataFunSummit
DataFunSummit
Sep 6, 2022 · Artificial Intelligence

Recent Advances in Self‑Supervised Learning for Text Recognition (OCR)

This article reviews recent progress in applying self‑supervised learning to OCR text recognition, covering mainstream model architectures, key considerations for self‑supervised tasks on text images, and detailed analyses of representative papers such as SeqCLR, SimAN, and DiG, highlighting their designs, experiments, and results.

OCRcomputer visioncontrastive learning
0 likes · 20 min read
Recent Advances in Self‑Supervised Learning for Text Recognition (OCR)
Laiye Technology Team
Laiye Technology Team
Aug 15, 2022 · Artificial Intelligence

Recent Advances in Self‑Supervised Learning for Text Recognition

This article reviews recent self‑supervised learning approaches for optical character recognition, covering mainstream OCR model architectures, key factors for applying contrastive and masked image modeling methods to text images, and detailed analyses of representative works such as SeqCLR, SimAN, and DiG, including their designs and experimental results.

OCRcontrastive learningmasked image modeling
0 likes · 19 min read
Recent Advances in Self‑Supervised Learning for Text Recognition
Youzan Coder
Youzan Coder
Jul 11, 2022 · Artificial Intelligence

How Contrastive Learning Revolutionizes Product Term Prediction in E‑commerce

By leveraging contrastive learning and large‑scale click‑through data, the article details a dual‑tower model that encodes product titles and queries, explains loss functions, batch‑negative sampling, distributed training tricks, and demonstrates how this approach outperforms traditional NER for product term and category prediction.

InfoNCEcontrastive learningdistributed training
0 likes · 16 min read
How Contrastive Learning Revolutionizes Product Term Prediction in E‑commerce
DataFunSummit
DataFunSummit
Jun 25, 2022 · Artificial Intelligence

Image and Text Pretraining: Methods, Practices, and Business Applications in Information Flow

This article reviews large‑scale image and multimodal pre‑training techniques—including contrastive learning, self‑supervised reconstruction, and multimodal alignment—explains data acquisition, model construction, evaluation metrics, and demonstrates how these methods are applied and optimized for real‑world information‑flow services.

.aiInformation Flowcontrastive learning
0 likes · 17 min read
Image and Text Pretraining: Methods, Practices, and Business Applications in Information Flow
Alimama Tech
Alimama Tech
Jun 15, 2022 · Artificial Intelligence

Multi-modal Multi-query Search Session Modeling with Heterogeneous Graph Neural Networks

The paper introduces MUVCOG, a heterogeneous graph neural network that models multi‑modal, multi‑query search sessions on Mobile Taobao by jointly learning attention‑based global and hierarchical local views through contrastive pre‑training, yielding universal session embeddings that markedly improve CTR prediction, query recommendation, and intent classification.

Graph Neural NetworkMulti-modalcontrastive learning
0 likes · 15 min read
Multi-modal Multi-query Search Session Modeling with Heterogeneous Graph Neural Networks
Alimama Tech
Alimama Tech
Jun 8, 2022 · Artificial Intelligence

CTR-Driven Advertising Text Generation and Bundle Creative Optimization (CREATER & CONNA)

Alibaba’s advertising team introduces CREATER, a CTR‑driven text generator that leverages user reviews, aspect control codes, and contrastive fine‑tuning, and CONNA, a non‑autoregressive bundle creator that predicts heterogeneous ad elements with set‑based loss, both delivering substantial online CTR gains and CPC reductions through dynamic creative optimization.

CTRDynamic creative optimizationNLP
0 likes · 25 min read
CTR-Driven Advertising Text Generation and Bundle Creative Optimization (CREATER & CONNA)
DaTaobao Tech
DaTaobao Tech
May 31, 2022 · Artificial Intelligence

Decoupling Popularity Bias in Dual‑Tower Retrieval Models

The paper proposes CDAN, a dual‑tower retrieval model that separates item attribute and popularity representations via a Feature Decoupling Module with orthogonal embeddings, aligns head‑tail attribute distributions using MMD and contrastive learning, and jointly trains biased and unbiased towers, achieving higher tail recall, lower exposure concentration, and measurable online click‑through improvements.

Domain AdaptationRecommendation Systemscontrastive learning
0 likes · 13 min read
Decoupling Popularity Bias in Dual‑Tower Retrieval Models
DataFunSummit
DataFunSummit
May 26, 2022 · Artificial Intelligence

Exploring Contrastive Learning in Kuaishou Recommendation Systems

This article presents a comprehensive overview of how contrastive learning can alleviate data sparsity and distribution bias in recommendation systems, detailing its theoretical advantages, recent research progress in computer vision and NLP, and a multi‑task self‑supervised framework applied to Kuaishou's short‑video ranking pipeline with significant offline and online performance gains.

.aiKuaishouRecommendation Systems
0 likes · 21 min read
Exploring Contrastive Learning in Kuaishou Recommendation Systems
DaTaobao Tech
DaTaobao Tech
May 17, 2022 · Artificial Intelligence

Self-Supervised Learning for Image Embeddings in Recommendation Systems: SwAV and M6 Applications at Meiping Meiwu

The paper demonstrates how self‑supervised models SwAV and M6 generate high‑quality image and multimodal embeddings for Meiping Meiwu’s recommendation system, delivering notable gains in scene/style consistency, ranking AUC, classification and retrieval performance, especially for cold‑start items, and achieving measurable production lifts.

A/B testingM6 multimodalRecommendation Systems
0 likes · 15 min read
Self-Supervised Learning for Image Embeddings in Recommendation Systems: SwAV and M6 Applications at Meiping Meiwu
Bilibili Tech
Bilibili Tech
May 10, 2022 · Artificial Intelligence

Glance Supervised Video Moment Retrieval via the ViGA Framework

The paper presents a glance‑supervised video moment retrieval approach that records a single annotator‑seen frame, introduces the ViGA contrastive learning framework to leverage this weak temporal cue, and demonstrates on three benchmarks performance rivaling fully supervised methods while keeping annotation cost minimal.

Glance SupervisionMultimodalViGA
0 likes · 8 min read
Glance Supervised Video Moment Retrieval via the ViGA Framework
Kuaishou Tech
Kuaishou Tech
Apr 18, 2022 · Artificial Intelligence

SSAN: A Novel Dual‑Stream Network for Domain‑Generalized Face Anti‑Spoofing

This paper proposes SSAN, a novel dual‑stream network that separates content and style features to achieve domain‑generalized face anti‑spoofing, employing adversarial learning for content, contrastive learning for style, and a large‑scale evaluation protocol across twelve public datasets, achieving state‑of‑the‑art performance.

SSANStyle Transfercontrastive learning
0 likes · 16 min read
SSAN: A Novel Dual‑Stream Network for Domain‑Generalized Face Anti‑Spoofing
DaTaobao Tech
DaTaobao Tech
Apr 12, 2022 · Artificial Intelligence

ArcCSE: Angular Margin Contrastive Learning for Self‑Supervised Text Representation

ArcCSE introduces an angular‑margin contrastive loss and both pairwise (dropout‑augmented) and triple‑wise (span‑masked) relationship modeling to self‑supervise text embeddings, yielding tighter decision boundaries, higher alignment and uniformity, and superior performance on unsupervised STS, SentEval, and Alibaba’s retrieval and recommendation systems.

NLPangular margincontrastive learning
0 likes · 8 min read
ArcCSE: Angular Margin Contrastive Learning for Self‑Supervised Text Representation
Laiye Technology Team
Laiye Technology Team
Apr 1, 2022 · Artificial Intelligence

Self‑Supervised Learning: Contrastive Methods and the MoCo Series (V1‑V3)

This article introduces the four types of machine learning, explains self‑supervised learning, details generative and contrastive approaches, and provides an in‑depth overview of the MoCo series (V1‑V3), including their architectures, training strategies, and experimental results on document image classification and text‑line detection tasks.

MoCoVision Transformerscontrastive learning
0 likes · 12 min read
Self‑Supervised Learning: Contrastive Methods and the MoCo Series (V1‑V3)
DataFunSummit
DataFunSummit
Feb 12, 2022 · Artificial Intelligence

Advances and Challenges in Post‑BERT Semantic Matching: Negative Sampling, Data Augmentation, and Applications

After the BERT era, this article reviews the limitations of pre‑trained language models for semantic matching, discusses negative‑sample sampling, data‑augmentation techniques, contrastive learning methods such as ConSERT and SimCSE, and practical deployment considerations in vector‑based retrieval systems.

contrastive learningdata augmentationpretrained language models
0 likes · 20 min read
Advances and Challenges in Post‑BERT Semantic Matching: Negative Sampling, Data Augmentation, and Applications
DataFunSummit
DataFunSummit
Jan 8, 2022 · Artificial Intelligence

Graph Information Bottleneck and AD‑GCL: Enhancing Graph Representation Learning and Robustness

This article introduces graph representation learning, explains the Graph Information Bottleneck (GIB) framework for obtaining robust graph embeddings, and presents AD‑GCL, a contrastive learning method that leverages GIB principles to improve graph neural network performance without requiring task labels.

Graph Neural NetworksGraph RepresentationRobustness
0 likes · 15 min read
Graph Information Bottleneck and AD‑GCL: Enhancing Graph Representation Learning and Robustness
Code DAO
Code DAO
Dec 22, 2021 · Artificial Intelligence

Understanding SimCLR: A Simple Contrastive Learning Framework for Visual Representations

This article explains SimCLR, the 2020 Google Research framework that advances self‑supervised visual pre‑training by using extensive data augmentations, a ResNet encoder, a projection‑head MLP, and the NT‑Xent loss to learn robust image representations that outperform many prior methods on ImageNet and other benchmarks.

NT-Xent lossResNetSimCLR
0 likes · 7 min read
Understanding SimCLR: A Simple Contrastive Learning Framework for Visual Representations
Baobao Algorithm Notes
Baobao Algorithm Notes
Dec 16, 2021 · Artificial Intelligence

Boost Model Robustness with 5 Lines of R‑Drop Contrastive Learning

This article introduces a simple five‑line implementation of R‑Drop, a contrastive self‑supervised learning technique that leverages dropout‑induced perturbations to improve model robustness, explains the underlying principle, provides the exact PyTorch code, and compares it with the ConSERT method.

DropoutPyTorchcontrastive learning
0 likes · 5 min read
Boost Model Robustness with 5 Lines of R‑Drop Contrastive Learning
DataFunTalk
DataFunTalk
Dec 11, 2021 · Artificial Intelligence

Graph Information Bottleneck (GIB) and AD‑GCL: Enhancing Graph Representation Learning and Robustness

This article introduces graph representation learning, explains the Graph Information Bottleneck (GIB) framework for obtaining robust graph embeddings, and presents AD‑GCL, a contrastive learning method that leverages GIB principles to improve robustness and performance without requiring task labels.

Graph Neural NetworksRobustnesscontrastive learning
0 likes · 16 min read
Graph Information Bottleneck (GIB) and AD‑GCL: Enhancing Graph Representation Learning and Robustness
DataFunTalk
DataFunTalk
Nov 1, 2021 · Artificial Intelligence

Reflections on Working as an Algorithm Engineer at Meituan and the Rise of Contrastive Learning

The author shares personal experiences as a Meituan algorithm engineer, emphasizing the critical role of labeled data, the emergence of contrastive (self‑supervised) learning across computer vision, NLP, and recommendation systems, and offers practical advice for algorithm engineers to stay competitive.

AI researchMeituanalgorithm engineering
0 likes · 8 min read
Reflections on Working as an Algorithm Engineer at Meituan and the Rise of Contrastive Learning
DataFunSummit
DataFunSummit
Oct 29, 2021 · Artificial Intelligence

Contrastive Learning Perspectives on Retrieval and Ranking Models in Recommendation Systems

This talk explains contrastive learning fundamentals, typical image‑domain models such as SimCLR, MoCo and SwAV, and shows how their principles—positive/negative sample construction, encoder design, loss functions, alignment and uniformity—can be applied to improve dual‑tower retrieval and ranking models, embedding normalization, temperature scaling, and graph‑based recommender systems.

InfoNCERecommendation Systemscontrastive learning
0 likes · 40 min read
Contrastive Learning Perspectives on Retrieval and Ranking Models in Recommendation Systems
DataFunTalk
DataFunTalk
Oct 26, 2021 · Artificial Intelligence

Contrastive Learning Perspective on Retrieval and Reranking Models in Recommendation Systems

This article explains how contrastive learning, originally popular in computer‑vision, can be interpreted and applied to recommendation‑system recall and coarse‑ranking models, covering its theoretical roots, typical architectures like SimCLR, MoCo and SwAV, and practical tricks such as in‑batch negatives, embedding normalization, temperature scaling, and graph‑based extensions.

Graph Neural NetworksRecommendation Systemscontrastive learning
0 likes · 40 min read
Contrastive Learning Perspective on Retrieval and Reranking Models in Recommendation Systems
Kuaishou Tech
Kuaishou Tech
Oct 20, 2021 · Artificial Intelligence

HiT: Hierarchical Transformer with Momentum Contrast for Video-Text Retrieval

This paper proposes HiT, a hierarchical transformer model with momentum contrast for video-text retrieval, addressing limitations in existing multimodal learning methods by introducing hierarchical cross-modal contrast matching and momentum cross-modal contrast to improve retrieval performance.

HCMMCCMoCo
0 likes · 9 min read
HiT: Hierarchical Transformer with Momentum Contrast for Video-Text Retrieval
DataFunTalk
DataFunTalk
Oct 19, 2021 · Artificial Intelligence

Graph Contrastive Learning: Foundations, Methods, and Recent Advances (GRACE & GCA)

This article reviews recent research on graph self‑supervised learning, focusing on contrastive learning fundamentals, the SimCLR‑style framework, representative models such as GRACE and its adaptive augmentation extension GCA, experimental evaluations, and future directions for graph contrastive methods.

GCAGraceGraph Neural Networks
0 likes · 16 min read
Graph Contrastive Learning: Foundations, Methods, and Recent Advances (GRACE & GCA)
DataFunSummit
DataFunSummit
Oct 19, 2021 · Artificial Intelligence

Deep Graph Contrastive Learning: GRACE and GCA

This article reviews recent advances in graph contrastive learning, introducing foundational concepts, the SimCLR framework, and representative models such as GRACE and its adaptive augmentation variant GCA, followed by experimental results, analysis, and future research directions.

GCAGraceGraph Representation
0 likes · 16 min read
Deep Graph Contrastive Learning: GRACE and GCA
DataFunSummit
DataFunSummit
Sep 26, 2021 · Artificial Intelligence

Contrastive Learning and Its Applications in Weibo Content Representation

This article explains the fundamentals of contrastive learning, reviews typical models such as SimCLR, MoCo, SwAV, BYOL, SimSiam and Barlow Twins, and demonstrates how these methods are applied to Weibo text and multimodal (text‑image) representation tasks like hashtag generation and image‑text matching.

MultimodalNLPWeibo
0 likes · 18 min read
Contrastive Learning and Its Applications in Weibo Content Representation
Laiye Technology Team
Laiye Technology Team
Sep 24, 2021 · Artificial Intelligence

Self‑Supervised Learning and Contrastive Methods for Computer Vision and OCR Applications

This article surveys self‑supervised learning techniques for computer‑vision tasks, explains common pretext tasks and contrastive loss designs, reviews representative models such as SimCLR, MoCo, SmAV and SimSiam, and demonstrates their practical impact on a captcha‑OCR system with measurable accuracy gains.

Deep LearningOCRSimCLR
0 likes · 23 min read
Self‑Supervised Learning and Contrastive Methods for Computer Vision and OCR Applications
DataFunTalk
DataFunTalk
Aug 30, 2021 · Artificial Intelligence

Contrastive Learning: Foundations, Typical Models, and Applications to Weibo Content Representation

This article explains the concept of contrastive learning, its relationship to self‑supervised and metric learning, describes key system components and loss functions, reviews major image, NLP and multimodal models such as SimCLR, MoCo, SwAV, BYOL, and demonstrates how contrastive learning is applied to Weibo hashtag generation, similar‑post retrieval, and text‑image matching using CD‑TOM and W‑CLIP models.

.aiMultimodalWeibo
0 likes · 19 min read
Contrastive Learning: Foundations, Typical Models, and Applications to Weibo Content Representation
58 Tech
58 Tech
Aug 5, 2021 · Artificial Intelligence

Exploration and Practice of Text Representation Algorithms in the 58 Security Scenario

This article presents a comprehensive study of text representation techniques—from weighted word‑vector methods to supervised SimBert and unsupervised contrastive learning models—applied to large‑scale unstructured data in 58's information‑security workflows, evaluating their effectiveness for classification and content‑recall tasks.

BERTPretrained ModelsSimCSE
0 likes · 11 min read
Exploration and Practice of Text Representation Algorithms in the 58 Security Scenario
DataFunTalk
DataFunTalk
Jun 6, 2021 · Artificial Intelligence

ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation Transfer

ConSERT introduces a contrastive self‑supervised framework that enhances BERT‑derived sentence embeddings by applying efficient embedding‑level data augmentations, achieving significant improvements on semantic textual similarity tasks, especially in low‑resource settings, and outperforming previous state‑of‑the‑art methods.

BERTcontrastive learningself-supervised
0 likes · 20 min read
ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation Transfer
Meituan Technology Team
Meituan Technology Team
Jun 3, 2021 · Artificial Intelligence

ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation Transfer

ConSERT is a contrastive self‑supervised framework that fine‑tunes BERT with augmented sentence views and NT‑Xent loss to overcome embedding collapse, delivering roughly 8% higher STS performance than prior methods, remaining robust in few‑shot and supervised scenarios, and now deployed in Meituan’s NLP pipelines.

BERTNLPcontrastive learning
0 likes · 20 min read
ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation Transfer
Amap Tech
Amap Tech
Dec 30, 2020 · Artificial Intelligence

LRC-BERT: Contrastive Learning based Knowledge Distillation with COS‑NCE Loss for Efficient NLP Models

The Amap team introduced LRC‑BERT, a contrastive‑learning‑based knowledge‑distillation framework that employs a novel COS‑NCE loss, gradient‑perturbation, and a two‑stage training schedule, enabling a 4‑layer student model to retain about 97 % of BERT‑Base accuracy while being 7.5× smaller and 9.6× faster, and it has already improved real‑world traffic‑event extraction performance.

BERTCOS-NCE lossNLP
0 likes · 16 min read
LRC-BERT: Contrastive Learning based Knowledge Distillation with COS‑NCE Loss for Efficient NLP Models