continual learning

Cognitive Technology Team

Feb 7, 2025 · Artificial Intelligence

Knowledge Distillation: Concepts, Techniques, Applications, and Future Directions

This article explains knowledge distillation—a technique introduced by Geoffrey Hinton that transfers knowledge from large teacher models to compact student models—covering its core concepts, loss functions, various distillation strategies, notable applications in edge computing, federated learning, continual learning, and emerging research directions.

Federated Learningcontinual learningdeep learning

0 likes · 7 min read

Knowledge Distillation: Concepts, Techniques, Applications, and Future Directions

DataFunTalk

Dec 1, 2022 · Artificial Intelligence

Advances and Challenges in Controllable Text Generation with Pretrained Language Models

This report reviews the background, recent research progress, practical applications, and future directions of controllable text generation using transformer‑based pretrained language models, highlighting methods such as decoding strategies, prompt learning, memory networks, continual learning, contrastive training, and knowledge integration.

continual learningcontrastive trainingcontrollable text generation

0 likes · 13 min read

Advances and Challenges in Controllable Text Generation with Pretrained Language Models

Alimama Tech

Sep 14, 2022 · Artificial Intelligence

Streaming Graph Neural Networks via Generative Replay

The paper introduces SGNN‑GR, a framework that pairs a graph neural network with a GAN‑based generative model to replay synthetic historical nodes, enabling continual learning on evolving graphs without storing raw data, achieving near‑retraining accuracy while being 3–6× faster per iteration.

Graph Neural Networkscontinual learninggenerative replay

0 likes · 10 min read

Streaming Graph Neural Networks via Generative Replay

DaTaobao Tech

Aug 30, 2022 · Artificial Intelligence

CTNet: Continual Transfer Learning for Cross-Domain Recommendation

CTNet is a continual transfer learning framework that uses a lightweight Adapter to map source‑domain features onto evolving target‑domain recommendation tasks, preserving all model parameters to avoid catastrophic forgetting and delivering substantial gains in click‑through rate, conversion, and overall business performance in Taobao’s cross‑domain e‑commerce scenario.

Adapter ModuleRecommendation systemscontinual learning

0 likes · 12 min read

CTNet: Continual Transfer Learning for Cross-Domain Recommendation