BestHub
Sign in
Tagged articles

Large Language Models

1206 articles · Page 13 of 13
DataFunTalk
DataFunTalk
Dec 24, 2021 · Artificial Intelligence

Large-Scale Pretrained Model Compression and Distillation: AdaBERT, L2A, and Meta‑KD

This article reviews three consecutive works from Alibaba DAMO Academy on compressing and distilling large pretrained language models—AdaBERT, L2A, and Meta‑KD—detailing their motivations, neural‑architecture‑search‑based designs, loss formulations, experimental results, and insights from a Q&A session.

AIKnowledge DistillationLarge Language Models
0 likes · 10 min read
Large-Scale Pretrained Model Compression and Distillation: AdaBERT, L2A, and Meta‑KD
Java High-Performance Architecture
Java High-Performance Architecture
Sep 3, 2021 · Artificial Intelligence

When AI Code Completion Leaks Fake ID Numbers: Copilot’s Privacy Risks

GitHub Copilot unexpectedly generated a fabricated ID for Bilibili CEO Chen Rui, sparking concerns about how large language models trained on public data can inadvertently expose synthetic personal information and highlighting broader privacy and ethical issues in AI‑driven code assistants.

AI privacyGitHub CopilotLarge Language Models
0 likes · 6 min read
When AI Code Completion Leaks Fake ID Numbers: Copilot’s Privacy Risks
Programmer DD
Programmer DD
Aug 29, 2021 · Artificial Intelligence

When AI Code Assistants Leak Fake IDs: What GitHub Copilot’s Slip Reveals

GitHub Copilot, powered by the Codex model, recently generated a seemingly real Chinese ID number for Bilibili CEO Chen Rui, sparking concerns about privacy leaks, model training data, and the broader risks of AI code assistants inadvertently exposing personal information.

AI code generationGitHub CopilotLarge Language Models
0 likes · 6 min read
When AI Code Assistants Leak Fake IDs: What GitHub Copilot’s Slip Reveals
DataFunTalk
DataFunTalk
Jul 1, 2021 · Artificial Intelligence

Pre‑Trained Models: Past, Present, and Future – A Comprehensive Survey

This article surveys the evolution of pre‑trained models, covering the origins of transfer and self‑supervised learning, the rise of transformer‑based PTMs such as BERT and GPT, efficient architecture designs, multimodal and multilingual extensions, theoretical analyses, and future research directions for scalable and robust AI systems.

AI researchEfficient TrainingLarge Language Models
0 likes · 27 min read
Pre‑Trained Models: Past, Present, and Future – A Comprehensive Survey
DataFunTalk
DataFunTalk
Jul 7, 2020 · Artificial Intelligence

Optimizing Pretrained Language Model Inference: Lessons from the NLPCC Small Model Competition and Deployment at Xiaomi

This article shares the Xiaomi AI Lab NLP team's experience in the NLPCC lightweight language model competition, discusses efficiency challenges of large pretrained models like BERT, and details practical inference optimizations—including model distillation, batching, FP16 quantization, and FasterTransformer integration—that dramatically reduce latency and hardware costs in production.

AIBERTInference Optimization
0 likes · 15 min read
Optimizing Pretrained Language Model Inference: Lessons from the NLPCC Small Model Competition and Deployment at Xiaomi