DataFunTalk
Mar 16, 2022 · Artificial Intelligence
Parameter-Efficient Sparsity Training for the PLUG Large-Scale Language Model
This article presents the PLUG 270‑billion‑parameter Chinese language model and introduces a parameter‑efficient sparsity training (PST) framework that combines unstructured and structured pruning with low‑rank decomposition to dramatically reduce model size while preserving downstream performance.
Deep LearningPLUGlarge language models
0 likes · 13 min read