Skip to content
Tagged articles

parameter-efficient training

3 articles · Page 1 of 1
Machine Learning Algorithms & Natural Language Processing
Machine Learning Algorithms & Natural Language Processing
Nov 1, 2026 · Artificial Intelligence

LoRA on Decoder Only: Half Parameters Match Full Fine-Tuning on DeepSeek-V4.1-Flash

The article explores optimal LoRA placement for DeepSeek-V4.1-Flash's causal encoder-decoder architecture, finding that applying LoRA only to decoder layers (20–39) with rank 16 matches full-model performance (0.296 vs 0.293 validation loss) while halving adapter parameters and enabling shared KV cache across adapters by freezing layer 20's KV projection.

DeepSeek V4.1 FlashKV cache sharingLoRA
0 likes · 32 min read
LoRA on Decoder Only: Half Parameters Match Full Fine-Tuning on DeepSeek-V4.1-Flash
System Architect Go
System Architect Go
Oct 24, 2024 · Artificial Intelligence

How to Fine‑Tune Translation Models on Kubernetes Docs with LoRA

This article walks through the complete process of fine‑tuning both domain‑specific and large‑language translation models on Kubernetes documentation, covering data preparation, model selection, training configurations, the differences between Seq2Seq and CausalLM, and how LoRA can dramatically reduce resource usage while improving performance.

AILLMLoRA
0 likes · 7 min read
How to Fine‑Tune Translation Models on Kubernetes Docs with LoRA
DataFunTalk
DataFunTalk
Mar 16, 2022 · Artificial Intelligence

Parameter-Efficient Sparsity Training for the PLUG Large-Scale Language Model

This article presents the PLUG 270‑billion‑parameter Chinese language model and introduces a parameter‑efficient sparsity training (PST) framework that combines unstructured and structured pruning with low‑rank decomposition to dramatically reduce model size while preserving downstream performance.

PLUGdeep learninglarge language models
0 likes · 13 min read
Parameter-Efficient Sparsity Training for the PLUG Large-Scale Language Model