Tagged articles
PEFT
13 articles · Page 1 of 1
This comprehensive guide walks through efficient LoRA/QLoRA fine-tuning of 7B and 13B language models on a single 24GB GPU, covering memory budgeting, environment setup, data preparation, training configuration, evaluation strategies, model merging, deployment verification, and production best practices with concrete code examples and benchmarks.
GPUHuggingFaceLLM
0 likes · 78 min read

Researchers from Zhejiang University, ByteDance Seed, and Tsinghua University propose Cortex, a method that consolidates contextual experience into model parameters via prospective rehearsal and context distillation, enabling agents to retain skills and knowledge without external retrieval, validated across tool use, fact updating, and agent tasks with minimal parameter growth.
Continual LearningKnowledge EditingLLM Agents
0 likes · 39 min read

VeriLoop Coder‑E1, an open‑source code model built on Qwen3.6‑27B, achieves top rankings on multiple software‑engineering benchmarks, leverages narrow‑domain PEFT fine‑tuning and a Self‑Harness execution loop, and introduces an evidence‑governed spiral that enables verified recursive self‑improvement while the community rapidly creates low‑precision quantizations.
Coder-E1Evidence-Governed SpiralPEFT
0 likes · 17 min read

The PEFT‑Arena benchmark reframes parameter‑efficient fine‑tuning evaluation as a stability‑plasticity trade‑off, measuring both downstream task gains and the preservation of pretrained general abilities through dual‑axis metrics, weight‑space and activation‑space analyses, and path‑wise diagnostics.
Activation GeometryInterpolation AnalysisModel Forgetting
0 likes · 12 min read

This article systematically presents ten large‑model training and fine‑tuning methods—from full‑parameter finetuning to parameter‑efficient PEFT—detailing their principles, suitable scenarios, step‑by‑step workflows, code examples, and practical selection guidance for AI product managers.
AdapterLoRAPEFT
0 likes · 13 min read

The article explains how pretraining on massive generic data creates a reusable base model, while fine‑tuning uses smaller, high‑quality task‑specific data to adapt the model, covering objectives, data scale, cost, methods, and why most projects prefer fine‑tuning.
LoRAPEFTPretraining
0 likes · 6 min read

This article examines three parameter‑efficient fine‑tuning (PEFT) techniques—LoRA, QLoRA, and DoRA—explaining their core mechanisms, providing implementation code, benchmark results, memory and speed trade‑offs, and offering guidance on which method best fits different hardware and accuracy requirements.
DORALoRAPEFT
0 likes · 20 min read

This tutorial walks through installing Python dependencies, preparing a JSON‑based QA dataset, and using Hugging Face's PEFT library to fine‑tune a small FLAN‑T5 model so it can answer questions about AC/DC and other bands without passing knowledge at inference time.
FAQ modelHugging FaceLLM fine-tuning
0 likes · 12 min read

The paper proposes a bidirectional optimization framework that fine‑tunes the low‑resource NLLB‑200 translation model with LoRA using data generated by ChatGPT, while also translating low‑resource prompts with NLLB before feeding them to LLMs, thereby improving multilingual translation quality yet requiring careful validation of noisy synthetic data.
LLMLoRANLLB
0 likes · 28 min read

This article presents a detailed overview of generative AI lifecycle management, covering practical use cases such as email summarization, the roles of providers, fine‑tuners and consumers, MLOps/LLMOps processes, retrieval‑augmented generation, efficient fine‑tuning methods like PEFT, and Amazon Bedrock services for model deployment and monitoring.
Amazon BedrockLLMOpsMLOps
0 likes · 14 min read


58 TechJun 3, 2024 · Artificial Intelligence
This article systematically analyzes popular parameter‑efficient fine‑tuning (PEFT) techniques for large language models—including Adapter Tuning, Prefix Tuning, LoRA, QLoRA, AdaLoRA, and SoRA—detailing their principles, implementation code, experimental results on NLU tasks, and practical acceleration using the Unsloth library.
AdaLoRALoRAPEFT
0 likes · 39 min read

This article provides a comprehensive technical guide on large language model fundamentals, covering tokenizer construction methods such as BPE, WordPiece, and SentencePiece, detailed explanations of Layer Normalization variants, Deep Norm concepts with code, and an overview of parameter‑efficient fine‑tuning techniques like LoRA and PEFT.
LLMLayer NormalizationPEFT
0 likes · 36 min read

This article introduces parameter‑efficient fine‑tuning (PEFT) techniques—including additive adapters, soft‑prompt methods, selection‑based BitFit, and re‑parameterization approaches like LoRA and AdaLoRA—explains their architectures, experimental results, and provides end‑to‑end code for fine‑tuning ChatGLM2‑6B on a Chinese medical QA dataset.
AdaLoRAAdapterLoRA
0 likes · 22 min read
