This article outlines essential LLM engineering skills, including scripts for converting various model checkpoints to Llama format, customizing modeling files for advanced features, building a multi‑GPU inference class, and adding channel‑aware loss tracking to fine‑tuning pipelines.
This article presents a theoretical and experimental analysis of converting Group Query Attention (GQA) models to Multi‑Head Linear Attention (MLA) using the TransMLA method, demonstrating superior expressiveness and performance on DeepSeek‑based large language models while keeping KV‑Cache costs unchanged.
This guide presents a hands‑on curriculum of core large‑model engineering tasks—including model conversion scripts, custom modeling wrappers, multi‑model inference utilities, and channel‑aware loss tracking—to help practitioners build practical, reusable tools without deep theoretical overhead.
This article introduces Pai‑Megatron‑Patch, an open‑source tool from Alibaba Cloud that streamlines large language model (LLM) training, weight conversion, FP8 mixed‑precision acceleration, and reinforcement‑learning workflows, providing detailed architecture, key features, code examples, and step‑by‑step usage instructions.
The article details a workflow that converts the PyTorch ChatGLM‑6B model to MNN, splits and compresses embeddings, applies int4/int8 quantization, supports dynamic shapes, and uses hybrid GPU/CPU or CPU‑only loading to enable low‑memory edge inference on PCs and mobile devices with competitive token‑per‑second performance.
The article introduces dl_inference, an open‑source deep learning inference platform that integrates TensorRT GPU acceleration, Intel MKL CPU optimization, and Caffe support, detailing its features, model conversion workflow, deployment steps, performance gains, and how developers can contribute.
