BestHub
Sign in
Tagged articles

Unsloth

24 articles · Page 1 of 1
Old Zhang's AI Learning
Old Zhang's AI Learning
Aug 23, 2026 · Artificial Intelligence

Qwen3.8-27B Quantization Selection Guide: Match Your Hardware to the Right GGUF Version

This guide analyzes Unsloth's Dynamic v3.0 quantizations of Qwen3.8-27B, showing Mean KLD divergence across versions, recommending UD-Q4_K_XL for 24GB GPUs, detailing hardware requirements, sampling parameters for thinking modes, and explaining why 4-bit is the baseline for tool use while 8-bit offers diminishing returns.

Dynamic v3.0GGUFMean KLD
0 likes · 10 min read
Qwen3.8-27B Quantization Selection Guide: Match Your Hardware to the Right GGUF Version
PaperAgent
PaperAgent
Aug 1, 2026 · Artificial Intelligence

How to Run DeepSeek‑V4‑Flash Locally on a 100 GB Server: Best‑Practice Guide

The article details the release of DeepSeek‑V4‑Flash‑0731, explains how its 284 B‑parameter, 13 B‑activated model can run losslessly on a machine with only 169 GB RAM using Unsloth’s UD‑Q8_K_XL quantization, compares quantization quality, and provides step‑by‑step deployment instructions via Unsloth Studio and llama.cpp.

AI Model DeploymentDeepSeek-V4-FlashLossless Quantization
0 likes · 7 min read
How to Run DeepSeek‑V4‑Flash Locally on a 100 GB Server: Best‑Practice Guide
AI Engineering
AI Engineering
Jul 21, 2026 · Artificial Intelligence

Unsloth Adds AMD Support: Train LLMs on 3 GB VRAM GPUs

Unsloth now supports AMD GPUs with custom ROCm‑optimized Triton kernels, delivering up to double the training speed and 70% lower memory usage, enabling over 500 LLMs to be trained on as little as 3 GB VRAM and providing detailed performance benchmarks on MI300X.

AMDGPULLM
0 likes · 5 min read
Unsloth Adds AMD Support: Train LLMs on 3 GB VRAM GPUs
Old Zhang's AI Learning
Old Zhang's AI Learning
Jul 8, 2026 · Artificial Intelligence

Deploying the Quantized DeepSeek‑V4‑Flash Locally: Performance, Benchmarks, and Tips

This article walks through DeepSeek‑V4‑Flash, a lightweight 284B MoE model with 13B active parameters and 1‑million‑token context, explains its three thinking modes, presents official benchmark results showing Flash can match Pro with higher reasoning budget, and provides step‑by‑step local deployment instructions using Unsloth’s GGUF quantization and llama.cpp fixes.

AIDeepSeek-V4-FlashGGUF
0 likes · 13 min read
Deploying the Quantized DeepSeek‑V4‑Flash Locally: Performance, Benchmarks, and Tips
Old Zhang's AI Learning
Old Zhang's AI Learning
Jun 2, 2026 · Artificial Intelligence

Turn Local LLMs into Actionable Agents – Unsloth Opens the MCP Path

Unsloth now lets locally‑run large language models act as real agents by exposing a Model Context Protocol (MCP) interface through a no‑code Studio UI or a llama.cpp + mcp‑cli command line, supporting tool calling, file access, web search, and multi‑model connections with detailed setup steps, hardware guidance, and security cautions.

AI AgentsMCPModel Context Protocol
0 likes · 17 min read
Turn Local LLMs into Actionable Agents – Unsloth Opens the MCP Path
Old Zhang's AI Learning
Old Zhang's AI Learning
May 7, 2026 · Artificial Intelligence

How Unsloth and NVIDIA Boost Consumer‑GPU LLM Training by ~25% with Three Simple Optimizations

Unsloth and NVIDIA identified three low‑level bottlenecks in LLM fine‑tuning on consumer GPUs—repeated packed‑sequence metadata construction, serialized copy‑and‑compute during gradient checkpointing, and per‑expert routing overhead in MoE—and applied targeted patches that together deliver roughly a 25% speedup without changing hardware, code, or frameworks.

GPU OptimizationGradient CheckpointingLLM training
0 likes · 12 min read
How Unsloth and NVIDIA Boost Consumer‑GPU LLM Training by ~25% with Three Simple Optimizations
Old Zhang's AI Learning
Old Zhang's AI Learning
May 1, 2026 · Artificial Intelligence

NVIDIA’s Open‑Source Multimodal Nemotron 3 Nano Omni: Run Locally on Consumer GPUs (English‑Only)

NVIDIA’s Nemotron 3 Nano Omni 30B‑A3B‑Reasoning model, an open‑source multimodal LLM with 30 B parameters, 256K context and video‑audio‑image‑text capabilities, outperforms comparable models by up to 9.2× in video throughput, runs on consumer GPUs via 4‑bit GGUF quantization, but currently supports only English input.

GGUFGPUNVIDIA
0 likes · 17 min read
NVIDIA’s Open‑Source Multimodal Nemotron 3 Nano Omni: Run Locally on Consumer GPUs (English‑Only)
Old Zhang's AI Learning
Old Zhang's AI Learning
Apr 26, 2026 · Artificial Intelligence

Distilling Claude Opus into Qwen3.6-27B – GGUF Lets You Run Locally on Consumer GPUs

The preview model Qwopus3.6-27B‑v1, distilled from Claude Opus onto Qwen3.6‑27B using SFT with the Unsloth stack and a curated 12 K high‑quality inference sample set, is evaluated on agentic reasoning, front‑end design, and Canvas/WebGL tasks with an RTX 5090, and can be deployed locally via llama.cpp GGUF quantizations with detailed memory guidelines.

Apache 2.0Claude OpusGGUF
0 likes · 7 min read
Distilling Claude Opus into Qwen3.6-27B – GGUF Lets You Run Locally on Consumer GPUs
Old Zhang's AI Learning
Old Zhang's AI Learning
Apr 19, 2026 · Artificial Intelligence

From Zero to Deployment: A Complete Qwen3.5 Fine‑Tuning Guide

This guide shows how to fine‑tune Qwen3.5 models—from 0.8B to 122B—using Unsloth Studio or pure code, covering text SFT, vision fine‑tuning, MoE models, reinforcement‑learning (GRPO), extensive GGUF quantization benchmarks, hardware requirements, export formats, and deployment tips.

LLMQwen3.5Unsloth
0 likes · 12 min read
From Zero to Deployment: A Complete Qwen3.5 Fine‑Tuning Guide
Old Zhang's AI Learning
Old Zhang's AI Learning
Apr 12, 2026 · Artificial Intelligence

How to Deploy MiniMax-M2.7 Quantized Models Locally on macOS and Linux

This guide explains the 22 GGUF quantized versions of MiniMax-M2.7 released by Unsloth, compares their accuracy and size, recommends the UD‑Q4_K_XL model for best quality‑to‑size trade‑off, and provides step‑by‑step instructions for local deployment via Unsloth Studio, llama.cpp, API server, or the MLX native solution, along with important pitfalls and performance‑tuning tips.

Dynamic 2.0MLXMiniMax M2.7
0 likes · 14 min read
How to Deploy MiniMax-M2.7 Quantized Models Locally on macOS and Linux
AI Explorer
AI Explorer
Mar 23, 2026 · Artificial Intelligence

How Unsloth Studio Turns Local AI Training into a Simple, High‑Performance Experience

Unsloth Studio, an open‑source local AI studio, combines a sleek web UI with a custom Triton kernel that claims up to 2× faster training, 70% VRAM savings (80% for RL), supports over 500 models, visual data‑recipe workflows, and both desktop and Python library usage for developers, researchers, and hobbyists.

AI StudioLocal AIModel Training
0 likes · 7 min read
How Unsloth Studio Turns Local AI Training into a Simple, High‑Performance Experience
Old Zhang's AI Learning
Old Zhang's AI Learning
Mar 12, 2026 · Artificial Intelligence

Distilling Claude Opus 4.6 into Qwen3.5‑27B: High‑Quality Reasoning on a Single RTX 3090

The article details how Claude Opus 4.6's chain‑of‑thought data were used to distill the 27‑billion‑parameter Qwen3.5‑27B model with Unsloth and LoRA, achieving full‑context inference on a single RTX 3090/4090, while outlining performance numbers, hyper‑parameter tips, benchmark gains and the trade‑offs of losing multimodal abilities.

Claude Opus 4.6GPU inferenceLoRA
0 likes · 7 min read
Distilling Claude Opus 4.6 into Qwen3.5‑27B: High‑Quality Reasoning on a Single RTX 3090
Old Zhang's AI Learning
Old Zhang's AI Learning
Mar 3, 2026 · Artificial Intelligence

How to Deploy and Fine‑Tune Qwen3.5 Small Models (0.8B‑9B) Locally

This guide walks you through deploying Qwen3.5's 0.8B, 2B, 4B and 9B models on CPUs or modest GPUs using Unsloth's GGUF quantization, explains hardware requirements, shows how to run them with llama.cpp, llama‑server, vLLM or SGLang, and provides a free Colab fine‑tuning workflow with export options.

AI modelsGGUFQwen3.5
0 likes · 19 min read
How to Deploy and Fine‑Tune Qwen3.5 Small Models (0.8B‑9B) Locally
Old Zhang's AI Learning
Old Zhang's AI Learning
Feb 21, 2026 · Artificial Intelligence

Why Fine‑Tuning Large Models Is Now Ridiculously Easy

The article explains how Unsloth dramatically lowers the barrier to fine‑tuning large language models, offering one‑click installation, free Colab GPU support, extensive model coverage, impressive speed and memory gains, and detailed step‑by‑step guides that let anyone with basic Python skills train powerful models.

ColabGPULoRA
0 likes · 14 min read
Why Fine‑Tuning Large Models Is Now Ridiculously Easy
Old Zhang's AI Learning
Old Zhang's AI Learning
Feb 17, 2026 · Artificial Intelligence

Running Qwen3.5 Locally: Step‑by‑Step Guide with Unsloth Dynamic Quantization

This article explains how to run the 397B Qwen3.5 model on a Mac by using Unsloth Dynamic 2.0 quantization (2‑bit, 3‑bit, or 4‑bit), outlines hardware requirements, provides compilation and download commands for llama.cpp, shows how to launch inference in thinking and non‑thinking modes, and compares several deployment options such as llama‑server, Transformers, SGLang/vLLM, and MLX.

Dynamic QuantizationGGUFLLM deployment
0 likes · 14 min read
Running Qwen3.5 Locally: Step‑by‑Step Guide with Unsloth Dynamic Quantization
Old Zhang's AI Learning
Old Zhang's AI Learning
Feb 5, 2026 · Artificial Intelligence

Distilling GLM‑4.7‑Flash with Claude‑Opus‑4.5 for Easy Consumer‑GPU Deployment

The article explains how TeichAI used Claude‑Opus‑4.5 to generate a high‑quality 250‑sample reasoning dataset and distill the GLM‑4.7‑Flash model into a compact GGUF version that runs on a single consumer‑grade GPU via llama.cpp, detailing the workflow, quantization options, and practical considerations.

AI datasetsGGUFUnsloth
0 likes · 6 min read
Distilling GLM‑4.7‑Flash with Claude‑Opus‑4.5 for Easy Consumer‑GPU Deployment
Fun with Large Models
Fun with Large Models
Mar 20, 2025 · Artificial Intelligence

Fine‑Tune DeepSeek‑R1 with Just a Few Lines of Code Using Unsloth

This guide walks through setting up an Anaconda environment, installing Unsloth, downloading the DeepSeek‑R1‑Distill‑Llama‑8B model, preparing a medical CoT dataset, configuring LoRA parameters, running a short fine‑tuning job, and evaluating the customized model with structured prompts.

DeepSeekLoRAPython
0 likes · 18 min read
Fine‑Tune DeepSeek‑R1 with Just a Few Lines of Code Using Unsloth
Ops Development & AI Practice
Ops Development & AI Practice
Feb 15, 2025 · Artificial Intelligence

How to Efficiently Fine‑Tune Llama 3 on a Free Colab T4 GPU with Unsloth

This article provides a step‑by‑step, code‑rich tutorial for fine‑tuning the open‑source Llama 3 1B and 3B models on Google Colab using the Unsloth library and LoRA, covering environment setup, model loading, adapter insertion, dataset preparation, training configuration, inference, and model saving, all while keeping GPU memory usage low.

AIColabGPU
0 likes · 13 min read
How to Efficiently Fine‑Tune Llama 3 on a Free Colab T4 GPU with Unsloth