BestHub
Sign in
Collection size
94 articles
Page 5 of 5
Alibaba Cloud Infrastructure
Alibaba Cloud Infrastructure
Jan 21, 2026 · Artificial Intelligence

Boost LLM Performance: Deploy Qwen3‑235B with PD‑Separation, MoE, SGLang & RBG

This article details how to deploy the 235‑billion‑parameter Qwen3‑235B model using PD‑separation and MoE techniques, explains the associated challenges, and demonstrates a production‑grade solution built on the high‑performance SGLang inference engine and the RoleBasedGroup (RBG) orchestration framework, complete with benchmark results and best‑practice YAML examples.

AIKubernetesLLM
0 likes · 21 min read
Boost LLM Performance: Deploy Qwen3‑235B with PD‑Separation, MoE, SGLang & RBG
Baobao Algorithm Notes
Baobao Algorithm Notes
Jan 7, 2025 · Artificial Intelligence

How Efficient Is DeepSeek V3? Calculating Its MFU Around 37%

This article derives DeepSeek V3's training Model FLOPs Utilization (MFU) using publicly available data, showing an MFU of roughly 37%—about a 60% improvement over V2—and provides detailed formulas, parameter settings, and a reproducible Python script.

AI performanceDeepSeekLarge Language Model
0 likes · 8 min read
How Efficient Is DeepSeek V3? Calculating Its MFU Around 37%
Old Zhang's AI Learning
Old Zhang's AI Learning
May 13, 2026 · Artificial Intelligence

Why vLLM Now Leads Open‑Source LLM Inference Benchmarks

vLLM tops the Artificial Analysis ranking by delivering the highest throughput for DeepSeek V3.2, Qwen 3.5 397B, and MiniMax‑M2.5 on identical NVIDIA Blackwell Ultra hardware, thanks to extensive kernel‑fusion optimizations that remain in the main branch.

DeepSeekLLM inferencePerformance benchmarking
0 likes · 7 min read
Why vLLM Now Leads Open‑Source LLM Inference Benchmarks
Alimama Tech
Alimama Tech
Feb 12, 2025 · Artificial Intelligence

HighService: A High‑Performance Pythonic AI Service Framework for Model Inference and Global Resource Scheduling

HighService, Alibaba’s Pythonic AI service framework, accelerates large‑model inference and maximizes GPU utilization by separating CPU‑GPU processes, offering out‑of‑the‑box quantization, parallelism and caching, and dynamically reallocating idle GPUs across clusters through a master‑worker scheduler to keep online latency low while boosting offline throughput for diffusion and LLM workloads.

AI ServiceDistributed SystemsPython
0 likes · 16 min read
HighService: A High‑Performance Pythonic AI Service Framework for Model Inference and Global Resource Scheduling
Tencent Technical Engineering
Tencent Technical Engineering
May 25, 2026 · Artificial Intelligence

vLLM Deep Dive: Continuous Batching and Paged Attention for Fast LLM Inference

This article walks through a two‑month source‑code study of vLLM, explaining how token‑level scheduling, continuous batching, and the Paged Attention mechanism reshape tensor dimensions to turn large‑model inference into a compute‑bound, high‑throughput process while managing GPU memory efficiently.

FlashAttentionGPU OptimizationLLM inference
0 likes · 29 min read
vLLM Deep Dive: Continuous Batching and Paged Attention for Fast LLM Inference
Geek Labs
Geek Labs
Mar 24, 2026 · Industry Insights

9 Must‑See GitHub Projects: MacBook‑Run LLM, WeChat AI, Multi‑Agent Collaboration and More

This article reviews nine standout GitHub open‑source projects, covering a C/Metal LLM engine for MacBooks, a Claude Code commercial‑analysis skill, multi‑agent communication tools, web‑enabled AI, autonomous research automation, WeChat AI integration, a minimalist terminal, a Codex console, and a lightweight WARP proxy.

AIDockerGitHub
0 likes · 10 min read
9 Must‑See GitHub Projects: MacBook‑Run LLM, WeChat AI, Multi‑Agent Collaboration and More
Machine Heart
Machine Heart
Jul 3, 2026 · Artificial Intelligence

Avoiding Pitfalls in Heterogeneous Token Factories: Industry‑Level Design Practices for Cross‑Hardware LLM Inference

The article analyzes a recent multi‑institution paper that maps the design space of heterogeneous Prefill‑Decode LLM inference, identifies three core boundary decisions, presents nine deployment best practices, and validates them with a production token‑factory case on MuXi C600 and NVIDIA Hopper GPUs.

KV CacheLLMdeployment best practices
0 likes · 11 min read
Avoiding Pitfalls in Heterogeneous Token Factories: Industry‑Level Design Practices for Cross‑Hardware LLM Inference
Alibaba Cloud Infrastructure
Alibaba Cloud Infrastructure
Feb 13, 2025 · Artificial Intelligence

Deploying DeepSeek‑R1 671B Distributed Inference Service on Alibaba Cloud ACK with vLLM and Dify

This article explains how to quickly deploy the full‑parameter DeepSeek‑R1 671B model in a multi‑node GPU‑enabled Kubernetes cluster on Alibaba Cloud ACK, covering prerequisites, model parallelism, vLLM‑Ray distributed deployment, service verification, and integration with Dify to build a private AI Q&A assistant.

DeepSeekDifyKubernetes
0 likes · 12 min read
Deploying DeepSeek‑R1 671B Distributed Inference Service on Alibaba Cloud ACK with vLLM and Dify
Alibaba Cloud Native
Alibaba Cloud Native
Jul 30, 2024 · Cloud Native

Deploy ComfyUI as a Serverless API for Scalable AI Image Generation

This article explains how to transform ComfyUI into a serverless API using Alibaba Cloud Function Compute, detailing the challenges of GPU resource costs, high concurrency, and usability, while providing a step‑by‑step guide, code examples, and best‑practice recommendations for building scalable AI drawing applications.

AI Image GenerationAPIComfyUI
0 likes · 21 min read
Deploy ComfyUI as a Serverless API for Scalable AI Image Generation
Geek Labs
Geek Labs
May 13, 2026 · Artificial Intelligence

Two LLM Inference Acceleration Projects: A Mac‑Local Engine vs a Data‑Center Engine

This article compares two recent GitHub LLM inference engines—ds4.c, a Metal‑optimized engine for DeepSeek V4 Flash on Apple Silicon Macs, and TokenSpeed, a Python/C++‑based, data‑center‑grade engine for GPU clusters—detailing their design choices, performance numbers, usage instructions, and suitable scenarios.

DeepSeekGPULLM
0 likes · 8 min read
Two LLM Inference Acceleration Projects: A Mac‑Local Engine vs a Data‑Center Engine
Old Zhang's AI Learning
Old Zhang's AI Learning
Apr 26, 2026 · Artificial Intelligence

Why Deploying DeepSeek‑V4 Locally with vLLM Is So Challenging

The article dissects DeepSeek‑V4’s local deployment using vLLM, explaining the steep hardware requirements, the complex heterogeneous KV‑cache architecture, and the aggressive kernel‑fusion and multi‑stream optimizations that together make high‑context inference both memory‑intensive and engineering‑heavy.

DeepSeek-V4GPU memoryKV Cache
0 likes · 15 min read
Why Deploying DeepSeek‑V4 Locally with vLLM Is So Challenging
Alibaba Cloud Developer
Alibaba Cloud Developer
Jan 15, 2026 · Artificial Intelligence

How Hierarchical Sparse Attention Breaks KVCache Limits for Ultra‑Long Context LLMs

This article explains how a hierarchical sparse‑attention framework redesigns KVCache storage across GPU, CPU, and remote memory, eliminates bandwidth and capacity bottlenecks, and enables efficient inference for 128K‑token and larger contexts with dramatically reduced GPU memory usage and higher throughput.

Dynamic Sparse AttentionGPU memory optimizationHierarchical Storage
0 likes · 20 min read
How Hierarchical Sparse Attention Breaks KVCache Limits for Ultra‑Long Context LLMs
Cambridge Mofang Notes
Cambridge Mofang Notes
Sep 2, 2026 · Artificial Intelligence

Inference Frameworks vs Platforms: How LLMs Actually Run on Your Hardware

This article distinguishes between inference frameworks (llama.cpp, vLLM, SGLang) that execute model computations and inference platforms (Ollama, LM Studio, Xinference) that manage deployment, explaining their roles, interactions, and how to choose tools for local or server-side LLM inference.

AI inferenceLLM servingLM Studio
0 likes · 16 min read
Inference Frameworks vs Platforms: How LLMs Actually Run on Your Hardware