Tagged articles
GPU optimization
73 articles · Page 1 of 1
This guide introduces the scaling ladder method for PhD students to predict model performance across compute scales, optimize MFU, and efficiently allocate limited academic GPU resources through structured experiments and practical configuration tips.
GPU optimizationLLM trainingMFU
0 likes · 13 min read

This comprehensive guide covers production deployment and performance tuning of LLM inference services using vLLM and TGI, including hardware planning, Docker/Kubernetes orchestration, key parameter optimization, monitoring with Prometheus, autoscaling with KEDA, troubleshooting common failures, and security hardening.
CUDA GraphDockerFP8 quantization
0 likes · 99 min read

Anthropic's Claude optimized 36 packages and 30+ open-source biomolecular models in under four weeks, delivering 1.6×–4.1× speedups across Exact, Fast, and Big modes via FlashPairformer kernels and engineering optimizations, enabling large-complex predictions on single nodes while maintaining accuracy within statistical noise.
AI for ScienceCUDA kernelsClaude
0 likes · 17 min read

The article analyzes how the Muon optimizer, adopted by DeepSeek‑V4 and Kimi‑K2, suffers a 2.2× overhead in distributed training, and how the DMuon system from Zibian Robot reduces that overhead to near‑AdamW levels, achieving up to 97.4× speedup and only 2% slower end‑to‑end training than AdamW.
DMuonGPU optimizationMuon
0 likes · 11 min read

The vLLM 0.25 update adds dozens of new model architectures, expands support to AMD ROCm, CPUs, Intel XPU and IBM Power, fixes speculative decoding for reasoning models, and introduces a suite of performance, KV‑cache, quantization, and API enhancements for large‑scale LLM inference.
AMD ROCmGPU optimizationKV Cache
0 likes · 8 min read

Alibaba Mama AI Infra and AMD collaborated to integrate SGLang‑ATOM plugin mode for Qwen 3.5/3.6 models on MI308X GPUs, detailing multimodal, GDN, KV‑cache, and parallel‑strategy optimizations that achieve low‑latency, high‑throughput inference suitable for complex advertising workloads.
AMD ATOMGPU optimizationLLM inference
0 likes · 18 min read

The DSpark system from DeepSeek combines batch decoding, speculative decoding, draft‑model tricks, Eagle‑MTP, DFlash parallelism, variable‑length scheduling and online confidence calibration to deliver up to 85% speedup and four‑fold throughput gains while maintaining generation quality.
Batch DecodingDFlashDSpark
0 likes · 12 min read

DSpark, DeepSeek’s new LLM inference framework, combines batch processing, speculative decoding, Eagle‑style draft models and DFlash‑style parallel generation with a lightweight sequential head and hardware‑aware scheduling, delivering 60‑85% speedups while preserving model quality.
DeepSeekGPU optimizationLLM inference
0 likes · 12 min read

alphaXiv’s new AutoArxiv feature lets users turn any arXiv paper URL into an automated reproduction workflow that fixes dependencies, runs a minimal experiment, estimates full‑scale resource costs, and can compress a classic model like "Attention Is All You Need" to run on a single GPU.
AI toolGPU optimizationNLP
0 likes · 7 min read

This article walks through a two‑month source‑code study of vLLM, explaining how token‑level scheduling, continuous batching, and the Paged Attention mechanism reshape tensor dimensions to turn large‑model inference into a compute‑bound, high‑throughput process while managing GPU memory efficiently.
FlashAttentionGPU optimizationLLM inference
0 likes · 29 min read

CODA rewrites Transformer blocks as GEMM‑epilogue programs, exposing five primitive building blocks that let both AI‑generated code and human programmers fuse memory‑intensive operations into the GEMM epilogue, eliminating costly tensor moves and achieving up to 1.8× speed‑ups on H100 GPUs for RMSNorm, SwiGLU, RoPE and other components, while preserving numerical accuracy.
CODACUDAGEMM
0 likes · 11 min read

The paper shows that applying lightweight L1 regularization can make over 99% of FFN activations zero, and by using a new tile‑wise ELLPACK (TwELL) format together with a hybrid routing scheme, inference speed improves up to 30% while memory usage drops over 24% and energy consumption is reduced, all with negligible impact on downstream task performance.
CUDAGPU optimizationHybrid Routing
0 likes · 8 min read

A new ICML 2026 paper by Sakana AI and NVIDIA shows that applying lightweight L1 regularization can make Feed‑Forward Network activations in Transformers over 99% sparse, and with the TwELL storage format and a hybrid routing scheme this sparsity translates into up to 20.5% inference speedup, 21.9% training‑step acceleration, lower energy consumption and reduced peak memory across 0.5‑2 B‑parameter models while preserving downstream performance.
CUDAGPU optimizationHybrid Routing
0 likes · 9 min read

Unsloth and NVIDIA identified three low‑level bottlenecks in LLM fine‑tuning on consumer GPUs—repeated packed‑sequence metadata construction, serialized copy‑and‑compute during gradient checkpointing, and per‑expert routing overhead in MoE—and applied targeted patches that together deliver roughly a 25% speedup without changing hardware, code, or frameworks.
GPU optimizationGradient CheckpointingLLM training
0 likes · 12 min read

This guide details how enterprises can move large language models from lab to production by defining specific SLI/SLO metrics, diagnosing hidden bottlenecks such as tokenizer latency, and applying four quantifiable optimization levers that dramatically improve latency, throughput, and cost efficiency.
GPU optimizationLoRAcontinuous batching
0 likes · 6 min read

DeepSeek has released TileKernels, a GPU kernel library written in the TileLang DSL, that targets H100/H200/B200 GPUs and claims to approach hardware limits in compute intensity and memory bandwidth, offering MoE routing, FP8/FP4 quantization, and dual‑language PyTorch references for deep‑learning engineers.
FP8 quantizationGPU optimizationLLM training
0 likes · 9 min read

Sol‑RL, a framework from NVIDIA, Hong Kong University and MIT, integrates NVFP4 inference for large‑scale rollout exploration and BF16 precision for high‑fidelity regeneration, delivering up to 4.64× faster convergence at equivalent reward levels while preserving BF16 training fidelity across SANA, FLUX.1 and SD3.5‑L models.
BF16FP4GPU optimization
0 likes · 9 min read


SuanNiMar 29, 2026 · Artificial Intelligence
This article analyzes the AVO system, an autonomous AI agent that replaces traditional evolutionary search pipelines to iteratively improve CUDA attention kernels on NVIDIA's Blackwell B200 GPU, achieving up to 10.5% higher throughput than hand‑tuned implementations after a week of nonstop optimization.
AIAttentionCUDA
0 likes · 13 min read

Unsloth offers a unified web UI that accelerates fine‑tuning by up to 2×, cuts VRAM usage by 70% (80% for RL), supports hundreds of open‑source models, and provides simple installation steps for rapid local AI experimentation.
AI workstationGPU optimizationLLM
0 likes · 6 min read

CUDA Agent, a large‑scale reinforcement‑learning system from ByteDance and Tsinghua, can automatically generate and optimize CUDA kernels that outperform torch.compile by up to 2× on simple kernels and achieve around 40% higher speed than proprietary models on the hardest benchmarks, while detailing its data‑synthesis pipeline, training workflow, and current limitations.
CUDAGPU optimizationKernelBench
0 likes · 10 min read

ByteDance and Tsinghua University have created the CUDA Agent, an AI compiler that automatically writes and optimizes GPU kernels, delivering up to double the performance, and heralding a shift where AI‑generated low‑level code could reshape the hardware‑software competition landscape.
AI CompilerByteDanceCUDA
0 likes · 6 min read

This article examines the Wan2.1 video diffusion model, identifies its scalability bottlenecks for long and real‑time video generation, and introduces the Self‑Forcing causal framework together with sequence‑parallel and RoPE optimizations that achieve sub‑second latency and up to 1.5× speed‑up on modern GPUs.
Causal InferenceGPU optimizationRotary Positional Encoding
0 likes · 14 min read

Analyzing recent DeepSeek flashmla repository commits, the article uncovers that the mysterious Model1 likely corresponds to DeepSeek‑V4, detailing architectural shifts to a 512‑dimensional head, full support for NVIDIA Blackwell GPUs, token‑level sparse MLA, and new mechanisms such as Value Vector Position Awareness and Engram.
DeepSeekGPU optimizationLLM
0 likes · 6 min read

This guide details how to replace native Transformers inference with the high‑performance vLLM engine, leveraging PagedAttention, continuous batching, tensor parallelism, and OpenAI‑compatible APIs to achieve 3‑4× higher throughput, lower latency, and scalable multi‑GPU deployments for production‑grade large language models.
GPU optimizationOpenAI API CompatibilityPagedAttention
0 likes · 61 min read

This article walks through two mainstream local deployment solutions—high‑performance VLLM for production Linux servers and lightweight Ollama for personal Windows machines—covering environment setup, model download, server launch, API testing, key configuration parameters, and the quantization technique that makes Ollama models compact.
GPU optimizationOllamaOpenAI API
0 likes · 18 min read


UbuntuJan 12, 2026 · Artificial Intelligence
The article explains why running AI locally on Ubuntu offers data security, zero token costs, offline capability, and millisecond response times, then provides a step‑by‑step guide to install Ollama via Snap, pull the DeepSeek Coder 6.7B model, optimize GPU drivers and memory, integrate with VS Code, and monitor resource usage in real time.
DeepSeek CoderGPU optimizationOllama
0 likes · 5 min read

This guide details how switching from vLLM to SGLang on eight NVIDIA H800 GPUs increased Llama‑3‑70B‑Instruct throughput from 180 to 420 tokens per second, covering SGLang’s core innovations, environment setup, configuration tweaks, performance benchmarks, troubleshooting tips, and production‑grade deployment scripts.
FlashInferGPU optimizationH800
0 likes · 19 min read

This guide walks through the complete process of deploying a high‑throughput large language model inference service using vLLM, covering environment preparation, installation, configuration tuning, performance testing, real‑world case studies, monitoring, troubleshooting, and backup strategies for production‑grade deployments.
GPU optimizationLLM inferenceOpenAI API
0 likes · 44 min read

FlashAttention accelerates transformer inference by combining nine techniques—including loss‑less attention, GPU memory‑pyramid optimization, SRAM‑reusing tiling, safe softmax scaling, online buffering, tile‑size constraints, parallel multiplication, reduced KV slicing, and integrated backward‑pass caching—to achieve efficient, high‑throughput computation on modern GPUs.
Attention MechanismFlashAttentionGPU optimization
0 likes · 8 min read

TTT‑Video‑Dit is an open‑source framework that uses test‑time‑training and hierarchical attention to generate coherent 63‑second videos with style‑transfer, dramatically reducing GPU memory requirements so a single RTX 4090 can replace costly H100 clusters, enabling creators and developers to produce long AI videos efficiently.
GPU optimizationTTT-Video-Ditlong video AI
0 likes · 11 min read

This article details the cost and speed challenges of embedding vectors in large‑scale log scenarios, analyzes inference framework choices, describes GPU utilization, priority queuing, and pipeline redesigns, and reports a 16‑fold throughput increase and dramatically lower per‑request costs.
EmbeddingGPU optimizationTriton
0 likes · 8 min read

This article explains why certain GPU performance gains require direct CUDA kernel edits and walks through fine‑grained techniques such as data‑layout restructuring, warp‑level primitives, tiled memory accesses, kernel fusion, and dynamic execution paths, backed by code examples and benchmark insights.
CUDAGPU optimizationMemory Layout
0 likes · 12 min read

The paper "Efficient Long Context Fine-tuning with Chunk Flow" introduces ChunkFlow, a training framework that reorganizes variable‑length sequences into fixed‑size chunks, achieving up to 4.53× speedup and more stable GPU memory usage for large language models.
ChunkFlowGPU optimizationLLM training
0 likes · 7 min read

This article reviews the SpargeAttn paper, describing how a training‑free sparse attention mechanism achieves 4‑7× inference speedup across language, video, and image models while preserving end‑to‑end accuracy, and outlines its challenges, algorithmic solutions, implementation details, and experimental results.
GPU optimizationModel AccelerationQuantized Inference
0 likes · 7 min read

DeepSeek’s Open‑Source Week unveiled five high‑performance projects—FlashMLA, DeepEP, DeepGEMM, DualPipe/EPLB, and 3FS—each delivering novel GPU optimizations, communication kernels, matrix‑multiplication libraries, parallelism strategies, and a distributed file system that together dramatically accelerate large‑scale AI training and inference workloads.
AI accelerationDeepSeekGPU optimization
0 likes · 9 min read

During DeepSeek’s Open‑Source Week (Feb 24‑28), five production‑tested projects were released, spanning GPU‑optimized MLA kernels, MoE communication libraries, high‑performance FP8 GEMM, dual‑pipeline parallelism, and a AI‑focused distributed file system, each delivering significant performance and efficiency gains for large‑scale AI workloads.
AIGPU optimizationdistributed training
0 likes · 13 min read


AIWalkerFeb 27, 2025 · Artificial Intelligence
This article provides a comprehensive, hands‑on guide for installing and configuring DeepSeek‑R1 with Ollama and vLLM, setting up multi‑node multi‑GPU environments, running performance benchmarks, optimizing runtime parameters, and even generating a full PyTorch distributed‑training script.
DeepSeek-R1GPU optimizationLLM Deployment
0 likes · 39 min read

The article analyzes DeepSeek’s three open‑source projects—FlashMLA, DeepEP, and DeepGEMM—showing how they optimize for the China‑only NVIDIA H800 GPU, contrast this with the abundant hardware resources of Western AI firms, and highlight the growing demand for talent that masters both AI models and GPU hardware.
AI hardwareDeepEPDeepGEMM
0 likes · 7 min read


AIWalkerFeb 25, 2025 · Artificial Intelligence
Sliding Tile Attention (STA) replaces costly full‑3D attention in video DiT models with a block‑wise sliding‑window scheme, achieving up to 10× attention speedup and a 3.53× end‑to‑end generation boost for HunyuanVideo without quality loss, as demonstrated by extensive benchmarks and kernel analyses.
GPU optimizationHunyuanVideoSliding Tile Attention
0 likes · 16 min read

Flash-MLA is an open‑source GPU kernel optimized for Nvidia Hopper GPUs that compresses the KV cache of multi‑head attention, cutting memory usage by up to 93.3% and delivering 580 TFLOPS compute, thereby dramatically accelerating large‑language‑model inference while lowering cost.
DeepSeekFlash-MLAGPU optimization
0 likes · 8 min read

The DeepSeek paper on Native Sparse Attention (NSA) presents a hardware‑aligned, trainable sparse‑attention architecture that slashes O(n²) costs, delivers up to 11.6× speedups and 2.3‑point accuracy gains on long‑context benchmarks, and reduces training expense by 47% while scaling to 64k tokens.
DeepSeekGPU optimizationLong-context modeling
0 likes · 11 min read

This article explains the FlashAttention algorithm, its memory‑efficient tiling and recomputation techniques, and how enabling the flash_attn flag dramatically speeds up Qwen‑series large models while outlining hardware, software requirements and potential trade‑offs.
AttentionFlashAttentionGPU optimization
0 likes · 8 min read

This guide outlines the hardware requirements for AI models ranging from lightweight 1.5 B parameters to massive 671 B models, detailing CPU cores, memory, GPU recommendations, storage needs, optimization tips, deployment suggestions, and suitable application scenarios.
AI hardwareDeepSeekGPU optimization
0 likes · 5 min read

This article presents NVIDIA's NeMo technology stack for end‑to‑end large language model (LLM) training, covering the full software pipeline, model alignment with reinforcement learning from human feedback (RLHF), performance optimizations such as model parallelism, FP8, TensorRT‑LLM inference, dynamic load balancing, and future research directions.
GPU optimizationLLMNeMo
0 likes · 24 min read


JavaEdgeNov 20, 2024 · Artificial Intelligence
The article explains why deploying large language models is challenging and presents seven practical techniques—including defining deployment boundaries, model quantization, inference optimization, infrastructure consolidation, model replacement planning, GPU utilization, and using smaller models—to make LLM deployment more efficient and cost‑effective.
GPU optimizationLLM DeploymentModel Scaling
0 likes · 24 min read

The 5th China Cloud Computing Infrastructure Developer Conference in Beijing highlighted cutting‑edge AI inference optimization, Knative‑based serverless acceleration, AMD PMU virtualization, and CDI‑driven GPU management, offering detailed technical insights and real‑world case studies that illustrate how cloud providers are tackling performance and cost challenges of modern workloads.
AI inferenceAMD virtualizationGPU optimization
0 likes · 9 min read

This article details a comprehensive engineering practice for optimizing AI inference services at ZhiZhuan, covering background analysis, selection of TorchServe over alternatives, GPU/CPU performance tuning, custom handlers, Torch‑TRT integration, and deployment on Kubernetes, with measured improvements in throughput and resource utilization.
AI inferenceGPU optimizationKubernetes
0 likes · 16 min read

This article details the performance‑focused optimizations applied to TorchRec, PyTorch's large‑scale recommendation system library, including CUDA graph capture, multithreaded kernel launches, pinned memory copies, and input‑distribution refinements that together achieve a 2.25× speedup on MLPerf DLRM‑DCNv2 across 16 DGX H100 nodes.
CUDA GraphGPU optimizationPyTorch
0 likes · 11 min read

The article details JD Advertising's technical challenges and solutions for large‑scale sparse recommendation models, describing GPU‑focused storage, compute and I/O optimizations for both training and low‑latency inference, including distributed pipelines, heterogeneous deployment, batch aggregation, multi‑stream execution, and compiler extensions.
GPU optimizationSparse ModelsTensorFlow
0 likes · 13 min read

The article presents three RLHF‑PPO acceleration techniques—TRT‑LLM‑based text generation speedups, selective activation recomputation with sequence parallelism for dynamic memory reduction, and overlapping pipeline stages for system‑level parallelism—demonstrating a 350 % throughput boost on a 10 B model using 16 A100 GPUs.
GPU optimizationPPO optimizationPerformance Tuning
0 likes · 16 min read

The RECom compiler introduces a subgraph‑parallel fusion technique and symbolic shape handling to dramatically speed up GPU inference of deep recommendation models with massive embedding columns, achieving up to 6.61× lower latency and 1.91× higher throughput than TensorFlow baselines, while eliminating redundant computations.
CompilerGPU optimizationembedding columns
0 likes · 10 min read

The PaddlePaddle Neural Network Compiler (CINN) combines a PIR‑based frontend and a hardware‑specific backend to apply graph‑level optimizations, operator fusion, schedule transformations and automatic tuning, delivering up to 4× faster kernels and 30‑60% overall speed‑ups for deep‑learning and scientific workloads.
CINNGPU optimizationOperator Fusion
0 likes · 19 min read


58UXDJun 13, 2024 · Artificial Intelligence
This article explains what ComfyUI is, how its node‑based workflow mirrors the underlying Stable Diffusion architecture, and why it outperforms WebUI in speed, GPU usage, real‑time preview, and workflow reuse, while also offering practical tips for new users.
AI image generationComfyUIGPU optimization
0 likes · 9 min read


JD TechMar 18, 2024 · Artificial Intelligence
The article describes how JD’s advertising team tackled the high‑concurrency, low‑latency challenges of online recommendation inference by designing a distributed graph heterogeneous computing framework, optimizing GPU kernel launches with TensorBatch, deep‑learning compiler techniques, and a multi‑stream GPU architecture, achieving significant throughput and latency improvements.
AI inferenceDeep Learning CompilerGPU optimization
0 likes · 14 min read

PAI‑TorchAcc, an Alibaba Cloud AI platform accelerator, offers a seamless PyTorch interface that integrates HuggingFace models and employs LazyTensor‑based static graph conversion, multi‑strategy distributed training, and extensive GPU optimizations to dramatically boost throughput for 1B‑175B parameter models, surpassing PyTorch native and Megatron‑LM performance.
AI accelerationAlibaba CloudGPU optimization
0 likes · 13 min read

The second Baidu Search Innovation Contest attracted over 2,800 participants from 45 regions, featured five AI‑focused tracks, and highlighted champion teams that employed techniques such as Lora‑fine‑tuned LLMs, vector‑intersection Top‑K search, GPU‑optimized algorithms, and diffusion‑based image generation to push the boundaries of search technology.
AI competitionGPU optimizationLLM fine-tuning
0 likes · 12 min read

This article explains how PAI‑Blade uses compiler optimizations, TorchDynamo, MHLO conversion, and aggressive kernel fusion to accelerate PyTorch training, provides simple two‑line integration code, showcases benchmark results on A10 and A100 GPUs, and details deployment steps on PAI‑DSW.
BladeDISCGPU optimizationModel Acceleration
0 likes · 8 min read

This article details Xiaohongshu's end‑to‑end GPU‑centric transformation for search‑related machine‑learning models, covering model characteristics, training and inference frameworks, system‑level GPU and CPU optimizations, multi‑card and compilation techniques, and future directions for scaling large sparse and dense models.
GPU optimizationModel ServingXiaohongshu
0 likes · 16 min read

Baidu's Search Architecture team details how its deep‑learning models have evolved to deliver direct answer results via semantic embeddings, describes a massive online inference pipeline that rewrites queries, ranks relevance, and classifies types, and outlines optimization techniques—including data I/O, CPU/GPU balancing, pruning, quantization, and distillation—to achieve high‑throughput, low‑latency search.
BaiduGPU optimizationInference System
0 likes · 13 min read

Megatron-LLaMA is an open‑source high‑performance training framework for LLaMA models, offering tensor, pipeline, and sequence parallelism, an overlapped optimizer, and near‑linear scalability, achieving up to 176% speedup on 32 GPUs and robust performance even with limited network bandwidth.
DeepSpeedGPU optimizationLlama
0 likes · 10 min read

Xiaohongshu transformed its recommendation, advertising, and search inference pipeline by migrating to GPU‑centric hardware, deploying a custom TensorFlow‑Core Lambda service, and applying system‑level, virtualization, and compute‑level optimizations—including NUMA binding, kernel fusion, dynamic scaling, and FP16 quantization—achieving roughly 30× compute capacity growth, over 10% user‑metric gains, and more than 50% cluster‑resource savings.
GPU optimizationMachine Learning InferenceSystem Performance
0 likes · 20 min read

This article explains how Baidu's intelligent cloud overcomes the compute and storage walls of large‑scale model training by combining hardware design, network topology, and software optimizations such as pipeline, tensor, and expert parallelism, cost‑model‑driven placement, and future‑proof AI infrastructure evolution.
AI infrastructureBaidu CloudGPU optimization
0 likes · 28 min read

This article details how to use NVIDIA profiling tools, mixed‑precision training, operator fusion, kernel optimizations, and INT8 quantization to identify and eliminate performance bottlenecks in Swin Transformer models, achieving up to 2.85× training speedup and up to 7.34× inference acceleration on modern GPUs.
AI performanceGPU optimizationOperator Fusion
0 likes · 23 min read

BladeDISC 0.3.0 introduces full PyTorch 2.0 compilation support, new TorchDynamo optimizations, extensive GPU memory‑intensive compute enhancements, Shape Constraint IR, experimental quantization across multiple hardware platforms, and a suite of compiler‑level improvements for training and inference acceleration.
BladeDISCCompilerGPU optimization
0 likes · 11 min read

The paper analyzes why Alibaba Mama’s intelligent creative video service suffers low GPU utilization—due to Python GIL blocking, lack of kernel fusion, and serialized CUDA streams—and details service‑level changes (separate CPU/GPU processes, shared‑memory queues, priority scheduling) and operator‑level kernel‑fusion techniques (channels‑last layouts, custom pooling, TensorRT conversion) that raise utilization from ~30 % to near 100 % and boost throughput by 75 %.
GPU optimizationMultithreadingPython
0 likes · 20 min read

PICASSO, Alibaba’s GPU‑centric sparse training engine for wide‑and‑deep recommender systems, merges identical embedding tables, interleaves data and kernel operations, and caches hot embeddings on GPU, eliminating the parameter server and delivering up to tenfold speedups over TensorFlow‑PS while maintaining model quality.
AlibabaGPU optimizationmachine learning
0 likes · 14 min read

Alibaba Cloud’s PAI team and the DAMO Academy introduced the low‑carbon M6 trillion‑parameter multimodal model, demonstrating that their self‑developed Whale framework can train such massive models on just 480 V100 GPUs, reducing energy consumption by over 80% and boosting training efficiency nearly eleven‑fold.
AIGPU optimizationWhale framework
0 likes · 12 min read

This article explains how Tencent's Light platform reduced the massive overhead of feature hashing in ad‑ranking by moving integer‑to‑string conversion and hash computation to the GPU, introducing custom contiguous string tensors, and achieving up to 12× speed‑up on V100 GPUs.
GPU optimizationPerformance TuningTensorFlow
0 likes · 14 min read

QuanTaichi introduces a new language abstraction and compiler system that quantizes simulation data, dramatically reducing memory and bandwidth usage so that high‑precision physical effects—once requiring multiple GPUs—can now run on a single GPU, even on mobile devices.
CompilerGPU optimizationGraphics
0 likes · 12 min read

This article explains how the MNN deep‑learning framework leverages OpenCL to achieve high‑performance inference on mobile, PC and embedded GPUs by diversifying memory objects, aligning data, using local‑memory reductions, selecting optimal work‑group sizes, applying pre‑inference auto‑tuning, caching compiled programs, and providing practical GPU‑friendly model design guidelines.
GPU optimizationMNNOpenCL
0 likes · 20 min read

This article describes the challenges of deploying a large BERT model as an online service for 360 Search and details engineering optimizations—including framework selection, model quantization, knowledge distillation, stream scheduling, caching, and dynamic sequence handling—that dramatically improve latency, throughput, and resource utilization.
BERTFP16 quantizationGPU optimization
0 likes · 12 min read

The article explains why lowering the frame rate of iOS animations can trade a slight loss in visual smoothness for significant GPU load reduction, describes the Core Animation rendering pipeline, compares different frame‑rate reduction techniques, and presents test results showing the impact on CPU, GPU, and overall app performance.
CADisplayLinkGPU optimizationcore animation
0 likes · 11 min read

By moving decoding, color conversion, preprocessing, inference, and re‑encoding entirely onto the GPU and enabling batch processing with flexible Python scripts, iQIYI’s video‑image enhancement service achieved ten‑fold throughput, over 90 % GPU utilization, and dramatically lower resource use, accelerating AI video inference deployment.
AI DeploymentDeepStreamGPU optimization
0 likes · 14 min read
