Mojo 1.0, the production‑ready language created by Chris Lattner, promises a unified, open‑source AI stack that combines Python simplicity, Rust safety, and C‑level performance via MLIR, breaking CUDA lock‑in and supporting any GPU, CPU or ASIC.
This article breaks down NVIDIA GPU fundamentals—contrasting GPU with CPU design, tracing CUDA’s evolution, detailing the hardware hierarchy from chips to SM units, explaining memory tiers, and presenting a step‑by‑step performance‑optimization methodology for AI training and inference workloads.
This article explains NVIDIA's CUDA platform, its CPU‑GPU division of labor, how massive parallelism accelerates AI large‑model training, and walks through a typical CUDA program workflow with concrete code examples and diagrams.
A CUDA/HIP‑based 3‑D fluid simulation was ported to an Apple M3 Pro device via a Clang‑HIP → SPIR‑V → Vulkan → MoltenVK conversion pipeline, achieving roughly ten‑fold speedup without any Metal‑specific API changes and preserving numerical accuracy.
This article provides a step‑by‑step, Linux‑focused workflow for recording driver and GPU versions, interpreting utilization versus memory metrics, locating memory‑consuming processes, handling container and Kubernetes mappings, checking temperature, power, ECC, MIG, driver health, OOM conditions, and setting up reliable monitoring and alert thresholds for data‑center GPUs.
Qualcomm announced a $3.9 billion all‑stock purchase of AI infrastructure software firm Modular, whose cross‑hardware MAX inference engine and Mojo language aim to fill Qualcomm’s AI software shortfall, reduce reliance on CUDA, and support a broader cloud‑to‑edge AI ecosystem.
This guide shows how to enable NVIDIA GPU passthrough in WSL 2, install the CUDA toolkit, set up a PyTorch GPU environment, verify GPU visibility, and run real‑world AI/ML workloads such as LLM inference, YOLO object detection, and Jupyter monitoring, while providing performance comparisons, optimization tips, and troubleshooting FAQs.
This article walks through implementing a row‑wise softmax kernel in Triton, explaining program‑instance mapping, block‑size selection, mask handling, pointer arithmetic, resource‑usage analysis, and a RTX 5090 benchmark that reveals performance cliffs compared to PyTorch.
The RTX Spark desktop processor delivers 1 PFLOP of AI compute—about 14 times the M5 Ultra—while the M5 Ultra retains a three‑times higher memory bandwidth and twice the memory capacity, making it superior for certain inference workloads; the article breaks down specs, benchmarks, ecosystem differences, pricing and market positioning to show how each platform fits distinct AI use cases.
CODA rewrites Transformer blocks as GEMM‑epilogue programs, exposing five primitive building blocks that let both AI‑generated code and human programmers fuse memory‑intensive operations into the GEMM epilogue, eliminating costly tensor moves and achieving up to 1.8× speed‑ups on H100 GPUs for RMSNorm, SwiGLU, RoPE and other components, while preserving numerical accuracy.
The paper shows that applying lightweight L1 regularization can make over 99% of FFN activations zero, and by using a new tile‑wise ELLPACK (TwELL) format together with a hybrid routing scheme, inference speed improves up to 30% while memory usage drops over 24% and energy consumption is reduced, all with negligible impact on downstream task performance.
A new ICML 2026 paper by Sakana AI and NVIDIA shows that applying lightweight L1 regularization can make Feed‑Forward Network activations in Transformers over 99% sparse, and with the TwELL storage format and a hybrid routing scheme this sparsity translates into up to 20.5% inference speedup, 21.9% training‑step acceleration, lower energy consumption and reduced peak memory across 0.5‑2 B‑parameter models while preserving downstream performance.
The article examines Nvidia’s competitive moat beyond CUDA, detailing how its token‑factory model, extensive supply‑chain commitments, and a flexible accelerator ecosystem contrast with Google’s TPU ASIC approach, while also exploring the impact of AI agents on future compute demand.
This article analyzes why vLLM's PagedAttention can cause GPU memory fragmentation and out‑of‑memory errors in production, presents four typical OOM scenarios, and provides concrete diagnostics, configuration tweaks, code examples, and monitoring strategies to eliminate the problem.
This article explains NVIDIA Jetpack’s three‑layer architecture, its relationship with the SDK Manager installer, step‑by‑step flashing procedures for Jetson devices, common failure points such as the 35.29% stall, and practical troubleshooting and hybrid manual‑automatic solutions.
This article traces three decades of government‑funded research in parallel computing, graphics systems, and stream processing, showing how those advances migrated to companies like Nvidia, evolved into CUDA and other GPU technologies, and ultimately enabled today’s AI revolution.
The article analyzes InCoder-32B Thinking, an industrial‑code large language model that incorporates error‑driven chain‑of‑thought and an Industrial Code World Model to predict execution outcomes, adapt reasoning depth, and achieve high accuracy across diverse hardware‑centric benchmarks.
The article analyzes InCoder-32B Thinking, an industrial code large model that integrates error‑driven chain‑of‑thought and a world‑model to predict real‑system outcomes, showing high accuracy on diverse benchmarks and demonstrating adaptive reasoning depth for tasks ranging from Verilog synthesis to CUDA kernel optimization.
This article analyzes the AVO system, an autonomous AI agent that replaces traditional evolutionary search pipelines to iteratively improve CUDA attention kernels on NVIDIA's Blackwell B200 GPU, achieving up to 10.5% higher throughput than hand‑tuned implementations after a week of nonstop optimization.
The GTC 2026 keynote revealed NVIDIA’s latest DLSS 5 technology using 3‑D guided neural rendering to deliver cinematic‑quality graphics in real time, outlined a 20‑year CUDA ecosystem flywheel that fuels AI acceleration across structured and unstructured data, showcased enterprise case studies like Nestlé’s data‑refresh breakthrough, and highlighted a vast partner network, illustrating how AI is moving from experimental labs to everyday production.
CUDA Agent, a large‑scale reinforcement‑learning system from ByteDance and Tsinghua, can automatically generate and optimize CUDA kernels that outperform torch.compile by up to 2× on simple kernels and achieve around 40% higher speed than proprietary models on the hardest benchmarks, while detailing its data‑synthesis pipeline, training workflow, and current limitations.
ByteDance and Tsinghua University have created the CUDA Agent, an AI compiler that automatically writes and optimizes GPU kernels, delivering up to double the performance, and heralding a shift where AI‑generated low‑level code could reshape the hardware‑software competition landscape.
Ubuntu 26.04 LTS adds automatic detection and installation of NVIDIA CUDA or AMD ROCm drivers and introduces pre‑configured Inference Snaps sandbox containers, building on the AI groundwork laid by 24.04 LTS to dramatically lower the setup barrier for local AI development.
The article walks through a step‑by‑step optimization of a simple elementwise addition kernel (C = A + B) on HyperAI's RTX 5090 cloud instance, covering FP32 baseline, vectorized FP32, several FP16 variants, benchmark methodology, performance results, and the reasoning behind thread‑block sizing.
PyTorch 2.10 introduces extensive upgrades for AMD ROCm, Intel XPU, and NVIDIA CUDA, adds new Torch XPU APIs, expands Python 3.14 support, and brings performance‑focused improvements such as fused kernels and enhanced quantization, all available via the official GitHub release.
This article walks Go developers through the fundamentals of GPU architecture and CUDA, demonstrates a complete CGO‑based matrix‑multiplication project, offers performance‑tuning tips such as minimizing PCIe transfers and leveraging shared memory, and presents a PureGo alternative for seamless Go‑GPU integration.
The article analyzes NVIDIA's newly announced cuTile, a tile‑based Python DSL for GPU kernels, examining its technical differences from CUDA's SIMT model, its potential to reshape the GPU programming ecosystem, community reactions, competition with Triton, and the uncertain future that hinges on ecosystem maturity and migration tools.
The article argues that traditional storage‑centric computer science curricula are becoming obsolete as AI‑driven, compute‑centric paradigms dominate hardware, data‑center operations, and software ecosystems, urging universities and students to rapidly adopt new teaching focus and skills.
This guide walks you through configuring OLLAMA for multi‑GPU load balancing, covering hardware checks, CUDA and Docker setup, native and containerized deployment methods, core parameter tuning, advanced sharding, dynamic monitoring, troubleshooting, production best practices, and a real‑world RTX 4090 case study.
NVIDIA’s CUDA Toolkit 13.1 introduces the CUDA Tile programming model, Green Context resource partitioning, upgraded developer tools, and accelerated math libraries, delivering major performance gains for AI and matrix workloads on the new Blackwell GPUs.
NVIDIA's CUDA Toolkit 13.1 introduces the groundbreaking CUDA Tile programming model, green context support, enhanced math libraries, and numerous performance improvements for AI and GPU workloads, while also adding new developer tools, MPS features, and deterministic options for CUB.
This article explains the fundamentals of CUDA performance tuning, covering GPU architectures from Kepler to Volta, the role of SMX, warp schedulers, registers and memory hierarchies, and provides practical guidance on launch configuration, latency hiding, and thread‑block sizing to maximize throughput.
Nvidia announced native Python support for its CUDA toolkit, detailing new Python‑centric APIs, projects like CuTile and Cutlass, and a layered strategy that democratizes GPU programming for AI developers while preserving performance and expanding the ecosystem.
This article provides a comprehensive overview of CUDA function type annotations—including __global__, __device__, __host__, combined annotations, and memory‑space qualifiers—explains their purposes, characteristics, and syntax, demonstrates practical examples, offers best‑practice guidelines, highlights common pitfalls, and introduces advanced topics such as dynamic parallelism and cooperative groups.
The article explains how bpftime extends eBPF to NVIDIA and AMD GPUs, exposing fine‑grained execution details that traditional CPU‑side tools miss, and demonstrates a unified, programmable observability stack that overcomes the limitations of existing GPU profilers in both synchronous and asynchronous workloads.
This article walks through the DeepSeek‑V3.2‑Exp inference codebase, detailing its MoE architecture, Multi‑Head Latent Attention, FP8 quantization, custom CUDA kernels, and 8‑GPU NCCL‑based distributed execution from initialization through prefill and decode stages.
This guide explains why Java struggles with high‑performance or data‑intensive workloads, introduces GPU acceleration with CUDA, compares integration options such as JCuda, JNI, and JNA, walks through a practical encryption use case with performance benchmarks, and provides production‑grade best practices for memory, threading, testing, security, and deployment.
This article examines how to achieve low‑latency network packet processing on NVIDIA GPUs by comparing CPU and GPU implementations, exploring memory optimizations, batch strategies, stream concurrency, persistent kernels, and CUDA graphs, and presenting detailed performance measurements for each technique.
This guide walks through configuring Windows 11 with WSL2 and Ubuntu 22.04 for GPU‑accelerated CUDA development, installing NVIDIA drivers and CUDA libraries, setting up SSH and firewall rules, running a CUDA stress‑test program, and using Nsight Systems, Nsight Compute, and NVIDIA DCGM for performance profiling and monitoring.
This guide walks you through configuring a high‑performance AI inference server on Oracle Linux, covering hardware specs, NVIDIA driver and CUDA installation, Conda environment setup, Xinference deployment, service startup, and example model loading commands, all with clear code snippets and images.
This article explains why certain GPU performance gains require direct CUDA kernel edits and walks through fine‑grained techniques such as data‑layout restructuring, warp‑level primitives, tiled memory accesses, kernel fusion, and dynamic execution paths, backed by code examples and benchmark insights.
This article examines NVIDIA's Kepler GPU architecture, highlighting its 28 nm process, increased transistor count, expanded CUDA core count, PCIe 3.0 support, enhanced memory hierarchy, new compute units, scheduling improvements like Hyper‑Q, and performance metrics of the Tesla K20X, illustrating the substantial gains over previous generations.
The article explains how replacing NumPy with the GPU‑compatible CuPy library can dramatically accelerate array computations, walks through installation prerequisites, demonstrates benchmark scripts showing up to ten‑fold speed improvements, discusses data type effects, custom kernels, and hybrid CPU‑GPU workflows for large‑scale data processing.
This article walks through GPU performance analysis, starting with CUDA architecture fundamentals, demonstrating matrix multiplication optimization, explaining PTX assembly, and introducing the Neutrino framework for programmable GPU probes that enable fine‑grained, custom instrumentation and detailed timing measurements of kernel execution.
This article explains how NVIDIA transitioned from gaming graphics cards to general‑purpose GPUs with the first data‑center Tesla GPU, detailing the unified shader architecture, the internal components of TPCs and SMs, CUDA 1.0 programming basics, and performance calculations that illustrate the massive computational advantage over contemporary CPUs.
This article traces the evolution of NVIDIA GPU architectures—from the early Tesla series through Fermi, Kepler, Maxwell, Pascal, Volta, Turing, Ampere, Hopper, and up to the upcoming Blackwell—explaining their hardware innovations, CUDA programming model, and how each generation enabled breakthroughs in high‑performance computing, deep learning, and AI applications.
This guide walks you through configuring OLLAMA for multi‑GPU load balancing, covering hardware checks, CUDA setup, native and Docker deployment methods, detailed parameter tuning, advanced sharding strategies, troubleshooting, performance optimization, and production‑grade monitoring to maximize throughput and stability of large language models.
This guide walks you through a complete workflow for profiling CUDA applications, covering GPU performance fundamentals, key metrics, NVIDIA Nsight tools, CUPTI programming, example code, common bottlenecks, and best‑practice recommendations to identify and eliminate performance limits.
Huawei is developing a new AI‑focused GPU‑style chip that mirrors Nvidia and AMD architectures, aiming to ease Chinese developers’ shift from Nvidia hardware, but still faces software compatibility hurdles due to reliance on CUDA and ongoing U.S. export restrictions.
This article introduces OpenAI's Triton language, compares its block‑wise programming model to traditional CUDA, walks through vector‑addition and fused‑softmax kernel implementations, and presents benchmark results that demonstrate significant speedups over native PyTorch operations.
Huawei is developing a new AI chip that functions more like a general‑purpose GPU, aiming to match Nvidia and AMD architectures and simplify the transition for Chinese AI developers, while still facing challenges such as adapting CUDA‑based software and overcoming export restrictions.
This article explains how modern GPUs evolved from graphics rendering to general‑purpose computing, details the CPU‑GPU heterogenous architecture, walks through a CUDA demo that adds two billion‑element arrays, compares CPU and GPU performance, and describes the compilation, loading, and execution pipeline of CUDA kernels.
This article explores how GPUs evolved from graphics accelerators to general‑purpose processors for AI, detailing the CPU‑GPU heterogeneous architecture, the CUDA programming workflow, compilation into fat binaries, kernel launch mechanics, hardware components, and the differences between SIMD and SIMT models, with performance comparisons and code examples.
This guide explains why Java architects should understand CUDA, describes the GPU programming model, compares CPU and GPU designs, and details three practical ways—JNI, JCuda, and TornadoVM—to integrate CUDA acceleration into Java applications, with tips for using A100 GPUs effectively.
This tutorial explains how to build an eBPF‑based tracing tool that intercepts CUDA runtime API calls via uprobes, captures detailed event data such as memory sizes, transfer directions, kernel launches and errors, and presents it in a readable format for debugging and performance analysis.
The article explains why standard GEMM libraries may fall short for special matrix shapes, introduces NVIDIA’s open‑source CUTLASS library, details its hierarchical tiling architecture, and walks through a complete device‑API example that customizes tile sizes and data layouts to achieve near‑hand‑written kernel performance on modern GPUs.
Tencent’s HunyuanVideo-Avatar converts a static portrait and an audio clip into a lip‑synced, expressive video using a multimodal diffusion Transformer, offering open‑source weights, detailed module designs, hardware requirements, code examples, and a candid assessment of its strengths and current limitations.
NVIDIA announced at GTC 2025 that CUDA will now natively support Python, allowing developers to write GPU‑accelerated code directly in Python without C/C++ knowledge, introducing new APIs, libraries, JIT compilation, performance tools, and a tile‑based programming model that aligns with Python’s array‑centric workflow.
NVIDIA announced native Python support in its CUDA toolkit, enabling developers to write GPU‑accelerated code directly in Python, detailing the new programming model, JIT‑based architecture, performance benefits, and the broader impact on AI development and the developer ecosystem.
This article analyzes the major AI system‑level compute platforms—NVIDIA's CUDA, Huawei's CANN, Cambricon's Neuware, and AMD's ROCm—examining their architectures, ecosystem support, performance features, compatibility layers, and how they shape the AI chip market.
This article presents a comprehensive benchmark of DeepGEMM, Cutlass, and Triton on NVIDIA H20 and H800 GPUs, analyzing TFLOPS, bandwidth, latency, and speedup across various matrix sizes, and concludes which library is optimal for different workload scenarios.
This step‑by‑step tutorial shows how to set up CUDA 12.4, install required packages, prepare a JSON dataset and custom reward, troubleshoot out‑of‑memory errors, and launch DeepSeek R1 training on an 8‑GPU A100 cluster using Accelerate, Deepspeed zero‑3 and vLLM configurations.
This article explains the cuda‑python ecosystem—including its core packages, installation via pip or conda, the experimental cuda.core API, a full Python‑to‑CUDA workflow with NVRTC compilation, performance comparison to C++, the covered APIs, and an overview of NVIDIA's new cuTicle programming model.
The article reviews Nvidia GTC 2025’s session on writing CUDA kernels with Python, compares tools such as Numba, CuPy, PyTorch extensions and cuda‑python, demonstrates a segmented reduction example with C++ and Python code, explains the underlying CUDA concepts, and shows how to install and use cuda‑python to simplify kernel development.
The article explains GPU fundamentals—from the end of Dennard scaling and why GPUs excel in parallel throughput, through CUDA programming basics like the SAXPY kernel and SIMT versus SIMD execution, to the evolution of the SIMT stack, modern scheduling, and a three‑step core architecture design.
The article explains CUDA’s architecture and SM basics, describes how the warp scheduler and dispatch units assign thread blocks to SMs, and concludes that external control cannot target a specific SM, while mentioning the NanoFlow intra‑device parallelism approach as a possible indirect optimization.
This article explains the origins, core functions, installation steps, and programming examples of NVIDIA’s Collective Communication Library (NCCL), detailing its role in multi‑GPU and multi‑node AI distributed training, topology discovery, path selection, channel search, and various collective communication operations.
This article explains why enabling NVCC's --fatbin-options -compress-all breaks remote GPU calls, describes the fatbin file layout, shows how to extract and analyze the binary with objcopy, and provides a step‑by‑step implementation of a decompression routine for both ELF and PTX sections.
This article walks through the NVCC compilation pipeline, explaining how CUDA source files are transformed into host and device binaries, detailing file extensions, compilation stages, command‑line options, intermediate artifacts, and the role of registration functions such as __nv_cudaEntityRegisterCallback and __sti____cudaRegisterAll.
This article walks through a practical APOD workflow for CUDA development—assessing bottlenecks, parallelizing with cuBLAS/cuFFT/Thrust, optimizing iteratively, and deploying—while covering nvcc compilation flags, PTX virtual ISA, nvprof profiling, core terminology (SP, SM, warp, grid, block, thread), indexing patterns, and unified memory references.
The article dissects NVCC’s compilation pipeline, showing how internal registration functions from host_runtime.h are injected into the host binary, how a simple CUDA demo is processed with --dryrun, and how the generated fatbin, PTX, and cubin files are linked and registered for GPU execution.
This article introduces GPU remote invocation, explains rCUDA's architecture, walks through installing the server and client, demonstrates running CUDA samples on a GPU‑less node, and shows how to deploy rCUDA on Kubernetes with example DaemonSet and Job manifests.
The article compares user‑mode and kernel‑mode GPU virtualization, detailing their layered architectures, how they intercept APIs, the advantages such as openness, isolation, and unified memory, and the drawbacks including API complexity, kernel intrusion, legal risks, and cross‑process limitations.
This article introduces Triton—a Python‑based GPU programming language—covers essential GPU architecture, walks through practical kernels for vector addition, matrix multiplication, and rotary position encoding, compares performance with PyTorch, and provides debugging tips for high‑performance deep‑learning workloads.
Ant Group’s Computing Systems Lab announced that its GPU‑accelerated fully homomorphic encryption framework WarpDrive, which exploits Tensor and CUDA cores for high‑throughput NTT operations and parallel kernel designs, has been accepted as a paper at the IEEE HPCA 2025 conference.
This tutorial walks you through cloning the open‑source Roop AI face‑swap repository, setting up a conda environment, installing CUDA‑enabled PyTorch, configuring FFmpeg, and using the Tongyi Lingma AI coding assistant to explore code, resolve errors, and fine‑tune runtime parameters for successful video swapping.
In this interview, NVIDIA founder Jensen Huang shares his management philosophy, emphasizing the value of tackling difficult tasks, maintaining a small yet empowered team, avoiding layoffs, fostering a zero‑market mindset, navigating the early challenges of CUDA, and leveraging AI to drive future innovation.
This article introduces Reader, an open‑source Jina AI tool that converts any web URL into a format optimized for large language models, and llm.c, a minimalist C and CUDA project that demonstrates how to train a GPT‑2‑style LLM from scratch.
This article introduces Numba, a Python just‑in‑time compiler, explains why it is advantageous over alternatives, demonstrates how to apply its @jit, @njit, @vectorize and other decorators, and shows how to run accelerated code on CPUs and GPUs using CUDA.
Andrej Karpathy’s LLM.c project demonstrates how a compact, pure‑C (and CUDA) codebase of roughly 1000 lines can train a GPT‑2 model, covering data preparation, memory management, layer implementations, compilation, and practical tips for running and testing the model on CPUs and GPUs.
Andrej Karpathy released LLM.C, a pure C/CUDA implementation that trains GPT‑2‑style models in about 1,000 lines of code, detailing manual forward/backward passes, memory allocation tricks, SIMD CPU acceleration, CUDA porting, and migration tutorials, while comparing it to PyTorch and discussing broader LLM OS implications.
2023 GPGPU research framework analysis reveals that while Chinese GPUs like BR100 and TianGai100 can match or exceed NVIDIA A100 in FP32, they still lag in FP64 and INT8 performance, and the domestic software ecosystem based on OpenCL trails far behind NVIDIA's CUDA, shaping short‑and‑term market dynamics.
NVIDIA's new DOCA GPUNetIO library enables GPU‑initiated communication, allowing packets to be received directly into GPU memory, processed by CUDA kernels, and sent without CPU involvement, offering lower latency, higher scalability, and detailed pipeline examples including IP checksum, HTTP filtering, traffic forwarding, and 5G Aerial SDK integration.
The article presents the background, challenges, and objectives of Bilibili's AI services, introduces the self‑developed InferX inference framework with its quantization and sparsity optimizations, details OCR‑specific enhancements, and describes how integrating InferX with Nvidia Triton dramatically improves throughput, latency, and GPU utilization.
By isolating CPU preprocessing and post‑processing from GPU inference into separate processes and applying TensorRT’s FP16/INT8 optimizations, the custom Python framework boosts Python vision inference services from roughly 4.5 to 27.4 QPS—a 5‑10× speedup—while reducing GPU utilization and cost.
This article introduces Numba, a Just‑in‑Time compiler for Python that transforms functions into fast machine code using LLVM, explains why it lets you stay in pure Python, demonstrates basic @jit/@njit usage, advanced decorators, GPU execution with CUDA, and interoperability with C/C++ libraries.
This article introduces Numba, a Python Just-in-time compiler, explains why it is advantageous over alternatives, demonstrates how to apply its decorators such as @jit, @njit, @vectorize, and @cuda for CPU and GPU acceleration, and provides practical code examples and tips for optimal performance.
This article explains how to leverage server‑side GPUs for hardware‑accelerated H.264 encoding and decoding with FFmpeg, covering installation, key API calls, format conversion to OpenGL textures, multi‑process considerations, and performance optimizations for cloud‑rendered visual effects.
The Shopee Off‑Platform Ads team built a GPU‑accelerated Creative Rendering System that uses a four‑layer architecture, CGO‑bridged C/C++ kernels, and template caching to process billions of product images daily, achieving roughly ten‑fold speedup, half the cost, and far reduced rack space while handling high concurrency.
NVIDIA has finally open‑sourced its Linux GPU kernel driver, a landmark move that promises tighter OS integration, easier debugging, and broader support for Turing and Ampere GPUs, while also reshaping the relationship between proprietary drivers, the Nouveau project, and major Linux distributions.
The article walks through diagnosing why PyTorch cannot access the GPU, reinstalling CUDA, selecting matching PyTorch builds, adjusting versions, and verifying that CUDA becomes available for accelerated training.
This article provides a comprehensive overview of GPUs, covering their history, architecture, processing workflow, and acceleration technologies such as CUDA and OpenCL, while comparing GPU and CPU designs and offering resources for further study.
This guide walks you through installing NVIDIA drivers, CUDA, and nvidia-docker2 on a Linux host, configuring Docker to access the GPU, and verifying the setup with commands and sample TensorFlow/PyTorch code, enabling deep‑learning workloads inside containers.
This article explains how CuPy mirrors NumPy's API to run array and matrix operations on NVIDIA GPUs, providing step‑by‑step installation, code examples, and benchmark results that demonstrate speedups ranging from 10× to over 700× compared to CPU‑only NumPy.
This article introduces TensorRT, explains its deployment workflow from model training to engine generation, shows how to register custom operators for ONNX and create TensorRT plugins, and explores deformable convolution (DCN) implementation strategies for high‑performance AI inference.
The article presents a GPU virtual‑sharing solution for AI inference workloads that isolates memory and compute resources via CUDA API interception, integrates with Kubernetes using the open‑source aliyun‑gpushare scheduler, and demonstrates doubled GPU utilization and minimal performance loss across multiple tests.
iQIYI created a custom GPU‑virtual‑sharing system that intercepts CUDA calls to enforce per‑container memory limits, rewrites kernel launches for compute isolation, and integrates with a Kubernetes scheduler extender, allowing multiple AI inference containers to share a single V100 with minimal overhead and more than doubling overall GPU utilization.
This article introduces Numba, a Python just‑in‑time compiler, explains why it’s advantageous over alternatives, and provides detailed guidance on using its decorators such as @jit, @njit, @vectorize, and @cuda.jit, including code examples for CPU and GPU acceleration.
This article outlines the progression of NVIDIA GPU architectures—from the early Fermi and Kepler designs through Maxwell, Pascal, Volta, Turing, and the latest Ampere—detailing compute capabilities, SM structures, FP64/FP32 ratios, Tensor Core introductions, and their impact on AI and high‑performance computing.
This article provides a comprehensive overview of NVIDIA's GPU architecture evolution—covering Fermi, Kepler, Maxwell, Pascal, Volta, Turing, and Ampere—detailing compute capabilities, SM structures, specialized units such as Tensor Cores, and their impact on AI and high‑performance computing workloads.
