Infra Learning Club shares study notes, cutting-edge technology, and career discussions.
The article investigates why Nvidia L40S and RTX A6000 GPUs show less memory via nvidia‑smi than their advertised 48 GB, revealing that enabled ECC memory reserves a few gigabytes, and demonstrates the effect by toggling ECC on a Tesla‑T4 card.
This guide walks you through installing Ollama, pulling the open‑source DeepSeek R1 model, and using LangChain and Streamlit to create a locally hosted Retrieval‑Augmented Generation (RAG) system that can answer questions from uploaded PDFs without any cloud API.
The article explains that DeepSeek is an open‑source LLM that many sellers monetize by offering paid installation packages, outlines the model lineup and size options, and provides a step‑by‑step guide to install and run DeepSeek locally with Ollama and Open WebUI.
The article surveys the rise of multi‑agent LLM systems, detailing how specialized agents collaborate on tasks such as travel planning, outlining their workflow, comparing them with single‑agent models, listing prominent frameworks, and discussing current challenges and research citations.
This article explains what LLM agents are, their core components—brain, memory, planning, and tool use—illustrates how they handle complex queries through task decomposition, surveys notable frameworks, and discusses key challenges such as limited context, long‑term planning difficulties, output inconsistency, and prompt dependence.
This guide walks through the fundamentals of Huawei Ascend NPU hardware, the CANN software stack, driver and firmware installation, Kubernetes integration via Docker runtime and device plugin, and a complete ResNet‑50 inference demo on Ascend 310P.
This article walks through a practical APOD workflow for CUDA development—assessing bottlenecks, parallelizing with cuBLAS/cuFFT/Thrust, optimizing iteratively, and deploying—while covering nvcc compilation flags, PTX virtual ISA, nvprof profiling, core terminology (SP, SM, warp, grid, block, thread), indexing patterns, and unified memory references.
The article dissects NVCC’s compilation pipeline, showing how internal registration functions from host_runtime.h are injected into the host binary, how a simple CUDA demo is processed with --dryrun, and how the generated fatbin, PTX, and cubin files are linked and registered for GPU execution.
This article introduces GPU remote invocation, explains rCUDA's architecture, walks through installing the server and client, demonstrates running CUDA samples on a GPU‑less node, and shows how to deploy rCUDA on Kubernetes with example DaemonSet and Job manifests.
The article compares user‑mode and kernel‑mode GPU virtualization, detailing their layered architectures, how they intercept APIs, the advantages such as openness, isolation, and unified memory, and the drawbacks including API complexity, kernel intrusion, legal risks, and cross‑process limitations.
