This guide walks through installing Anaconda, creating a Python 3.10 conda environment, adding PyTorch and ModelScope libraries, installing domain-specific dependencies, verifying NLP pipelines, and using PHPY to call ModelScope's OCR model from PHP, complete with code snippets and troubleshooting tips.
This article explains how to use autoencoders and a UNet architecture implemented in PyTorch to remove mosaic blocks from images, detailing the underlying principles, dataset preparation, network components, training procedure, and sample results.
This article explains the principle of LoRA (Low‑Rank Adaptation) for large language models, demonstrates how to decompose weight updates into low‑rank matrices, and provides a complete PyTorch implementation that fine‑tunes a small VGG‑19 network on a custom goldfish dataset.
Ant Group’s newly open‑sourced ATorch library extends PyTorch with a layered architecture and automated resource‑aware strategies, boosting large‑model training efficiency up to 60% utilization, enhancing stability, and delivering significant throughput gains across multi‑node, multi‑GPU deployments.
This article walks through the core LLaMA implementation in HuggingFace’s Transformers library, detailing the inheritance hierarchy, configuration defaults, model initialization, embedding and stacked decoder layers, the RMSNorm‑based attention and MLP modules, and the forward pass that produces normalized hidden states.
This article explains how PAI‑Blade uses compiler optimizations, TorchDynamo, MHLO conversion, and aggressive kernel fusion to accelerate PyTorch training, provides simple two‑line integration code, showcases benchmark results on A10 and A100 GPUs, and details deployment steps on PAI‑DSW.
This guide provides comprehensive techniques for optimizing PyTorch training performance and efficiency, covering all model types such as CNNs, RNNs, GANs, and transformers, and applicable across domains like computer vision and natural language processing, targeting AI/ML platform engineers, data engineers, backend developers, MLOps, SREs, architects, and machine learning engineers.
This guide explains how Ant Group uses Alluxio to overcome storage I/O, capacity, and latency challenges, delivering stability, performance, and scalability improvements for large‑scale PyTorch model training while reducing infrastructure costs and providing practical optimization techniques and code examples.
SWIFT is an open‑source, PyTorch‑based framework that integrates multiple efficient fine‑tuning methods such as LoRA, QLoRA, Adapter, and the proprietary ResTuning, enabling developers to fine‑tune large language and multimodal models on consumer‑grade GPUs with significantly reduced memory and compute requirements.
The article introduces Coeus, Bilibili's cloud‑native AI platform built on Kubernetes with Alluxio integration, explains how it solves major data and compute challenges, improves training performance, and promotes a free PyTorch performance‑tuning guide for engineers.
This article introduces word‑level representations—from one‑hot encoding to dense word embeddings via Word2Vec—explains cosine similarity, then walks through the structure, limitations, and PyTorch implementation of a vanilla RNN, including a custom forward function and verification against the library API.
This article explains how to overcome GPU memory limits by using PyTorch 1.9's meta device to create an empty model, load large‑scale model weights layer‑by‑layer, move each part to a 16 GB GPU for inference, and release memory, enabling a 70B FP16 model to run on a single consumer‑grade GPU.
This tutorial explains how to use the open‑source RobustVideoMatting project with Python, PyTorch, and OpenCV to perform human portrait segmentation and replace video backgrounds, covering repository setup, model loading, custom segmentation functions, and full video compositing steps.
Hugging Face has released Candle, a Rust‑written machine‑learning framework aimed at serverless inference, offering lightweight binaries, GPU support, and performance gains over Python‑based PyTorch, while sparking debate over Rust’s learning curve and the future of AI deployment.
This tutorial walks you through implementing a full Transformer model in PyTorch, starting from basic linear‑regression code, adding attention mechanisms, multi‑head attention, encoder‑decoder architecture, training loops, and inference, all reinforced with practical debugging tips.
AudioCraft is a PyTorch library that bundles state‑of‑the‑art AI models—MusicGen, AudioGen, and the EnCodec codec—to generate high‑quality audio from text or reference sounds, and the article explains its architecture, evaluation results, and how to install and run it.
This article introduces neural style transfer, explains its underlying principles using VGG19 feature extraction, content and style loss definitions, and provides a complete PyTorch implementation with code for loading images, extracting features, computing Gram matrices, and optimizing the output image.
This article explains the principle behind using deep‑learning autoencoders and UNet architectures to reconstruct mosaicked images, provides a complete PyTorch implementation with dataset preparation, network definition, training, and inference, and demonstrates the restored results.
The iQIYI backend team traced random OOM crashes in PyTorch Distributed Data Parallel on an A100 cluster to a malformed DDP message injected by a security scan, which forced a near‑terabyte allocation; using jemalloc for diagnostics, they mitigated the issue by adjusting scan policies and collaborating with PyTorch to harden the protocol.
This tutorial demonstrates how to download the Fashion‑MNIST dataset, build a four‑layer fully connected neural network with PyTorch, and train it using loss functions, Adam optimizer, learning‑rate strategies, and Dropout to achieve high‑accuracy multi‑class image classification.
This article provides a comprehensive tutorial on building the core components of a Transformer model—including multi‑head attention, layer normalization, feed‑forward networks, encoder and decoder layers—and assembles them into a complete PyTorch implementation, supplemented with explanatory diagrams and runnable code.
This article explains how to build the input processing stage of a Transformer—including tokenization with Hugging Face tokenizers, token‑to‑embedding conversion using BERT models, custom BPE tokenizers, and positional encoding—providing complete Python code examples and test results.
This article introduces ten essential Python libraries and frameworks, providing practical examples and code implementations for each to help readers understand and master their applications in various domains.
This article presents Ctrip's end‑to‑end solution for precise ad‑inventory forecasting using an LSTM model combined with entity embedding, covering data preprocessing, K‑means clustering, model architecture, offline‑online incremental training, early‑stop mechanisms, evaluation metrics, and Python service deployment.
When training large‑batch deep learning models, developers often use recompute to trade computation for memory, but in dynamic graph frameworks this can trigger synchronization errors in distributed data parallel training; the article explains the underlying DDP mechanics, illustrates the error, and offers a practical no_sync workaround with code examples.
This article introduces the basic architectures of recurrent neural networks (RNN), LSTM, and GRU, explains PyTorch APIs such as nn.RNN, nn.LSTM, nn.GRU, details their parameters, demonstrates code examples for building and testing these models, and provides practical insights for deep learning practitioners.
This article explains how Tencent’s AI team leverages the PyTorch‑based TorchRec library and a custom dynamic embedding solution to train billion‑scale recommendation models efficiently, detailing the benefits of TorchRec, GPU embedding, optimized kernels, embedding partition strategies, experimental results, and practical deployment guidance.
The article provides a detailed overview of PyTorch 2.0, highlighting its stable and beta features such as torch.compile, accelerated transformers, MPS backend, new quantization support, and prototype parallelism tools, while emphasizing performance improvements for dynamic shapes, distributed training, and CPU/GPU inference.
This article introduces the Transformer architecture, explains the self‑attention mechanism with visual illustrations, and provides a full, runnable PyTorch code example that implements the encoder‑decoder structure for sequence‑to‑sequence tasks.
This article presents a comprehensive overview of the Efficient Conformer model for large‑scale end‑to‑end speech recognition, detailing its architectural improvements such as progressive downsampling and grouped multi‑head self‑attention, the PyTorch implementation in WeNet, streaming inference handling, experimental CER gains on AISHELL‑1 and production data, and future development plans.
A PyTorch‑nightly supply‑chain attack introduced a malicious "torchtriton" package on PyPI that exfiltrated IP addresses, hostnames, usernames and SSH keys from Linux systems, prompting an urgent uninstall notice and a swift response from the PyTorch team.
BladeDISC 0.3.0 introduces full PyTorch 2.0 compilation support, new TorchDynamo optimizations, extensive GPU memory‑intensive compute enhancements, Shape Constraint IR, experimental quantization across multiple hardware platforms, and a suite of compiler‑level improvements for training and inference acceleration.
This article explains the motivation behind Spectral Normalization for stabilizing GAN training, derives its mathematical foundation, shows how it enforces the Lipschitz constraint, and provides a complete PyTorch implementation with detailed code explanations.
This tutorial walks through the complete FCN pipeline for semantic segmentation, covering VOC dataset loading, data augmentation, collate functions, model construction, training loops, loss computation with cross‑entropy (including ignore‑index handling), and inference, while providing full PyTorch code snippets for each step.
This article provides a concise introduction to a wide range of popular Python libraries for artificial intelligence and data science, such as NumPy, OpenCV, scikit-image, Pillow, Scikit-learn, TensorFlow, PyTorch, and many others, accompanied by practical code snippets and performance comparisons.
This article walks readers through building, training, and evaluating a Vision Transformer (ViT) model for a five‑class flower classification task, providing detailed code snippets, model architecture explanations, training script adjustments, and experimental results that highlight the importance of pre‑trained weights.
This article explains the GANomaly model for semi‑supervised anomaly detection, detailing its generator‑encoder‑discriminator architecture, loss functions, testing phase scoring, and provides annotated PyTorch source code to help readers implement and understand the approach.
This article introduces the fundamentals of Vision Transformers (ViT) for computer‑vision developers, starting with an overview of the transformer architecture, detailed explanation of self‑attention and multi‑head attention, and step‑by‑step PyTorch code examples that illustrate query, key, value computation and attention scoring.
This article explains why traditional GANs suffer from training instability, introduces the Wasserstein (Earth‑Mover) distance as a smoother alternative, derives the WGAN objective, discusses Lipschitz constraints, and provides practical PyTorch code modifications to convert a vanilla GAN into a stable WGAN.
This article introduces the EGBAD model, an efficient GAN‑based anomaly detection method that replaces AnoGAN's costly latent variable search with an encoder, provides detailed PyTorch code for data loading, model construction, training, and inference, and compares its testing speed with AnoGAN.
Graph4NLP is a PyTorch‑ and DGL‑based open‑source library that provides a full pipeline—from static and dynamic graph construction to embedding, learning, prediction, and inference—for applying graph neural networks to a wide range of NLP tasks, with extensive documentation, demos, and future scalability plans.
This article walks through the development of a simple advertising recommendation system using Python, covering data collection, preprocessing with label encoding, text embedding via Torch, constructing an MLP model, and initiating training, while reflecting on the challenges faced by Python developers in the big‑data era.
This article introduces the MAE (Masked AutoEncoder) self‑supervised learning method, explains its asymmetric encoder‑decoder design and high masking ratio, evaluates its performance, and provides a step‑by‑step guide to reproduce MAE using Alibaba’s EasyCV framework, including code snippets, training tips, and troubleshooting.
This article walks through a complete PyTorch implementation of the SRCNN model for image super‑resolution, covering dataset preparation, patch extraction, model architecture, training on a GTX 770 GPU for 2500 epochs, PSNR evaluation, and visual comparisons with bicubic up‑sampling.
This article walks through building and analyzing a minimal GAN—PointGAN—that learns to output the single value 1, covering the linear generator, a two‑layer discriminator, training loops, loss visualizations, instability diagnostics, and practical fixes such as loss easing, weighted examples, weight decay, and noisy generator parameters.
This article presents a Python-based tutorial for cartoonizing GIFs and videos using PyTorch models, detailing environment setup, core network architecture, processing functions, and practical results with code snippets and usage instructions, including visual examples.
The article explains CNN inference optimization by applying PyTorch quantization and module‑fusion techniques, compares model size and latency before and after quantization, shows code for building, quantizing, and fusing a simple CNN, and presents benchmark results on CPU, highlighting a four‑fold size reduction and up to 1.7× speed‑up.
This article walks through the core PyTorch visualization utilities—making image grids, drawing bounding boxes, segmentation masks, and keypoints—explaining why they are needed, how to set up the pipeline, and providing complete code examples for each computer‑vision task.
This guide explains PyTorch’s distributed training, contrasting single‑node DataParallel with multi‑node DistributedDataParallel, detailing essential parameters, group communication setup, proper use of DistributedSampler for data loading, handling IO bottlenecks, and avoiding common pitfalls such as memory imbalance, unsynchronized buffers, and unused‑parameter errors.
This article explains how to build a Physics‑Informed Neural Network (PINN) in PyTorch to solve a simple logistic ordinary differential equation, covering the underlying theory, loss formulation with equation residuals and boundary conditions, network architecture, automatic differentiation, and training results.
This tutorial walks through the theory and implementation of Neural Radiance Fields (NeRF) in PyTorch, covering positional encoding, the MLP architecture, differentiable volume rendering, hierarchical sampling, training tricks, and references to the original research.
The article introduces Angel Graph, a large‑scale graph computing platform built on Angel's parameter‑server architecture and Spark, detailing its evolution, framework components (including Spark‑on‑Angel and PyTorch‑on‑Angel), data and model partitioning strategies, communication and computation optimizations, stability mechanisms, usability features, and real‑world applications across recommendation, risk control, social and gaming domains.
This article explains how to resize images using OpenCV's cv2.resize function and how to scale multi‑dimensional tensors in PyTorch with torch.nn.functional.interpolate, providing detailed parameter descriptions and practical code examples for both single images and batch processing.
This article introduces the most important Python machine‑learning libraries of 2021—including TensorFlow, Scikit‑Learn, NumPy, Keras, PyTorch, LightGBM, Eli5, SciPy, Theano and Pandas—explaining their purposes, distinctive characteristics, and why they are essential tools for modern AI development.
Elasticsearch 8.0 introduces major updates including 7.x REST API compatibility headers, default-enabled security with registration tokens, known issues on ARM/macOS, a preview KNN search API using dense_vector, storage reductions for keyword and text fields, faster geo indexing, PyTorch model support, and numerous other enhancements across aggregations, allocation, analysis, authentication, and core infrastructure.
This article compares PyTorch and TensorFlow in 2022 across model availability, deployment ease, and ecosystem support, using data from HuggingFace, research papers, and industry tools, and offers tailored recommendations for industry engineers, researchers, educators, career changers, hobbyists, and beginners.
This article provides a detailed technical walkthrough of Masked Autoencoders (MAE) for computer vision, covering its BERT‑inspired masking strategy, asymmetric encoder‑decoder design, implementation specifics, experimental findings on mask ratios and decoder depth, and the resulting performance gains over supervised ViT models.
This article explains how Intel and Facebook's BF16 support, combined with the Intel Extension for PyTorch (IPEX) and oneDNN, automates type and layout conversions and adds graph‑fusion optimizations, delivering 1.4×‑4.3× inference and up to 2.4× training speedups on Xeon CPUs for models such as DLRM, BERT‑Large, and ResNext‑101‑32x4d.
This article explains how to configure and use PyTorch optimizers, their attributes and methods, and demonstrates various learning‑rate scheduling techniques—including manual updates and built‑in schedulers such as LambdaLR, StepLR, MultiStepLR, ExponentialLR, CosineAnnealingLR, and ReduceLROnPlateau—through clear code examples.
This article walks through the complete workflow for building, testing, and launching a distributed PyTorch training job on AzureML using the CLI v2, covering local script preparation, Accelerate configuration, Docker environment setup, dataset registration, compute target definition, job YAML creation, and job submission with monitoring.
An in‑depth 2022 comparison of PyTorch and TensorFlow evaluates model availability, deployment ease, and ecosystem support, showing PyTorch dominates research while TensorFlow excels in deployment, and offers tailored recommendations for industry professionals, researchers, educators, career changers, hobbyists, and beginners.
This article explains the fundamentals of graph neural networks, covering graph concepts, node classification via neighborhood aggregation, message‑passing mechanics, mathematical notation, a full DGL‑PyTorch implementation on the Reddit dataset, and training results showing accuracy improvements up to 91 %.
This article explains how to use PyTorch's named_parameters() and parameters() to retrieve a network's layer names and weights, demonstrates printing each parameter, and shows several common weight‑initialization techniques for convolutional and linear layers within a CNN architecture.
This article explains a NeurIPS 2021 paper that introduces a learnable autocorrelation correction for neural network time‑series models, shows the underlying theory, provides concise PyTorch code implementing the adjustment, reports a ~17% average performance gain across datasets, and lists additional practical tricks for time‑series forecasting.
This article walks through using the UNETR transformer architecture to segment 3D brain MRI scans from the BRATS dataset, detailing environment setup, data preprocessing with MONAI, model construction, training with DiceCE loss, validation metrics, and visualizing the best‑performing model outputs.
This article introduces a simple five‑line implementation of R‑Drop, a contrastive self‑supervised learning technique that leverages dropout‑induced perturbations to improve model robustness, explains the underlying principle, provides the exact PyTorch code, and compares it with the ConSERT method.
This article walks through constructing a learnable chess AI by integrating AlphaZero‑style Monte Carlo Tree Search with a decoder‑only Transformer, detailing the game tree logic, model architecture, input and output encodings, self‑play training loop, and code implementation in PyTorch.
This article explains how to apply a Deep Convolutional GAN in PyTorch to create realistic synthetic images of marine plastic, addressing dataset scarcity, detailing the network architecture, training procedure, and showing loss curves and generated samples.
This article walks through the design of Compact Transformers, explaining scaled dot‑product self‑attention, positional embeddings, multi‑head attention, and Vision Transformer architecture, and provides full PyTorch code so readers can train lightweight CV and NLP classifiers on a single PC.
This article explains how optimizers and learning‑rate schedulers work, how to configure their hyperparameters and parameter groups, and how to apply differential learning rates and adaptive schedules in PyTorch and Keras to improve model training and transfer‑learning performance.
This article demonstrates how to implement DropBlock in PyTorch, explains why Dropout fails on image data, details the gamma calculation and mask generation, and shows visual comparisons that illustrate the superiority of contiguous region dropping over random pixel dropout.
This article walks through building a PyTorch ShuffleNetV2 model, preparing the Kaggle Flowers dataset, training with transfer learning on a GPU, visualizing loss and accuracy, and performing inference on five test images, achieving nearly 90% validation accuracy after 95 epochs.
This article walks through creating a satellite image classification pipeline using PyTorch and a pretrained ResNet34 model, covering dataset preparation, project structure, data loading, model definition, training, validation, loss/accuracy plotting, and inference on new images with detailed code examples and results.
This article provides a step‑by‑step guide to building, training, and evaluating a custom object detection model using PyTorch Faster R‑CNN on a microcontroller dataset, covering data preparation, configuration, model modification, training loops, loss visualization, and inference on new images.
This article explains how to modify the Soft Actor‑Critic (SAC) algorithm—originally designed for continuous actions—to work with discrete action environments, presents the required changes to the actor and critic loss functions, provides a full PyTorch implementation, and evaluates the method on the CartPole‑v1 benchmark.
This article provides a comprehensive guide on YOLOv5, covering its background from YOLOv3, detailed code analysis of the model architecture, step‑by‑step instructions for running detect.py, configuring yolov5s.yaml, exporting the model to TorchScript JIT and ONNX formats, and practical inference examples using PyTorch and ONNX Runtime.
This tutorial explains how to visualize a PyTorch convolutional neural network's architecture and training dynamics using tools such as HiddenLayer, torchviz, TensorBoardX, and Visdom, providing step‑by‑step code examples and screenshots for each method.
The article walks through diagnosing why PyTorch cannot access the GPU, reinstalling CUDA, selecting matching PyTorch builds, adjusting versions, and verifying that CUDA becomes available for accelerated training.
This article explains the internal mechanism of PyTorch's DataLoader when using multiple worker processes, detailing how tensors are serialized, shared via multiprocessing.Queue, and reconstructed in the main process to avoid unnecessary memory copies.
This article reviews self‑supervised learning techniques, common computer‑vision pretext tasks, contrastive loss functions, popular frameworks such as SimCLR, MoCo and SimSiam, and demonstrates their application to OCR captcha recognition with detailed implementation and experimental results.
This article walks through building a convolutional neural network in Python to label and recognize image captchas, covering data collection, preprocessing, model coding, training on GPU/CPU, testing accuracy, and deploying the model as a Flask API.
This article introduces Google’s JAX library, covering its origins, core features such as automatic differentiation, JIT compilation, parallel and vectorized mapping, installation steps, code examples, and a comparative overview with TensorFlow and PyTorch for deep‑learning practitioners.
An overview of ten widely used Python machine‑learning libraries—including TensorFlow, Scikit‑Learn, NumPy, Keras, PyTorch, LightGBM, Eli5, SciPy, Theano, and Pandas—detailing their core features, typical applications, and why they are essential tools for data scientists and AI developers.
This article presents practical strategies for building high‑performance PyTorch training pipelines, covering bottleneck identification, efficient data loading, RAM‑based datasets, profiling tools, multi‑GPU training with DataParallel and DistributedDataParallel, custom loss implementation, and hardware‑vs‑software trade‑offs to accelerate deep‑learning workloads.
This article details the background, MMoE model architecture, baseline setup, environment configuration, data preprocessing, training process, evaluation results, and department information for the 58.com recruitment recommendation AI competition using the WPAI platform.
This article explains how PyTorch's BP Engine dynamically builds the computation graph for back‑propagation, detailing its C++ class structure, thread management, task queues, and key functions such as start_threads, compute_dependencies, execute, and evaluate_function, with illustrative code examples.
This article explains how PyTorch implements its dynamic autograd graph using the core C++ classes Variable, Function, and Engine, detailing their initialization, inheritance hierarchy, code structures, and the execution flow of backward propagation.
Bagua is a distributed training framework co‑developed by Kuaishou and ETH Zürich that combines algorithmic and system‑level optimizations—such as decentralized, asynchronous, and compressed communication—to achieve up to 60% higher performance than existing frameworks like PyTorch‑DDP, Horovod, and BytePS across various AI workloads.
An in‑depth comparison of TensorFlow 2.x and PyTorch 1.8 highlights new features, deployment options like TensorFlow Lite and PyTorch Mobile, coding style differences, and practical guidance on choosing the right deep‑learning library for various projects and skill levels.
This article explains the motivation behind residual networks, describes the architecture of ResNet including residual blocks and skip connections, lists available Keras implementations, and provides a step‑by‑step PyTorch tutorial with complete code to construct and test ResNet‑50/101/152 models.
This article reviews the latest releases of PyTorch 1.8 and TensorFlow 2.5, outlining their new functionalities, ecosystem tools such as TensorFlow.js, Lite, and TFX, as well as PyTorch Mobile and Lightning, and provides guidance on selecting the most suitable framework for different deep‑learning projects.
This tutorial explains how to install, launch, and use TensorBoard and Visdom with PyTorch, providing step‑by‑step commands, code examples for logging training metrics, and visualizing images and plots to monitor deep‑learning experiments.
This article explains how Tubi leverages machine learning—particularly PyTorch, Databricks, and cloud services—to improve content understanding, advertising technology, and recommendation systems within its advertising‑based video‑on‑demand platform, outlining the three AVOD pillars, technical stack, and future research directions.
This article explains how to build a code clone detection service using a CodeBERT classification model, create a custom TorchServe handler, package the model with torch-model-archiver, launch the service, and test it with example code pairs to demonstrate clone and non‑clone predictions.
This developer community weekly roundup highlights CCTV's new big‑data governance platform, RedMonk's programming language rankings, Chromium‑based browsers adopting a four‑week release cycle, PyTorch 1.8 with AMD support, the world’s first AI‑driven earthquake monitoring system, Red Hat OpenShift 4.7, a deep meta‑learning model for city sales prediction, and a CVPR breakthrough in controllable human image generation.
PyTorch 1.8, released by the PyTorch team, bundles over 3,000 commits since 1.7, introducing AMD ROCm support, enhanced Python function conversion, stable FFT and linear‑algebra APIs, complex‑tensor autograd, distributed‑training improvements, new mobile tutorials, performance tools, and several prototype features.
Facebook’s 2020 open‑source report reveals a rapid expansion to over 700 repositories and 1.3 million GitHub followers, highlights AI breakthroughs like the M2M‑100 multilingual model and ReBel algorithm, and showcases frontend tools such as React Native and the newly released Recoil library.
An open‑source PyTorch‑based project enables users to remove unwanted objects from videos simply by drawing a bounding box around them, offering a practical demo, step‑by‑step instructions, and a GitHub repository with over 2 k stars.
This article provides a step‑by‑step illustrated guide to the Transformer architecture, covering its macro structure, detailed self‑attention mechanisms, multi‑head attention, positional encoding, residual connections, decoder operation, training process, loss functions, and includes complete PyTorch and custom Python code examples.
This guide explains five key PyTorch tensor functions—expand, permute, tolist, narrow, and where—detailing their purpose, usage, and code examples so you can manipulate tensors efficiently on CPU or GPU.
