AudioCraft is a PyTorch library that bundles state‑of‑the‑art AI models—MusicGen, AudioGen, and the EnCodec codec—to generate high‑quality audio from text or reference sounds, and the article explains its architecture, evaluation results, and how to install and run it.
This article introduces neural style transfer, explains its underlying principles using VGG19 feature extraction, content and style loss definitions, and provides a complete PyTorch implementation with code for loading images, extracting features, computing Gram matrices, and optimizing the output image.
This article explains the principle behind using deep‑learning autoencoders and UNet architectures to reconstruct mosaicked images, provides a complete PyTorch implementation with dataset preparation, network definition, training, and inference, and demonstrates the restored results.
The iQIYI backend team traced random OOM crashes in PyTorch Distributed Data Parallel on an A100 cluster to a malformed DDP message injected by a security scan, which forced a near‑terabyte allocation; using jemalloc for diagnostics, they mitigated the issue by adjusting scan policies and collaborating with PyTorch to harden the protocol.
This tutorial demonstrates how to download the Fashion‑MNIST dataset, build a four‑layer fully connected neural network with PyTorch, and train it using loss functions, Adam optimizer, learning‑rate strategies, and Dropout to achieve high‑accuracy multi‑class image classification.
This article provides a comprehensive tutorial on building the core components of a Transformer model—including multi‑head attention, layer normalization, feed‑forward networks, encoder and decoder layers—and assembles them into a complete PyTorch implementation, supplemented with explanatory diagrams and runnable code.
This article explains how to build the input processing stage of a Transformer—including tokenization with Hugging Face tokenizers, token‑to‑embedding conversion using BERT models, custom BPE tokenizers, and positional encoding—providing complete Python code examples and test results.
This article introduces ten essential Python libraries and frameworks, providing practical examples and code implementations for each to help readers understand and master their applications in various domains.
This article presents Ctrip's end‑to‑end solution for precise ad‑inventory forecasting using an LSTM model combined with entity embedding, covering data preprocessing, K‑means clustering, model architecture, offline‑online incremental training, early‑stop mechanisms, evaluation metrics, and Python service deployment.
When training large‑batch deep learning models, developers often use recompute to trade computation for memory, but in dynamic graph frameworks this can trigger synchronization errors in distributed data parallel training; the article explains the underlying DDP mechanics, illustrates the error, and offers a practical no_sync workaround with code examples.
This article introduces the basic architectures of recurrent neural networks (RNN), LSTM, and GRU, explains PyTorch APIs such as nn.RNN, nn.LSTM, nn.GRU, details their parameters, demonstrates code examples for building and testing these models, and provides practical insights for deep learning practitioners.
This article explains how Tencent’s AI team leverages the PyTorch‑based TorchRec library and a custom dynamic embedding solution to train billion‑scale recommendation models efficiently, detailing the benefits of TorchRec, GPU embedding, optimized kernels, embedding partition strategies, experimental results, and practical deployment guidance.
The article provides a detailed overview of PyTorch 2.0, highlighting its stable and beta features such as torch.compile, accelerated transformers, MPS backend, new quantization support, and prototype parallelism tools, while emphasizing performance improvements for dynamic shapes, distributed training, and CPU/GPU inference.
This article introduces the Transformer architecture, explains the self‑attention mechanism with visual illustrations, and provides a full, runnable PyTorch code example that implements the encoder‑decoder structure for sequence‑to‑sequence tasks.
This article presents a comprehensive overview of the Efficient Conformer model for large‑scale end‑to‑end speech recognition, detailing its architectural improvements such as progressive downsampling and grouped multi‑head self‑attention, the PyTorch implementation in WeNet, streaming inference handling, experimental CER gains on AISHELL‑1 and production data, and future development plans.
A PyTorch‑nightly supply‑chain attack introduced a malicious "torchtriton" package on PyPI that exfiltrated IP addresses, hostnames, usernames and SSH keys from Linux systems, prompting an urgent uninstall notice and a swift response from the PyTorch team.
BladeDISC 0.3.0 introduces full PyTorch 2.0 compilation support, new TorchDynamo optimizations, extensive GPU memory‑intensive compute enhancements, Shape Constraint IR, experimental quantization across multiple hardware platforms, and a suite of compiler‑level improvements for training and inference acceleration.
This article explains the motivation behind Spectral Normalization for stabilizing GAN training, derives its mathematical foundation, shows how it enforces the Lipschitz constraint, and provides a complete PyTorch implementation with detailed code explanations.
This tutorial walks through the complete FCN pipeline for semantic segmentation, covering VOC dataset loading, data augmentation, collate functions, model construction, training loops, loss computation with cross‑entropy (including ignore‑index handling), and inference, while providing full PyTorch code snippets for each step.
This article provides a concise introduction to a wide range of popular Python libraries for artificial intelligence and data science, such as NumPy, OpenCV, scikit-image, Pillow, Scikit-learn, TensorFlow, PyTorch, and many others, accompanied by practical code snippets and performance comparisons.
This article walks readers through building, training, and evaluating a Vision Transformer (ViT) model for a five‑class flower classification task, providing detailed code snippets, model architecture explanations, training script adjustments, and experimental results that highlight the importance of pre‑trained weights.
This article explains the GANomaly model for semi‑supervised anomaly detection, detailing its generator‑encoder‑discriminator architecture, loss functions, testing phase scoring, and provides annotated PyTorch source code to help readers implement and understand the approach.
This article introduces the fundamentals of Vision Transformers (ViT) for computer‑vision developers, starting with an overview of the transformer architecture, detailed explanation of self‑attention and multi‑head attention, and step‑by‑step PyTorch code examples that illustrate query, key, value computation and attention scoring.
This article explains why traditional GANs suffer from training instability, introduces the Wasserstein (Earth‑Mover) distance as a smoother alternative, derives the WGAN objective, discusses Lipschitz constraints, and provides practical PyTorch code modifications to convert a vanilla GAN into a stable WGAN.
This article introduces the EGBAD model, an efficient GAN‑based anomaly detection method that replaces AnoGAN's costly latent variable search with an encoder, provides detailed PyTorch code for data loading, model construction, training, and inference, and compares its testing speed with AnoGAN.
Graph4NLP is a PyTorch‑ and DGL‑based open‑source library that provides a full pipeline—from static and dynamic graph construction to embedding, learning, prediction, and inference—for applying graph neural networks to a wide range of NLP tasks, with extensive documentation, demos, and future scalability plans.
This article walks through the development of a simple advertising recommendation system using Python, covering data collection, preprocessing with label encoding, text embedding via Torch, constructing an MLP model, and initiating training, while reflecting on the challenges faced by Python developers in the big‑data era.
This article introduces the MAE (Masked AutoEncoder) self‑supervised learning method, explains its asymmetric encoder‑decoder design and high masking ratio, evaluates its performance, and provides a step‑by‑step guide to reproduce MAE using Alibaba’s EasyCV framework, including code snippets, training tips, and troubleshooting.
This article walks through a complete PyTorch implementation of the SRCNN model for image super‑resolution, covering dataset preparation, patch extraction, model architecture, training on a GTX 770 GPU for 2500 epochs, PSNR evaluation, and visual comparisons with bicubic up‑sampling.
This article walks through building and analyzing a minimal GAN—PointGAN—that learns to output the single value 1, covering the linear generator, a two‑layer discriminator, training loops, loss visualizations, instability diagnostics, and practical fixes such as loss easing, weighted examples, weight decay, and noisy generator parameters.
This article presents a Python-based tutorial for cartoonizing GIFs and videos using PyTorch models, detailing environment setup, core network architecture, processing functions, and practical results with code snippets and usage instructions, including visual examples.
The article explains CNN inference optimization by applying PyTorch quantization and module‑fusion techniques, compares model size and latency before and after quantization, shows code for building, quantizing, and fusing a simple CNN, and presents benchmark results on CPU, highlighting a four‑fold size reduction and up to 1.7× speed‑up.
This article walks through the core PyTorch visualization utilities—making image grids, drawing bounding boxes, segmentation masks, and keypoints—explaining why they are needed, how to set up the pipeline, and providing complete code examples for each computer‑vision task.
This guide explains PyTorch’s distributed training, contrasting single‑node DataParallel with multi‑node DistributedDataParallel, detailing essential parameters, group communication setup, proper use of DistributedSampler for data loading, handling IO bottlenecks, and avoiding common pitfalls such as memory imbalance, unsynchronized buffers, and unused‑parameter errors.
This article explains how to build a Physics‑Informed Neural Network (PINN) in PyTorch to solve a simple logistic ordinary differential equation, covering the underlying theory, loss formulation with equation residuals and boundary conditions, network architecture, automatic differentiation, and training results.
This tutorial walks through the theory and implementation of Neural Radiance Fields (NeRF) in PyTorch, covering positional encoding, the MLP architecture, differentiable volume rendering, hierarchical sampling, training tricks, and references to the original research.
The article introduces Angel Graph, a large‑scale graph computing platform built on Angel's parameter‑server architecture and Spark, detailing its evolution, framework components (including Spark‑on‑Angel and PyTorch‑on‑Angel), data and model partitioning strategies, communication and computation optimizations, stability mechanisms, usability features, and real‑world applications across recommendation, risk control, social and gaming domains.
This article explains how to resize images using OpenCV's cv2.resize function and how to scale multi‑dimensional tensors in PyTorch with torch.nn.functional.interpolate, providing detailed parameter descriptions and practical code examples for both single images and batch processing.
This article introduces the most important Python machine‑learning libraries of 2021—including TensorFlow, Scikit‑Learn, NumPy, Keras, PyTorch, LightGBM, Eli5, SciPy, Theano and Pandas—explaining their purposes, distinctive characteristics, and why they are essential tools for modern AI development.
Elasticsearch 8.0 introduces major updates including 7.x REST API compatibility headers, default-enabled security with registration tokens, known issues on ARM/macOS, a preview KNN search API using dense_vector, storage reductions for keyword and text fields, faster geo indexing, PyTorch model support, and numerous other enhancements across aggregations, allocation, analysis, authentication, and core infrastructure.
This article compares PyTorch and TensorFlow in 2022 across model availability, deployment ease, and ecosystem support, using data from HuggingFace, research papers, and industry tools, and offers tailored recommendations for industry engineers, researchers, educators, career changers, hobbyists, and beginners.
This article provides a detailed technical walkthrough of Masked Autoencoders (MAE) for computer vision, covering its BERT‑inspired masking strategy, asymmetric encoder‑decoder design, implementation specifics, experimental findings on mask ratios and decoder depth, and the resulting performance gains over supervised ViT models.
This article explains how Intel and Facebook's BF16 support, combined with the Intel Extension for PyTorch (IPEX) and oneDNN, automates type and layout conversions and adds graph‑fusion optimizations, delivering 1.4×‑4.3× inference and up to 2.4× training speedups on Xeon CPUs for models such as DLRM, BERT‑Large, and ResNext‑101‑32x4d.
This article explains how to configure and use PyTorch optimizers, their attributes and methods, and demonstrates various learning‑rate scheduling techniques—including manual updates and built‑in schedulers such as LambdaLR, StepLR, MultiStepLR, ExponentialLR, CosineAnnealingLR, and ReduceLROnPlateau—through clear code examples.
This article walks through the complete workflow for building, testing, and launching a distributed PyTorch training job on AzureML using the CLI v2, covering local script preparation, Accelerate configuration, Docker environment setup, dataset registration, compute target definition, job YAML creation, and job submission with monitoring.
An in‑depth 2022 comparison of PyTorch and TensorFlow evaluates model availability, deployment ease, and ecosystem support, showing PyTorch dominates research while TensorFlow excels in deployment, and offers tailored recommendations for industry professionals, researchers, educators, career changers, hobbyists, and beginners.
This article explains the fundamentals of graph neural networks, covering graph concepts, node classification via neighborhood aggregation, message‑passing mechanics, mathematical notation, a full DGL‑PyTorch implementation on the Reddit dataset, and training results showing accuracy improvements up to 91 %.
This article explains how to use PyTorch's named_parameters() and parameters() to retrieve a network's layer names and weights, demonstrates printing each parameter, and shows several common weight‑initialization techniques for convolutional and linear layers within a CNN architecture.
This article explains a NeurIPS 2021 paper that introduces a learnable autocorrelation correction for neural network time‑series models, shows the underlying theory, provides concise PyTorch code implementing the adjustment, reports a ~17% average performance gain across datasets, and lists additional practical tricks for time‑series forecasting.
This article walks through using the UNETR transformer architecture to segment 3D brain MRI scans from the BRATS dataset, detailing environment setup, data preprocessing with MONAI, model construction, training with DiceCE loss, validation metrics, and visualizing the best‑performing model outputs.
This article introduces a simple five‑line implementation of R‑Drop, a contrastive self‑supervised learning technique that leverages dropout‑induced perturbations to improve model robustness, explains the underlying principle, provides the exact PyTorch code, and compares it with the ConSERT method.
This article walks through constructing a learnable chess AI by integrating AlphaZero‑style Monte Carlo Tree Search with a decoder‑only Transformer, detailing the game tree logic, model architecture, input and output encodings, self‑play training loop, and code implementation in PyTorch.
This article explains how to apply a Deep Convolutional GAN in PyTorch to create realistic synthetic images of marine plastic, addressing dataset scarcity, detailing the network architecture, training procedure, and showing loss curves and generated samples.
This article walks through the design of Compact Transformers, explaining scaled dot‑product self‑attention, positional embeddings, multi‑head attention, and Vision Transformer architecture, and provides full PyTorch code so readers can train lightweight CV and NLP classifiers on a single PC.
This article explains how optimizers and learning‑rate schedulers work, how to configure their hyperparameters and parameter groups, and how to apply differential learning rates and adaptive schedules in PyTorch and Keras to improve model training and transfer‑learning performance.
This article demonstrates how to implement DropBlock in PyTorch, explains why Dropout fails on image data, details the gamma calculation and mask generation, and shows visual comparisons that illustrate the superiority of contiguous region dropping over random pixel dropout.
This article walks through building a PyTorch ShuffleNetV2 model, preparing the Kaggle Flowers dataset, training with transfer learning on a GPU, visualizing loss and accuracy, and performing inference on five test images, achieving nearly 90% validation accuracy after 95 epochs.
This article walks through creating a satellite image classification pipeline using PyTorch and a pretrained ResNet34 model, covering dataset preparation, project structure, data loading, model definition, training, validation, loss/accuracy plotting, and inference on new images with detailed code examples and results.
This article provides a step‑by‑step guide to building, training, and evaluating a custom object detection model using PyTorch Faster R‑CNN on a microcontroller dataset, covering data preparation, configuration, model modification, training loops, loss visualization, and inference on new images.
This article explains how to modify the Soft Actor‑Critic (SAC) algorithm—originally designed for continuous actions—to work with discrete action environments, presents the required changes to the actor and critic loss functions, provides a full PyTorch implementation, and evaluates the method on the CartPole‑v1 benchmark.
This article provides a comprehensive guide on YOLOv5, covering its background from YOLOv3, detailed code analysis of the model architecture, step‑by‑step instructions for running detect.py, configuring yolov5s.yaml, exporting the model to TorchScript JIT and ONNX formats, and practical inference examples using PyTorch and ONNX Runtime.
This tutorial explains how to visualize a PyTorch convolutional neural network's architecture and training dynamics using tools such as HiddenLayer, torchviz, TensorBoardX, and Visdom, providing step‑by‑step code examples and screenshots for each method.
The article walks through diagnosing why PyTorch cannot access the GPU, reinstalling CUDA, selecting matching PyTorch builds, adjusting versions, and verifying that CUDA becomes available for accelerated training.
This article explains the internal mechanism of PyTorch's DataLoader when using multiple worker processes, detailing how tensors are serialized, shared via multiprocessing.Queue, and reconstructed in the main process to avoid unnecessary memory copies.
This article reviews self‑supervised learning techniques, common computer‑vision pretext tasks, contrastive loss functions, popular frameworks such as SimCLR, MoCo and SimSiam, and demonstrates their application to OCR captcha recognition with detailed implementation and experimental results.
This article walks through building a convolutional neural network in Python to label and recognize image captchas, covering data collection, preprocessing, model coding, training on GPU/CPU, testing accuracy, and deploying the model as a Flask API.
This article introduces Google’s JAX library, covering its origins, core features such as automatic differentiation, JIT compilation, parallel and vectorized mapping, installation steps, code examples, and a comparative overview with TensorFlow and PyTorch for deep‑learning practitioners.
An overview of ten widely used Python machine‑learning libraries—including TensorFlow, Scikit‑Learn, NumPy, Keras, PyTorch, LightGBM, Eli5, SciPy, Theano, and Pandas—detailing their core features, typical applications, and why they are essential tools for data scientists and AI developers.
This article presents practical strategies for building high‑performance PyTorch training pipelines, covering bottleneck identification, efficient data loading, RAM‑based datasets, profiling tools, multi‑GPU training with DataParallel and DistributedDataParallel, custom loss implementation, and hardware‑vs‑software trade‑offs to accelerate deep‑learning workloads.
This article details the background, MMoE model architecture, baseline setup, environment configuration, data preprocessing, training process, evaluation results, and department information for the 58.com recruitment recommendation AI competition using the WPAI platform.
This article explains how PyTorch's BP Engine dynamically builds the computation graph for back‑propagation, detailing its C++ class structure, thread management, task queues, and key functions such as start_threads, compute_dependencies, execute, and evaluate_function, with illustrative code examples.
This article explains how PyTorch implements its dynamic autograd graph using the core C++ classes Variable, Function, and Engine, detailing their initialization, inheritance hierarchy, code structures, and the execution flow of backward propagation.
Bagua is a distributed training framework co‑developed by Kuaishou and ETH Zürich that combines algorithmic and system‑level optimizations—such as decentralized, asynchronous, and compressed communication—to achieve up to 60% higher performance than existing frameworks like PyTorch‑DDP, Horovod, and BytePS across various AI workloads.
An in‑depth comparison of TensorFlow 2.x and PyTorch 1.8 highlights new features, deployment options like TensorFlow Lite and PyTorch Mobile, coding style differences, and practical guidance on choosing the right deep‑learning library for various projects and skill levels.
This article explains the motivation behind residual networks, describes the architecture of ResNet including residual blocks and skip connections, lists available Keras implementations, and provides a step‑by‑step PyTorch tutorial with complete code to construct and test ResNet‑50/101/152 models.
This article reviews the latest releases of PyTorch 1.8 and TensorFlow 2.5, outlining their new functionalities, ecosystem tools such as TensorFlow.js, Lite, and TFX, as well as PyTorch Mobile and Lightning, and provides guidance on selecting the most suitable framework for different deep‑learning projects.
This tutorial explains how to install, launch, and use TensorBoard and Visdom with PyTorch, providing step‑by‑step commands, code examples for logging training metrics, and visualizing images and plots to monitor deep‑learning experiments.
This article explains how Tubi leverages machine learning—particularly PyTorch, Databricks, and cloud services—to improve content understanding, advertising technology, and recommendation systems within its advertising‑based video‑on‑demand platform, outlining the three AVOD pillars, technical stack, and future research directions.
This article explains how to build a code clone detection service using a CodeBERT classification model, create a custom TorchServe handler, package the model with torch-model-archiver, launch the service, and test it with example code pairs to demonstrate clone and non‑clone predictions.
This developer community weekly roundup highlights CCTV's new big‑data governance platform, RedMonk's programming language rankings, Chromium‑based browsers adopting a four‑week release cycle, PyTorch 1.8 with AMD support, the world’s first AI‑driven earthquake monitoring system, Red Hat OpenShift 4.7, a deep meta‑learning model for city sales prediction, and a CVPR breakthrough in controllable human image generation.
PyTorch 1.8, released by the PyTorch team, bundles over 3,000 commits since 1.7, introducing AMD ROCm support, enhanced Python function conversion, stable FFT and linear‑algebra APIs, complex‑tensor autograd, distributed‑training improvements, new mobile tutorials, performance tools, and several prototype features.
Facebook’s 2020 open‑source report reveals a rapid expansion to over 700 repositories and 1.3 million GitHub followers, highlights AI breakthroughs like the M2M‑100 multilingual model and ReBel algorithm, and showcases frontend tools such as React Native and the newly released Recoil library.
An open‑source PyTorch‑based project enables users to remove unwanted objects from videos simply by drawing a bounding box around them, offering a practical demo, step‑by‑step instructions, and a GitHub repository with over 2 k stars.
This article provides a step‑by‑step illustrated guide to the Transformer architecture, covering its macro structure, detailed self‑attention mechanisms, multi‑head attention, positional encoding, residual connections, decoder operation, training process, loss functions, and includes complete PyTorch and custom Python code examples.
This guide explains five key PyTorch tensor functions—expand, permute, tolist, narrow, and where—detailing their purpose, usage, and code examples so you can manipulate tensors efficiently on CPU or GPU.
This guide walks researchers through selecting a Tencent Cloud GN7 GPU instance, installing NVIDIA drivers, CUDA 10.2, cuDNN, setting up PyTorch and Jupyter, and configuring remote development with PyCharm, enabling efficient, cost‑effective AI development on a Tesla T4 GPU server.
This article introduces Boa, a Node.js library that bridges CPython, enabling JavaScript developers to import and use Python packages—especially machine‑learning libraries like PyTorch—seamlessly within Node.js, complete with practical code examples and API explanations.
This article explains PyTorch's autograd system, covering tensor creation, the requires_grad flag, detaching tensors, disabling gradient tracking with no_grad, the Function class and computational graph, and demonstrates forward and backward passes with code examples illustrating gradient computation and Jacobian‑vector products.
dl_inference is an open‑source inference platform that simplifies deployment of TensorFlow and PyTorch models in production, offering unified gRPC access, load‑balanced multi‑node serving, GPU/CPU options, customizable pre‑ and post‑processing, and extensible architecture for future AI workloads.
A detailed case study reveals why PyTorch training on UFS file storage suffers severe I/O bottlenecks, compares it with local SSD and SSHFS, and presents practical optimizations such as using cv2.imdecode, caching DataLoader handles, and converting small‑file datasets into large UFS files to close the performance gap.
This article explains the concept, mathematics, and practical PyTorch implementation of dilated (or atrous) convolutions, showing how to keep image dimensions unchanged while dramatically increasing the receptive field and discussing their advantages and typical applications.
The article presents a comprehensive overview of 58.com’s AI platform built on Kubernetes, detailing its layered architecture, resource scheduling, offline training pipelines, debugging environment, distributed TensorFlow/PyTorch training, performance benchmarks, and online inference services, highlighting how the system empowers various business units with scalable AI capabilities.
This article introduces ten essential Python libraries—TensorFlow, Scikit‑Learn, NumPy, Keras, PyTorch, LightGBM, Eli5, SciPy, Theano, and Pandas—detailing their features, typical use cases, and adoption in machine‑learning and data‑science projects, while highlighting each library's performance advantages, community support, and integration capabilities to help developers choose the right tool for their AI workflows.
This article provides an in‑depth overview of BERT, covering its NLP background, GLUE benchmark achievements, Transformer‑based architecture, pre‑training strategies (MLM and NSP), downstream fine‑tuning methods, and includes detailed PyTorch code implementations of its core components.
The article explains how transpose (de)convolution works as the spatial inverse of standard convolution, detailing its relationship to fully‑connected layers, padding, stride, output size formulas, odd‑case handling, and practical implementation in frameworks like PyTorch and TensorFlow.
This article reviews major deep‑learning object detection approaches—including one‑stage YOLO and SSD and two‑stage RCNN, Fast RCNN, and Faster RCNN—then provides a step‑by‑step explanation of Faster RCNN’s architecture, region‑proposal network, RoI pooling, loss functions, and sample PyTorch code.
This article introduces five highly rated Python libraries—PyWren, Tfdeploy, Luigi, Kubelib, and PyTorch—that streamline data handling, cloud execution, workflow orchestration, and GPU acceleration, helping machine‑learning engineers boost productivity and tackle complex projects more efficiently.
