This paper investigates the embedding dimensional collapse problem that hinders scaling of recommendation models and proposes a Multi-Embedding paradigm that learns multiple embeddings per feature with independent expert networks, demonstrating consistent performance gains across major CTR benchmarks and real‑world ad systems.
This article explains the fundamentals of transformer models, why they outperform earlier neural networks, their core components such as self‑attention and positional encoding, practical use cases across language and biology, and how they differ from RNNs, CNNs, and other architectures.
This article examines Meta's SUM (Scaling User Modeling) system, detailing its upstream‑downstream architecture, the SOAP online asynchronous serving platform, production optimizations, and extensive offline and online experiments that demonstrate significant gains in ad personalization performance.
The article explains how AI‑powered anomaly detection—using statistical 3‑sigma/Z-score methods, unsupervised machine‑learning like Isolation Forest, and deep‑learning models such as LSTM, Transformer and Pyraformer—overcomes the limits of threshold‑based monitoring by preprocessing data, reducing false alerts, and delivering high‑precision observability metrics.
This article explains PaddlePaddle tensor indexing, covering basic slicing, integer and boolean advanced indexing, ellipsis and newaxis usage, assignment in dynamic and static graphs, automatic gradient propagation, and demonstrates practical applications such as semantic segmentation, object detection, and NLP sequence masking.
LivePortrait is an open‑source, controllable portrait video generation framework that transfers facial expressions and poses from a driving video to static or dynamic portraits in real time, leveraging a 69M‑frame mixed video‑image training set, stitching and retargeting modules, and achieving high quality with low latency.
This article explains the backpropagation algorithm—its origins, mathematical basis, step‑by‑step workflow, importance for efficient neural network training, and widespread applications in image recognition, natural language processing, and recommendation systems.
Large model weights, the core parameters that shape neural network behavior, are crucial for tasks ranging from image recognition to natural language processing; this article explains what they are, how they’re initialized, trained, shared, stored, and leveraged in transfer learning.
Meta’s new paper introduces the Wukong model, demonstrating that expanding dense‑layer parameters and computational FLOPs in large‑scale recommendation systems follows a clear scaling law, yielding consistent performance gains across massive internal datasets, with detailed analysis of feature modules, parameter impacts, and experimental results.
This article explains the fundamental concepts and architecture of artificial neural networks, describes their learning process, compares them with biological neural systems, and highlights both the similarities and key differences in structure, learning mechanisms, flexibility, and energy efficiency.
The 2024 China College Students AI Innovation Competition, now in its sixth year and recognized as a national university contest, has opened for global students to create AI large‑model applications using Baidu’s PaddlePaddle, Wenxin and the zero‑code PaddleX pipeline, with a training camp and an August 15 registration deadline.
Generative AI, a rapidly advancing branch of artificial intelligence, uses models like GANs, VAEs, and large language models to create new content across fields such as media, VR/AR, medical imaging, and gaming, while facing challenges related to data bias, ethics, and computational complexity.
The CVPR NTIRE 2024 short-form video quality assessment competition introduced the KVQ dataset, attracted over 200 teams, evaluated submissions using SROCC and PLCC metrics, and highlighted the winning approaches of SJTU MMLab, IH‑VQA, and TVQE, showcasing advances in AI‑driven video quality evaluation.
Over the past year, Vision‑Language Models have surged from early multimodal experiments to competitive open‑source systems rivaling GPT‑4, driven by higher‑resolution processing, richer vision encoders, better projection layers, and larger curated datasets, yet they still face evaluation difficulties, hallucinations, speed limits, and limited multimodal output.
This article explains the principle of spatial attention in convolutional neural networks, details the underlying algorithmic steps, and provides a complete PyTorch implementation including the attention module, full network architecture, and practical considerations for integrating spatial attention into deep learning models.
This presentation reviews the technical scope of document intelligence, its specific applications and challenges in finance, recent advances in document analysis, recognition, and understanding, and outlines future research directions for large‑model and multimodal solutions in processing complex financial documents.
This article provides a comprehensive introduction to large language models, covering their historical development, core architecture, training process, prompt engineering techniques, Retrieval‑Augmented Generation, agent frameworks, multimodal capabilities, safety challenges, and future research directions.
Transformers, introduced in 2017, have reshaped neural networks by leveraging attention mechanisms to outperform RNNs and CNNs across NLP, computer vision, and speech tasks, offering parallel processing, long‑range dependency capture, and versatile applications such as translation, text generation, image classification, and speech recognition.
Machine learning, a core AI discipline, encompasses traditional algorithms like supervised, unsupervised, and reinforcement learning as well as neural network models such as CNNs, RNNs, GANs, and VAEs, each with distinct principles, strengths, and typical application scenarios.
Deep LearningTraditional Algorithmsmachine learning
This article provides a comprehensive overview of user profiling algorithms, covering the evolution from ontology‑based traditional methods to modern deep‑learning approaches, including structured label prediction, representation learning, active learning, and large‑model integration, while discussing challenges, practical applications, and future research directions.
Active LearningDeep LearningRepresentation Learning
The IEEE CVPR2024 conference in Seattle accepted 2,719 papers out of 11,532 submissions, and Ant Group contributed 24 papers covering computer vision, deep learning, digital humans, large models, multimodal remote sensing, vision‑language distillation, federated incremental learning, model‑stealing defense, and more, with one highlighted as a highlight.
Xiaohongshu’s audio‑video architecture team secured second place in the RAIM challenge and first in the S‑UGC VQA challenge at CVPR NTIRE 2024 by improving generative image restoration with SUPIR, DeSRA and a Fusion model, and enhancing video quality assessment using LIQE, Q‑Align and FAST‑VQA, then deploying these methods for live‑stream denoising, intelligent transcoding and cloud‑based super‑resolution, achieving high PLCC/SROCC scores and up to 33 % bandwidth savings.
This article provides a detailed, English-language walkthrough of the HRNet source code, covering how the COCO keypoint dataset is built, the online data‑augmentation techniques applied during training, and the end‑to‑end training and inference procedures for human pose estimation.
In a keynote at the 2024 Beijing Zhiyuan Conference, Baidu’s CTO Wang Haifeng explained how large‑model universality and comprehensive capabilities are driving artificial general intelligence forward, highlighting scale laws, multimodal advances, agent technologies, and the industrial‑scale production of AI.
This article provides a comprehensive, beginner-friendly explanation of the Transformer architecture, covering its encoder‑decoder structure, self‑attention, multi‑head attention, positional encoding, residual connections, decoding process, final linear and softmax layers, and training considerations, illustrated with numerous diagrams and code snippets.
In a candid interview, Geoff Hinton reflects on his AI journey—from early disappointments in physiology and philosophy to breakthroughs in neural networks, scaling laws, multimodal learning, fast‑weight concepts, and the ethical challenges shaping the future of artificial intelligence.
This article presents a mathematically modeled algorithm that extracts color palettes from any Pokémon image and applies them to another, optimizing the swap via deep‑feature distance and dense color‑transform space, demonstrating superior visual results and subjective evaluations compared to traditional hue‑shift and other recoloring methods.
This article presents a novel approach that combines debiased deep learning with double machine learning to estimate and infer average treatment effects across multiple simultaneous online experiments, detailing problem definition, a semi‑parametric theoretical framework, and extensive field‑experiment validation on a large video‑platform dataset.
HelixDock, a full‑atom diffusion model pretrained on a billion‑scale simulated docking dataset covering ~200,000 protein targets, delivers state‑of‑the‑art docking accuracy—85.6% success on PoseBusters and strong generalization on cross‑docking benchmarks—showing that massive data and model scaling dramatically improve AI‑driven drug discovery, and its code and data are fully open‑source.
This article explains the nature of moiré patterns in digital imaging, reviews manual mitigation techniques, introduces direct and indirect AI‑based recognition methods—including traditional feature extraction and deep‑learning models such as CNNs and diffusion frameworks—and details practical applications and evaluation metrics used by Huolala.
This article explains how GPUs, with their parallel architecture and extensive software ecosystem, have become essential for accelerating AI training and inference, outperforming CPUs and shaping the future of artificial intelligence across various industries.
Ilya Sutskever, former OpenAI chief scientist, shared a curated list of 27 seminal AI research papers—including the Annotated Transformer, Attention Is All You Need, and Deep Residual Learning—with links, claiming mastering them covers roughly 90% of today’s essential artificial‑intelligence knowledge.
HelixFold‑Multimer, a new Baidu PaddleHelix model, outperforms AlphaFold 3 on antigen‑antibody and peptide‑protein complex predictions, achieving mean DockQ scores of 0.41 and 0.38 respectively and success rates up to 77 % when epitope data are used, and is already deployed in large‑molecule drug pipelines.
The article details how JD's advertising retrieval platform tackles the core challenge of balancing limited compute resources with massive data by optimizing compute allocation, improving model scoring efficiency, and enhancing iteration speed through distributed execution graphs, adaptive algorithms, and platform‑level infrastructure improvements.
This article explains the pivotal role of GPUs in today’s generative AI era, describes their architecture and applications, compares them with CPUs, ASICs, and FPGAs, and offers guidance on selecting the right processor for AI workloads while also noting related reference resources.
This article explains how Transformer models work by detailing the encoder‑decoder structure, self‑attention, multi‑head attention, positional encoding, and feed‑forward networks, and shows their applications in machine translation, recommendation systems, and large language models.
This article walks through the complete process of creating, training, evaluating, saving, and loading a MNIST handwritten digit recognition model using TensorFlow in Java, comparing it with the equivalent Python implementation and covering required knowledge, environment setup, and code details.
This tutorial demonstrates how to train a MNIST digit‑recognition model using the Rust‑based Burn framework on top of the cross‑platform wgpu API, covering model export from PyTorch to ONNX, code generation, data loading, training loops, and performance comparison across CPU, GPU, and other backends.
The article reviews recent generative‑AI breakthroughs such as GPT‑5 and AI software engineers, explains that AI systems are deterministic rather than black boxes, and then teaches neural‑network fundamentals—including activation functions, back‑propagation, and a hands‑on MNIST digit‑recognition example with discussion of overfitting and regularization.
This article provides a comprehensive overview of Transformer models, covering their attention-based architecture, encoder-decoder structure, training procedures including teacher forcing, inference workflow, advantages over RNNs, and various applications in natural language processing such as translation, summarization, and classification.
This article provides a comprehensive, code‑rich walkthrough of the Transformer architecture using PaddlePaddle, covering the encoder and decoder components, residual connections, layer normalization, feed‑forward networks, scaled dot‑product and multi‑head attention, and shows how to assemble the full model with training and inference functions.
This article details the end‑to‑end deep‑learning pipeline used by 58.com for real‑estate recommendation, covering business background, a six‑layer architecture, vector‑based recall, various embedding and ranking models, multi‑task and multi‑scenario optimization techniques, and future directions for large‑model integration.
This article provides a friendly, step‑by‑step overview of five core deep‑learning architectures—CNN, RNN, GAN, Transformers, and encoder‑decoder—explaining their structures, key components, and typical use cases in image and natural‑language processing.
This article provides a comprehensive overview of user portrait algorithms, covering their historical development, ontology‑based traditional approaches, deep‑learning enhancements, representation‑learning techniques such as lookalike, active‑learning driven iteration, and the integration of large‑model world knowledge, while also discussing current challenges and future research directions.
This article explains OpenAI's Sora video generation model, detailing its latent diffusion foundation, video compression network, spacetime patch representation, Diffusion Transformer processing, and decoding pipeline, while also reviewing related Stable Diffusion and Transformer concepts that enable high‑quality text‑to‑video synthesis.
This article introduces PyTorch as a leading open‑source deep‑learning framework, outlines its key components such as dynamic computation graphs, tensors, autograd, modules, optimizers, data loading, distributed training and TorchScript, and provides a complete Python example that defines a simple CNN and trains it on the CIFAR‑10 dataset.
This article explains the Transformer architecture, its self‑attention mechanism, encoder‑decoder design, and the profound impact it has had on natural language processing, computer vision, and large‑scale language models like GPT.
Researchers from Alibaba Cloud AI and Zhejiang University present AMFormer, a Transformer‑based model that incorporates arithmetic feature interaction, demonstrating superior fine‑grained modeling, sample efficiency, and generalization on synthetic and real‑world tabular datasets, establishing a new state‑of‑the‑art in deep tabular learning.
Alibaba Cloud’s AI platform PAI showcased four cutting‑edge papers at AAAI‑2024—introducing AMFormer for deep tabular learning via arithmetic feature interaction, MuLTI for efficient video‑language understanding, M2SD for few‑shot class‑incremental learning, and M2Doc for multi‑modal document layout analysis—demonstrating the platform’s growing impact on artificial‑intelligence research.
This article provides an in‑depth overview of regression prediction, covering linear models like OLS, Lasso, Ridge, and Bayesian approaches, as well as non‑linear techniques such as tree ensembles, SVR, KNN, neural networks, and advanced deep learning frameworks for tabular data.
This article presents a GitHub‑hosted collection of 25 recent research papers on Mamba and its variants, summarizing each work’s core contributions across sequence modeling, vision, medical imaging, graph analysis, and multimodal tasks, and highlighting their performance gains over prior methods.
Bilibili’s self‑supervised video super‑resolution system upgrades low‑resolution streams to 4K by using three parallel degradation‑branch networks—texture‑enhancing, line‑recovering, and noise‑removing—tailored to anime, game, and real‑world content, delivering sharper edges, finer textures, and measurable quality gains across its online playback pipeline.
This article introduces causal learning, explains its distinction from traditional correlation‑based machine learning, outlines its three main parts, discusses the two primary paradigms—learning with known causal graphs and learning via causal discovery—and highlights their advantages, challenges, and recent research directions.
This report presents recent advances in optimizing deep learning click‑through‑rate models for advertising, including improved embedding mechanisms, novel feature‑interaction and architecture designs such as attention‑based behavior sequencing, multi‑tower and Mixture‑of‑Experts networks, dynamic ID handling, hourly updates, incremental training, and outlines future multi‑modal and embedding‑importance research.
OpenAI’s newly unveiled Sora system can generate 60‑second, high‑quality videos from plain text prompts, leveraging a data‑driven physical engine trained on synthetic data from Unreal Engine 5, with contributions from researchers like Tim Brooks and Bill Peebles, marking a major AI video‑generation breakthrough.
The article introduces STAN, a multi‑task recommendation framework that leverages user lifecycle segmentation to jointly optimize CTR, stay‑time, and CVR, detailing the business context, key challenges, solution architecture, offline and online evaluations, and future research directions.
This article explains the fundamental differences between static and dynamic computation graphs, compares their memory and performance characteristics, shows how automatic differentiation works in each paradigm, and provides a step‑by‑step implementation of a toy static‑graph AutoDiff engine with Python code examples.
This article reviews the evolution from GANs to diffusion models, explains the components of Stable Diffusion—including the CLIP text encoder, VAE, and UNet—and provides step‑by‑step Python code using HuggingFace's Diffusers library to generate images from text prompts.
This article reviews distributed training techniques for large deep‑learning models, covering data parallelism, model parallelism (including pipeline and tensor parallelism), gradient bucketing and accumulation, 3D parallelism, and practical implementations such as Megatron‑LM and 360AI platform optimizations.
This article explains the principle of LoRA (Low‑Rank Adaptation) for large language models, demonstrates how to decompose weight updates into low‑rank matrices, and provides a complete PyTorch implementation that fine‑tunes a small VGG‑19 network on a custom goldfish dataset.
This article explains the fundamentals of graph neural networks and graph representation learning, outlines how graphs enhance recommendation systems, and details OPPO's practical implementation of a hybrid dual‑tower and graph sub‑network model to improve recall and ranking performance.
This article reviews the latest research from Alibaba DAMO Academy on real-world image quality problems, covering spatial, temporal, and color enhancement methods, advanced quality assessment metrics, multimodal diffusion models, and future directions toward large‑model integration and lightweight deployment.
This article walks through the core LLaMA implementation in HuggingFace’s Transformers library, detailing the inheritance hierarchy, configuration defaults, model initialization, embedding and stacked decoder layers, the RMSNorm‑based attention and MLP modules, and the forward pass that produces normalized hidden states.
This article explains how a deep‑learning‑driven AI platform tackles the complex challenges of multilingual and multi‑timezone testing for a rapidly expanding international app, detailing the architecture, data pipelines, model training, and the resulting efficiency, accuracy, and coverage gains.
Bilibili’s AI‑driven frame‑interpolation pipeline upgrades low‑frame-rate videos to smooth high‑frame-rate 1080p playback by optimizing optical‑flow models for large motion, texture and text artifacts, pruning for speed, and deploying via the BVT SDK across on‑demand and live streams.
This guide provides comprehensive techniques for optimizing PyTorch training performance and efficiency, covering all model types such as CNNs, RNNs, GANs, and transformers, and applicable across domains like computer vision and natural language processing, targeting AI/ML platform engineers, data engineers, backend developers, MLOps, SREs, architects, and machine learning engineers.
This article reviews the growing interest in causal learning within machine learning, explaining what causal learning is, its advantages over purely correlational methods, and detailing two main paradigms—learning with known causal structures and learning via causal discovery—along with examples, challenges, and future directions.
Airbnb’s new computer‑vision pipeline trains a deep‑learning aesthetic model with an EMD loss to rank photos, automatically sorts new‑listing images by design and room type, and scales real‑time similarity search via HNSW‑based ANN on AWS OpenSearch, boosting click‑through, bookings, and enabling unsupervised visual recommendations.
A recent ETH Zurich paper shows that standard Transformer blocks can be drastically simplified by removing residual connections, LayerNorm, projection and value parameters, and even MLP sub‑block components, achieving up to 16% fewer parameters and comparable training speed and downstream performance on both GPT‑style decoders and BERT models.
The article showcases a series of humorous and surprising AI‑generated image expansions from Douyin, explains the underlying outpainting technology, and discusses why such tools are both entertaining and useful despite occasional odd results.
This article details HuoLala's journey from a weekly release cycle to a cloud‑based record‑and‑replay mobile UI automation platform, covering background challenges, industry analysis, technical design—including deep‑learning based control detection, SIFT image matching, script generation, playback handling, and platform features—while demonstrating significant testing efficiency gains and future AI‑driven enhancements.
This guide explains how to implement reverse image search by extracting visual features with a multilingual CLIP model, storing the vectors in Elasticsearch 8.x, and using its k‑NN plugin to retrieve similar images, covering architecture, tools, code snippets, and results.
This article reviews recent progress in advertising recommendation technologies, covering deep learning‑based ranking, sequence modeling, self‑supervised learning, online and reinforcement learning, multimodal recommendation, and fairness, and details four key breakthroughs—data‑driven incremental learning, dynamic group parameter modeling, bilateral interactive graph convolution, and a relation‑aware diffusion model for poster layout generation, along with experimental results and future challenges.
Deep LearningGraph Neural Networksadvertising recommendation
Learn step-by-step how to create, compile, train, evaluate, and deploy convolutional neural networks in Python using TensorFlow and Keras for general image classification and a practical face‑recognition example, complete with code snippets and data‑preprocessing techniques.
This article explains the Transformer model, comparing it with RNNs, detailing its encoder‑decoder structure, multi‑head and scaled dot‑product attention, embedding layers, feed‑forward networks, and the final linear‑softmax output, supplemented with diagrams and code examples.
The article analyzes the limitations of traditional speech‑driven lip‑sync methods and explains how Wav2Lip introduces a pretrained multi‑frame expert sync discriminator, a two‑stage GAN training pipeline, and a specialized generator architecture to produce high‑quality, audio‑aligned facial videos.
The article surveys sequential recommendation models from early non-deep approaches like FPMC, through RNN-based GRU4Rec and CNN-based Caser, to Transformer-based methods such as SASRec, BERT4Rec, TiSASRec, and recent contrastive-learning techniques, recommending SASRec or its variants for production use.
Renowned AI pioneer Geoffrey Hinton cautions that rapidly advancing artificial intelligence may surpass human control in as little as five years, highlighting self‑modifying code, the "black‑box" problem, and the urgent need for robust safety regulations.
This article reviews animal recognition technology, covering its background, basic principles, image‑processing, feature extraction, machine‑learning and deep‑learning methods, dataset construction, preprocessing, and feature‑selection techniques, and provides Python code examples for implementing CNNs and traditional classifiers.
This article explains the principles and structures of LSTM networks, introduces the ELMO contextual embedding model with its two‑stage pre‑training and downstream usage, and provides an overview of the Transformer architecture, highlighting their roles in modern NLP tasks.
This article explains the mathematical definition of convolution, walks through discrete and continuous examples, demonstrates its use in signal smoothing with moving averages, and surveys its wide-ranging applications in signal processing, communications, computer vision, seismology, medical imaging, and statistics.
The DataFunSummit2023 online conference brings together experts from Tencent and Kuaishou to present cutting‑edge research on causal inference for large‑scale A/B testing, including deep‑learning‑based multi‑experiment effect estimation, a distributed causal inference framework (Fast‑Causal‑Inference), and strategies for evaluating long‑term policy impacts.
The paper introduces QIN, a query‑driven user interest network that combines a Relevance Search Unit and a Fused Attention Unit to effectively leverage full‑history user behavior for personalized search, demonstrating significant performance gains in offline benchmarks and online A/B tests.
This article compiles Meituan's cutting‑edge computer‑vision research and engineering achievements—including CVPR award‑winning segmentation, YOLOv6 releases, GPU inference optimizations, the Food2K dataset, and numerous paper digests—to provide practical insights for visual AI practitioners.
This article examines the challenges of video copyright protection, critiques traditional visible and invisible watermark methods, and presents a deep‑learning based AI digital watermark solution that balances invisibility and robustness, detailing its network architecture, degradation layer, loss functions, block encoding, anchor calibration, and large‑scale experimental results.
This article presents a comprehensive case study of NIO's Power swap‑station ecosystem, detailing the business context, key forecasting challenges, the evolution from classical statistical models to deep‑learning architectures with specialized embeddings, and the practical outcomes and future plans for improving prediction accuracy.
This article introduces Social4Rec, a video recommendation algorithm that tackles user cold‑start problems by extracting and integrating social interest information through coarse‑ and fine‑grained interest extractors, attention‑based fusion, and extensive offline and online experiments demonstrating significant CTR improvements.
BILIVQA is Bilibili’s deep‑learning, no‑reference video quality assessment system that trains on a proprietary 5,000‑video UGC dataset, extracts spatial and temporal features via MobileNet‑V2 and X3D, uses mixed‑dataset regression for strong generalization, and deploys a GPU‑optimized TensorRT pipeline with percentile‑based scoring for reliable quality monitoring and downstream applications.
This article surveys the development of natural language processing from early rule‑based systems through statistical n‑gram models, neural language models, RNNs, LSTMs, ELMo, Transformers and BERT, and then details the architecture, training methods, advantages and limitations of the GPT‑1, GPT‑2, GPT‑3, ChatGPT and GPT‑4 models, concluding with a discussion of future challenges and references.
DVIS introduces a decoupled video instance segmentation framework that splits the task into segmentation, tracking, and refinement modules, achieving state-of-the-art performance across VIS, VPS, and VSS benchmarks while maintaining low computational overhead, and demonstrates robustness in both online and offline settings.
FlashAttention‑2 is an IO‑aware exact attention algorithm that cuts GPU HBM traffic through tiling and recomputation, optimizes non‑matmul FLOPs, expands sequence‑parallelism and warp‑level work distribution, delivering up to 2× speedup over FlashAttention, near‑GEMM efficiency, and enabling longer‑context Transformer training and inference for AIGC with fastunet and negligible accuracy loss.
The paper introduces ROPE, the first large‑scale cross‑domain product recognition dataset covering detail pages, short videos and live streams, and proposes COPE, a dual‑tower multimodal model that learns unified product embeddings using contrastive and classification losses, achieving superior retrieval and few‑shot classification performance across domains.
This paper introduces LPR4M, a 4‑million‑pair multimodal dataset for livestreaming product recognition, and proposes the RICE model that combines instance‑level contrastive learning with patch‑level cross‑view semantic alignment, demonstrating state‑of‑the‑art performance on both LPR4M and MovingFashion benchmarks.
Deep LearningMultimodal Datasetcross-view alignment
The paper presents an AI‑driven invisible video watermarking system that combines a convolutional encoder/decoder with SE blocks, a simulated‑JPEG degradation layer, multi‑term loss, block‑wise processing, anchor‑based alignment and redundancy voting, achieving high visual fidelity and robust recovery after double‑compression in large‑scale platforms like Bilibili.
This article details how AutoHome's homepage pop‑up leverages precise targeting, extensive feature engineering, and multi‑stage DeepFM‑based models with attention and LHUC modules to accurately identify car‑buying users, improve vehicle‑series recommendations, and achieve a 355% conversion rate increase.
From Turing’s 1950 imitation game to GPT‑4’s multimodal vision‑language capabilities, the field has evolved from simple rule‑based programs like ELIZA and PARRY, through statistical learning and the 2017 Transformer breakthrough, to large-scale generative models that achieve fluent conversation yet still grapple with hallucination and true understanding.
This article introduces the open‑source Pai‑Megatron‑Patch tool from Alibaba Cloud, explains its non‑intrusive patch architecture, enumerates supported models and features such as weight conversion, Flash‑Attention 2.0, FP8 training with Transformer Engine, and provides detailed command‑line examples for model conversion, pre‑training, supervised fine‑tuning, inference, and RLHF reinforcement learning pipelines.
This article reviews two recent AI research papers from NetEase Cloud Music Lab: TG‑Critic, a timbre‑guided, reference‑free singing evaluation model that classifies vocal performance using only audio, and TrOMR, a Transformer‑based end‑to‑end polyphonic optical music recognition system that improves note‑sequence prediction and dataset realism.
This tutorial walks you through implementing a full Transformer model in PyTorch, starting from basic linear‑regression code, adding attention mechanisms, multi‑head attention, encoder‑decoder architecture, training loops, and inference, all reinforced with practical debugging tips.
This article reviews the fundamentals, feature extraction methods, evaluation metrics, and common datasets of content‑based image retrieval (CBIR), discussing traditional low‑level features, local descriptors, unsupervised and supervised learning approaches, and recent deep‑learning models for improving retrieval performance.