Tagged articles

Transformer

444 articles · Page 4 of 5

Feb 16, 2024 · Artificial Intelligence

Can OpenAI’s Sora Redefine Text‑to‑Video Generation? An In‑Depth Technical Review

OpenAI’s newly unveiled Sora model transforms short text prompts into up‑to‑one‑minute high‑definition videos, showcasing advanced diffusion‑Transformer architecture, improved occlusion handling, and detailed visual fidelity, while the article examines its technical breakthroughs, compares it to earlier models, and discusses emerging safety and misuse concerns.

AI safetyOpenAISora

0 likes · 12 min read

Can OpenAI’s Sora Redefine Text‑to‑Video Generation? An In‑Depth Technical Review

Huawei Cloud Developer Alliance

Dec 29, 2023 · Artificial Intelligence

Unlocking LLaMA2: Key Architecture Insights and Deployment Tricks

This recap of the MindSpore public course reviews LLaMA2 fundamentals, compares its Transformer structure, details upgrades from LLaMA1, explains core components like RMSNorm, RoPE, KV‑Cache, Grouped Multi‑Query Attention and SwiGLU, outlines industry LLM optimization methods, and previews the upcoming lecture on the Pengcheng Brain 200B model.

Llama2MindSporeTransformer

0 likes · 5 min read

Unlocking LLaMA2: Key Architecture Insights and Deployment Tricks

Rare Earth Juejin Tech Community

Dec 20, 2023 · Artificial Intelligence

BERT Model Overview: Inputs, Encoder, Fine‑tuning, and Variants

This article explains BERT's WordPiece tokenization, input embeddings (token, segment, and position embeddings), encoder architecture for Base and Large models, fine‑tuning strategies for various NLP tasks, and introduces popular variants such as RoBERTa and ALBERT.

BERTNLPSelf-supervised Learning

0 likes · 12 min read

BERT Model Overview: Inputs, Encoder, Fine‑tuning, and Variants

Rare Earth Juejin Tech Community

Dec 8, 2023 · Artificial Intelligence

Simplifying Transformer Blocks: Removing Residual Connections, LayerNorm, and Other Components without Losing Performance

A recent ETH Zurich paper shows that standard Transformer blocks can be drastically simplified by removing residual connections, LayerNorm, projection and value parameters, and even MLP sub‑block components, achieving up to 16% fewer parameters and comparable training speed and downstream performance on both GPT‑style decoders and BERT models.

AILLMSignal Propagation

0 likes · 11 min read

Simplifying Transformer Blocks: Removing Residual Connections, LayerNorm, and Other Components without Losing Performance

Amap Tech

Dec 4, 2023 · Artificial Intelligence

End-to-End BEV+Transformer Perception and Modeling for High-Definition Map Production

By fusing LiDAR point clouds and camera images into a unified bird‑eye‑view space and applying Transformer‑based perception, multi‑sensor fusion, and graph‑diffusion modeling, the proposed BEV+Transformer framework automatically detects and smooths ground‑level line features and signs for high‑definition maps with centimeter‑level accuracy, boosting production efficiency and reducing cost.

BEVHD mapTransformer

0 likes · 20 min read

End-to-End BEV+Transformer Perception and Modeling for High-Definition Map Production

Rare Earth Juejin Tech Community

Dec 4, 2023 · Artificial Intelligence

An Overview of BERT: Architecture, Pre‑training Tasks, Comparisons, and Applications

This article provides a comprehensive English overview of BERT, covering its original paper, model architecture, pre‑training objectives (Masked Language Model and Next Sentence Prediction), differences from ELMo, GPT and vanilla Transformers, parameter counts, main contributions, and a range of NLP application scenarios such as text classification, sentiment analysis, NER, and machine translation.

BERTNLPNext Sentence Prediction

0 likes · 16 min read

An Overview of BERT: Architecture, Pre‑training Tasks, Comparisons, and Applications

Rare Earth Juejin Tech Community

Nov 26, 2023 · Artificial Intelligence

Overview of T5 (Text-to-Text Transfer Transformer): Architecture, Variants, Experiments, and Applications

This article provides a comprehensive overview of Google's T5 model, detailing its unified text‑to‑text formulation, encoder‑decoder architecture, three model variants, attention mask designs, training strategies, model sizes, experimental results, and key contributions to natural language processing.

Artificial IntelligenceNLPT5

0 likes · 14 min read

Overview of T5 (Text-to-Text Transfer Transformer): Architecture, Variants, Experiments, and Applications

DataFunSummit

Nov 20, 2023 · Artificial Intelligence

Personalized Title Generation and Automatic Cover Image Synthesis for Content Feeds

This article presents a comprehensive overview of personalized title generation—covering keyword‑based, click‑sequence‑based, and author‑style‑based methods using transformer and LSTM models—and describes an end‑to‑end pipeline for automatic cover image synthesis that combines image restoration, Seq2Seq key‑phrase extraction, object detection, and layout generation to improve user engagement in information‑flow scenarios.

AINLPTransformer

0 likes · 12 min read

Personalized Title Generation and Automatic Cover Image Synthesis for Content Feeds

Rare Earth Juejin Tech Community

Nov 15, 2023 · Artificial Intelligence

Understanding the Transformer Architecture: Encoder, Decoder, and Attention Mechanisms

This article explains the Transformer model, comparing it with RNNs, detailing its encoder‑decoder structure, multi‑head and scaled dot‑product attention, embedding layers, feed‑forward networks, and the final linear‑softmax output, supplemented with diagrams and code examples.

Artificial IntelligenceEncoder-DecoderTransformer

0 likes · 10 min read

Understanding the Transformer Architecture: Encoder, Decoder, and Attention Mechanisms

Rare Earth Juejin Tech Community

Nov 12, 2023 · Artificial Intelligence

A Comprehensive Introduction to RNN, LSTM, Attention Mechanisms, and Transformers for Large Language Models

This article provides a thorough overview of large language models, explaining the relationship between NLP and LLMs, the evolution from RNN to LSTM, the fundamentals of attention mechanisms, and the architecture and operation of Transformer models, all illustrated with clear examples and diagrams.

Artificial IntelligenceLSTMNLP

0 likes · 25 min read

A Comprehensive Introduction to RNN, LSTM, Attention Mechanisms, and Transformers for Large Language Models

Baidu Tech Salon

Nov 10, 2023 · Artificial Intelligence

Baidu Search Deep Learning Model Architecture and Optimization Practices

Baidu's Search Architecture team details how its deep‑learning models have evolved to deliver direct answer results via semantic embeddings, describes a massive online inference pipeline that rewrites queries, ranks relevance, and classifies types, and outlines optimization techniques—including data I/O, CPU/GPU balancing, pruning, quantization, and distillation—to achieve high‑throughput, low‑latency search.

BaiduGPU OptimizationInference System

0 likes · 13 min read

Baidu Search Deep Learning Model Architecture and Optimization Practices

NetEase Media Technology Team

Nov 6, 2023 · Artificial Intelligence

Overview of Sequential Recommendation Models

The article surveys sequential recommendation models from early non-deep approaches like FPMC, through RNN-based GRU4Rec and CNN-based Caser, to Transformer-based methods such as SASRec, BERT4Rec, TiSASRec, and recent contrastive-learning techniques, recommending SASRec or its variants for production use.

Transformercontrastive learningdeep learning

0 likes · 17 min read

Overview of Sequential Recommendation Models

DaTaobao Tech

Oct 25, 2023 · Artificial Intelligence

Prompt Engineering, LLM Supervised Fine‑Tuning, and Mobile Tmall AI Assistant Application

The article explains prompt engineering techniques, supervised fine‑tuning of large language models, and their practical deployment in the Mobile Tmall AI shopping assistant, detailing ChatGPT’s generation steps, Transformer architecture, prompt clarity, delimiters, role‑play, few‑shot and chain‑of‑thought prompting, SFT versus pre‑training, LoRA adapters, data collection, Qwen‑14B training configuration, SDK‑based inference, and comprehensive evaluation.

AI assistantLLM fine-tuningModel deployment

0 likes · 14 min read

Prompt Engineering, LLM Supervised Fine‑Tuning, and Mobile Tmall AI Assistant Application

Rare Earth Juejin Tech Community

Oct 21, 2023 · Artificial Intelligence

Understanding LSTM, ELMO, and Transformer Models for Natural Language Processing

This article explains the principles and structures of LSTM networks, introduces the ELMO contextual embedding model with its two‑stage pre‑training and downstream usage, and provides an overview of the Transformer architecture, highlighting their roles in modern NLP tasks.

ELMoLSTMNLP

0 likes · 12 min read

Understanding LSTM, ELMO, and Transformer Models for Natural Language Processing

DataFunTalk

Oct 16, 2023 · Artificial Intelligence

Personalized Title Generation and Automatic Cover Image Synthesis for Information‑Flow Scenarios

This article presents technical approaches for generating personalized article titles and automatically synthesizing cover images, covering keyword‑based, click‑sequence‑based, and author‑style‑based title models, as well as image restoration, key‑information extraction, object detection, and layout generation techniques to improve user engagement in recommendation and search systems.

AI recommendationLSTMTransformer

0 likes · 11 min read

Personalized Title Generation and Automatic Cover Image Synthesis for Information‑Flow Scenarios

Alibaba Cloud Big Data AI Platform

Oct 7, 2023 · Artificial Intelligence

How Alibaba Cloud’s New Transformers and Model Fingerprinting Are Shaping ICCV 2023

Alibaba Cloud’s PAI platform showcased three breakthrough papers at ICCV 2023—including the Scale‑Aware Modulation Transformer for efficient vision backbones, the Stable‑DINO detection transformer with improved matching, and a non‑invasive fingerprinting method for deep image‑restoration models—highlighting its growing impact in AI research.

Model FingerprintingTransformerimage restoration

0 likes · 10 min read

How Alibaba Cloud’s New Transformers and Model Fingerprinting Are Shaping ICCV 2023

Zhuanzhuan Tech

Sep 28, 2023 · Artificial Intelligence

Evolution of Language Models and an Overview of the GPT Series

This article surveys the development of natural language processing from early rule‑based systems through statistical n‑gram models, neural language models, RNNs, LSTMs, ELMo, Transformers and BERT, and then details the architecture, training methods, advantages and limitations of the GPT‑1, GPT‑2, GPT‑3, ChatGPT and GPT‑4 models, concluding with a discussion of future challenges and references.

Artificial IntelligenceGPTNLP

0 likes · 30 min read

Evolution of Language Models and an Overview of the GPT Series

Kuaishou Large Model

Sep 27, 2023 · Artificial Intelligence

DVIS: Decoupled Framework that Sets New SOTA in Video Instance Segmentation

DVIS introduces a decoupled video instance segmentation framework that splits the task into segmentation, tracking, and refinement modules, achieving state-of-the-art performance across VIS, VPS, and VSS benchmarks while maintaining low computational overhead, and demonstrates robustness in both online and offline settings.

Transformercomputer visiondeep learning

0 likes · 12 min read

DVIS: Decoupled Framework that Sets New SOTA in Video Instance Segmentation

DaTaobao Tech

Sep 27, 2023 · Artificial Intelligence

FlashAttention-2: Efficient Attention Algorithm for Transformer Acceleration and AIGC Applications

FlashAttention‑2 is an IO‑aware exact attention algorithm that cuts GPU HBM traffic through tiling and recomputation, optimizes non‑matmul FLOPs, expands sequence‑parallelism and warp‑level work distribution, delivering up to 2× speedup over FlashAttention, near‑GEMM efficiency, and enabling longer‑context Transformer training and inference for AIGC with fastunet and negligible accuracy loss.

AIGCAttention optimizationFlashAttention-2

0 likes · 20 min read

FlashAttention-2: Efficient Attention Algorithm for Transformer Acceleration and AIGC Applications

MaGe Linux Operations

Sep 25, 2023 · Artificial Intelligence

How ChatGPT Works: Inside the Neural Network That Generates Human‑Like Text

Stephen Wolfram explains the inner workings of ChatGPT, covering its transformer architecture, probability‑based word selection, training on massive text corpora, the role of embeddings, neural network layers, attention mechanisms, and the challenges of modeling language, offering a deep technical overview for AI enthusiasts.

AIChatGPTTransformer

0 likes · 80 min read

How ChatGPT Works: Inside the Neural Network That Generates Human‑Like Text

Ant R&D Efficiency

Sep 19, 2023 · Artificial Intelligence

From the Turing Test to GPT‑4: A Historical Overview of Chatbots and Deep Learning

From Turing’s 1950 imitation game to GPT‑4’s multimodal vision‑language capabilities, the field has evolved from simple rule‑based programs like ELIZA and PARRY, through statistical learning and the 2017 Transformer breakthrough, to large-scale generative models that achieve fluent conversation yet still grapple with hallucination and true understanding.

Artificial IntelligenceChatbot HistoryGPT-4

0 likes · 25 min read

From the Turing Test to GPT‑4: A Historical Overview of Chatbots and Deep Learning

Alipay Experience Technology

Sep 12, 2023 · Artificial Intelligence

Demystifying ChatGPT: From Transformer Basics to Business Applications

This article offers a non‑algorithmic engineer’s clear overview of large language models, explaining ChatGPT’s generative‑pre‑training‑transformer foundation, core mechanisms like attention, practical prompt‑engineering tips, and how enterprises can integrate LLMs into data analysis, smart‑customer service, and other business workflows while noting associated risks.

AI ApplicationsChatGPTPrompt Engineering

0 likes · 28 min read

Demystifying ChatGPT: From Transformer Basics to Business Applications

Open Source Linux

Sep 8, 2023 · Artificial Intelligence

How ChatGPT Works: Inside the Neural Network That Generates Human‑Like Text

This article explains the inner workings of ChatGPT, covering how large language models predict the next token using probability distributions, the role of embeddings, the transformer architecture with attention heads, training methods, loss functions, and why such a massive neural network can produce coherent, human‑like language.

ChatGPTTransformerembeddings

0 likes · 79 min read

NetEase Cloud Music Tech Team

Sep 6, 2023 · Artificial Intelligence

Timbre‑Guided TG‑Critic and Transformer‑Based TrOMR: AI Advances in Music Evaluation

This article reviews two recent AI research papers from NetEase Cloud Music Lab: TG‑Critic, a timbre‑guided, reference‑free singing evaluation model that classifies vocal performance using only audio, and TrOMR, a Transformer‑based end‑to‑end polyphonic optical music recognition system that improves note‑sequence prediction and dataset realism.

Audio AnalysisMusic EvaluationOptical Music Recognition

0 likes · 6 min read

Timbre‑Guided TG‑Critic and Transformer‑Based TrOMR: AI Advances in Music Evaluation

Alibaba Cloud Developer

Sep 4, 2023 · Artificial Intelligence

Hands‑On Building a Transformer from Scratch with PyTorch

This tutorial walks you through implementing a full Transformer model in PyTorch, starting from basic linear‑regression code, adding attention mechanisms, multi‑head attention, encoder‑decoder architecture, training loops, and inference, all reinforced with practical debugging tips.

NLPPyTorchTransformer

0 likes · 17 min read

Hands‑On Building a Transformer from Scratch with PyTorch

php Courses

Aug 26, 2023 · Artificial Intelligence

Understanding Generative AI: Concepts, Common Models, and Development Guide

Generative AI, a branch of artificial intelligence that creates novel content such as text, images, and music, works by learning patterns from training data, with common models including GANs, VAEs, autoregressive and Transformer-based architectures, and its development involves task definition, data preparation, model design, training, evaluation, and ethical considerations.

Artificial IntelligenceGaNTransformer

0 likes · 8 min read

Understanding Generative AI: Concepts, Common Models, and Development Guide

Network Intelligence Research Center (NIRC)

Aug 22, 2023 · Artificial Intelligence

LONGNET: Extending Transformers to Over 1 Billion Tokens

LONGNET introduces dilated attention to enable Transformers to process sequences exceeding one billion tokens with linear computational cost, preserving performance on shorter inputs and demonstrating strong results on long‑sequence modeling and standard language tasks.

Dilated AttentionLONGNETLinear Complexity

0 likes · 6 min read

LONGNET: Extending Transformers to Over 1 Billion Tokens

Network Intelligence Research Center (NIRC)

Aug 19, 2023 · Artificial Intelligence

Detecting Time‑Series Anomalies with the Anomaly Transformer’s Association Discrepancy

The article explains how the Anomaly Transformer leverages prior‑ and series‑association discrepancies, a learnable Gaussian kernel, and a Minimax training strategy to distinguish normal from abnormal points in time‑series data, achieving state‑of‑the‑art results on five benchmark datasets.

Association DiscrepancyMinimax TrainingSOTA

0 likes · 6 min read

Detecting Time‑Series Anomalies with the Anomaly Transformer’s Association Discrepancy

Model Perspective

Jul 31, 2023 · Artificial Intelligence

From RNN to ChatGPT: How AIGC Evolved with Transformers and Large Models

This article traces the evolution of AI‑generated content (AIGC) from early RNN‑based Seq2Seq models through the transformative impact of the Transformer architecture, covering key milestones such as UniLM, T5, BART, the GPT series, InstructGPT, and the emergence of ChatGPT.

AI content generationAIGCGPT

0 likes · 9 min read

From RNN to ChatGPT: How AIGC Evolved with Transformers and Large Models

Rare Earth Juejin Tech Community

Jul 31, 2023 · Artificial Intelligence

Overview of Deep Neural Network Architectures

This article provides a comprehensive overview of deep neural network families, introducing twelve major architectures—including Feedforward, CNN, RNN, LSTM, DBN, GAN, Autoencoder, Residual, Capsule, Transformer, Attention, and Deep Reinforcement Learning—explaining their principles, structures, training methods, and offering Python/TensorFlow/PyTorch code examples.

CNNGaNPython

0 likes · 29 min read

Overview of Deep Neural Network Architectures

Rare Earth Juejin Tech Community

Jul 30, 2023 · Artificial Intelligence

ChatGPT Technical Analysis Series – Part 2: GPT1, GPT2, and GPT3 (Encoder vs Decoder, Zero‑Shot, and Scaling)

This article reviews the evolution of the GPT family from GPT‑1 to GPT‑3, comparing encoder‑decoder architectures, explaining the shift from supervised fine‑tuning to zero‑shot and few‑shot learning, and highlighting the architectural and training innovations that enabled large‑scale language models.

Fine-tuningGPTLLM

0 likes · 13 min read

ChatGPT Technical Analysis Series – Part 2: GPT1, GPT2, and GPT3 (Encoder vs Decoder, Zero‑Shot, and Scaling)

Network Intelligence Research Center (NIRC)

Jul 29, 2023 · Artificial Intelligence

Getting Started with GPT: How Generative Pre‑Training and Discriminative Fine‑Tuning Work

This article explains GPT's two‑stage learning—unsupervised generative pre‑training on large raw corpora followed by discriminative fine‑tuning on labeled tasks—detailing the underlying Transformer decoder architecture, loss functions, and task‑specific input transformations.

Fine-tuningGPTGenerative Pre‑Training

0 likes · 5 min read

Getting Started with GPT: How Generative Pre‑Training and Discriminative Fine‑Tuning Work

Rare Earth Juejin Tech Community

Jul 27, 2023 · Artificial Intelligence

Implementing Text‑Based Image Search Using OCR, Transformers, and Vector Databases

This article explains how to build a text‑to‑image search system by first extracting text with OCR, then storing image paths and textual embeddings in a SQLite or Milvus vector database, and finally improving retrieval with Transformer‑based sentence embeddings and image‑captioning models.

MilvusOCRPython

0 likes · 16 min read

Implementing Text‑Based Image Search Using OCR, Transformers, and Vector Databases

Sohu Tech Products

Jul 26, 2023 · Artificial Intelligence

Attention Mechanism, Transformer Architecture, and BERT: An In-Depth Overview

This article provides a comprehensive overview of the attention mechanism, its mathematical foundations, the transformer model architecture—including encoder and decoder components—and the BERT pre‑training model, detailing their principles, implementations, and applications in natural language processing.

Attention MechanismBERTEncoder-Decoder

0 likes · 13 min read

Attention Mechanism, Transformer Architecture, and BERT: An In-Depth Overview

AsiaInfo Technology: New Tech Exploration

Jul 19, 2023 · Artificial Intelligence

How ChatGPT Illuminates the Future Evolution of Data Intelligence

The article examines the rise of artificial general intelligence since 2022, analyzes ChatGPT and other multimodal large models, explains the Transformer architecture, discusses multimodal semantic alignment for AGI, and proposes a four‑level data‑intelligence framework—data, information, knowledge, wisdom—offering a roadmap for future development.

AGIArtificial IntelligenceData Intelligence

0 likes · 16 min read

How ChatGPT Illuminates the Future Evolution of Data Intelligence

Nightwalker Tech

Jul 19, 2023 · Artificial Intelligence

Step‑by‑Step Implementation of Transformer Blocks, Attention, Normalization, Feed‑Forward, Encoder and Decoder in PyTorch

This article provides a comprehensive tutorial on building the core components of a Transformer model—including multi‑head attention, layer normalization, feed‑forward networks, encoder and decoder layers—and assembles them into a complete PyTorch implementation, supplemented with explanatory diagrams and runnable code.

EncoderPyTorchTransformer

0 likes · 13 min read

Step‑by‑Step Implementation of Transformer Blocks, Attention, Normalization, Feed‑Forward, Encoder and Decoder in PyTorch

Nightwalker Tech

Jul 18, 2023 · Artificial Intelligence

Implementing the Input Processing Layer of a Transformer Model: Tokenization, Embedding, and Positional Encoding

This article explains how to build the input processing stage of a Transformer—including tokenization with Hugging Face tokenizers, token‑to‑embedding conversion using BERT models, custom BPE tokenizers, and positional encoding—providing complete Python code examples and test results.

BPEEmbeddingPositional Encoding

0 likes · 14 min read

Implementing the Input Processing Layer of a Transformer Model: Tokenization, Embedding, and Positional Encoding

Rare Earth Juejin Tech Community

Jul 12, 2023 · Artificial Intelligence

Comprehensive Guide to Vision Transformer (ViT): Architecture, Patch Tokenization, Embedding, Fine‑tuning, and Performance

This article provides an in‑depth, English‑language overview of Vision Transformer (ViT), covering its Transformer‑based architecture, patch‑to‑token conversion, token and position embeddings, fine‑tuning strategies such as 2‑D interpolation, experimental results versus CNNs, and the model’s broader significance for multimodal AI research.

Fine‑tuningPatch EmbeddingTransformer

0 likes · 25 min read

Comprehensive Guide to Vision Transformer (ViT): Architecture, Patch Tokenization, Embedding, Fine‑tuning, and Performance

Network Intelligence Research Center (NIRC)

Jun 24, 2023 · Artificial Intelligence

How DFX Achieves Low-Latency Multi-FPGA Acceleration for Transformer Text Generation

The article reviews the DFX system—a multi‑FPGA server that uses model‑parallelism and a ring‑topology interconnect to accelerate GPT‑2 text generation, showing 3.78× higher throughput, 3.99× better energy efficiency, and 8.21× greater cost‑effectiveness compared with a four‑GPU V100 baseline.

Cost EfficiencyFPGAGPT-2

0 likes · 6 min read

How DFX Achieves Low-Latency Multi-FPGA Acceleration for Transformer Text Generation

Rare Earth Juejin Tech Community

Jun 11, 2023 · Artificial Intelligence

Comprehensive Technical Overview of GPT Series, Transformers, and Emerging Capabilities in Large Language Models

This article provides a detailed technical review of the evolution of GPT models, the Transformer architecture, large language model training methods, emergent abilities such as in‑context learning and chain‑of‑thought, multimodal extensions, and the challenges of data, scaling, and alignment, offering a holistic view for researchers and practitioners.

AIGPTInstructGPT

0 likes · 28 min read

Comprehensive Technical Overview of GPT Series, Transformers, and Emerging Capabilities in Large Language Models

Network Intelligence Research Center (NIRC)

Jun 5, 2023 · Artificial Intelligence

How DETR and Its Successors Evolve: A Deep Dive into the DETR Series for Object Detection

This article reviews the original DETR model, analyzes its strengths and weaknesses, and then examines two major follow‑up works—Deformable‑DETR and DAB‑DETR—explaining how they modify attention mechanisms, introduce deformable convolutions and dynamic anchor boxes to accelerate convergence and improve small‑object detection.

DAB-DETRDETRDeformable-DETR

0 likes · 12 min read

How DETR and Its Successors Evolve: A Deep Dive into the DETR Series for Object Detection

Architects' Tech Alliance

May 15, 2023 · Artificial Intelligence

How Transformer Powers ChatGPT: A Deep Dive into Attention and Architecture

This article provides a comprehensive analysis of the Transformer model behind ChatGPT, covering its origin, core mechanisms such as embedding, positional encoding, self‑attention, multi‑head attention, a step‑by‑step translation example, and the broader implications for AI research and industry.

AI architectureAttention MechanismChatGPT

0 likes · 19 min read

How Transformer Powers ChatGPT: A Deep Dive into Attention and Architecture

Full-Stack Trendsetter

May 15, 2023 · Artificial Intelligence

Do You Really Understand ChatGPT, the Era‑Defining AI?

This article explains what ChatGPT is, how it builds on natural-language-processing and the Transformer-based GPT series, details its model-size growth, architectural enhancements, multilingual support, and walks through the tokenization-to-generation pipeline that enables coherent AI-driven conversations.

ChatGPTGPT-3NLP

0 likes · 8 min read

Do You Really Understand ChatGPT, the Era‑Defining AI?

Rare Earth Juejin Tech Community

May 8, 2023 · Artificial Intelligence

Understanding the Principles Behind ChatGPT: NLP, Transformers, and Reinforcement Learning

This article explains how ChatGPT works by covering the fundamentals of natural language processing, generative language models, deep learning, the Transformer architecture, attention mechanisms, few‑shot learning, and the reinforcement‑learning techniques that align its outputs with human preferences.

AIChatGPTNLP

0 likes · 24 min read

Understanding the Principles Behind ChatGPT: NLP, Transformers, and Reinforcement Learning

DataFunSummit

May 6, 2023 · Artificial Intelligence

The Convergence of NLP and Computer Vision: Unified Neural Architectures and Pre‑training Strategies

This talk reviews the recent trend of unifying natural‑language processing and computer‑vision models through shared transformer architectures, masked‑image‑modeling pre‑training, brain‑inspired prediction mechanisms, and practical benefits such as knowledge sharing, multimodal applications, and cost efficiency, while highlighting the evolution of Swin Transformer and its next‑generation variants.

AINLPTransformer

0 likes · 20 min read

The Convergence of NLP and Computer Vision: Unified Neural Architectures and Pre‑training Strategies

21CTO

Apr 27, 2023 · Artificial Intelligence

Demystifying Transformers: A Step‑by‑Step Guide to Self‑Attention and Architecture

This article explains the Transformer model—from its encoder‑decoder structure and self‑attention mechanism to multi‑head attention, positional encoding, residual connections, training loss, and inference strategies—providing a clear, visual walkthrough for readers new to modern NLP architectures.

Self-AttentionTransformerdeep learning

0 likes · 21 min read

Demystifying Transformers: A Step‑by‑Step Guide to Self‑Attention and Architecture

Kuaishou Tech

Apr 26, 2023 · Artificial Intelligence

Dual-Interest Decomposition Head Attention for Sequence Recommendation with Positive and Negative Feedback

The paper proposes a dual‑interest decomposition head‑attention model that uses a feedback‑aware encoding layer, a factorized head attention mechanism, and separate positive/negative interest towers to improve sequence recommendation performance on short‑video and e‑commerce datasets.

AISequence ModelingTransformer

0 likes · 8 min read

Dual-Interest Decomposition Head Attention for Sequence Recommendation with Positive and Negative Feedback

Nightwalker Tech

Apr 26, 2023 · Artificial Intelligence

Understanding GPT: Meaning, Evolution, and Training Process

This article explains what GPT (Generative Pre‑trained Transformer) is, traces its development from early neural networks to the latest GPT‑4 models, and details the three‑stage training pipeline of unsupervised learning, supervised fine‑tuning, and reinforcement learning with human feedback.

GPTTransformer

0 likes · 15 min read

Understanding GPT: Meaning, Evolution, and Training Process

JD Tech

Apr 20, 2023 · Artificial Intelligence

Comprehensive Overview of ChatGPT: AI Background, Technical Foundations, and Commercial Applications

This extensive report examines ChatGPT’s origins, the evolution of artificial intelligence and natural language processing, details the underlying Transformer architecture and GPT series, discusses its limitations, and explores the wide-ranging commercial applications and future prospects of generative AI.

AIGCArtificial IntelligenceChatGPT

0 likes · 34 min read

Comprehensive Overview of ChatGPT: AI Background, Technical Foundations, and Commercial Applications

Python Crawling & Data Mining

Apr 5, 2023 · Artificial Intelligence

Why ChatGPT Works: Inside Transformers, RLHF, and AI’s Latest Breakthroughs

This article explores how ChatGPT’s remarkable abilities stem from the Transformer architecture, reinforcement learning from human feedback, and the insights presented in the fourth edition of "Artificial Intelligence: A Modern Approach," highlighting key AI milestones and technical foundations.

Artificial IntelligenceChatGPTRLHF

0 likes · 9 min read

Why ChatGPT Works: Inside Transformers, RLHF, and AI’s Latest Breakthroughs

DataFunTalk

Mar 18, 2023 · Artificial Intelligence

Review of Deep Learning Model Evolution, Current Limitations, and Future Trends

The article reviews the historical development of deep learning models, highlights scaling limits, universality, interpretability challenges, and hardware constraints, and then outlines future directions such as efficient architectures, self‑supervised training, broader applications, and emerging AI hardware, while also promoting a related ebook.

AI hardwareAI trendsTransformer

0 likes · 6 min read

Review of Deep Learning Model Evolution, Current Limitations, and Future Trends

360 Quality & Efficiency

Mar 10, 2023 · Artificial Intelligence

What Is ChatGPT? Overview, Performance, and Underlying Technologies

This article explains what ChatGPT is, its impressive conversational performance across tasks such as daily dialogue, document writing, math solving, and coding, and details the underlying Transformer architecture, massive data training, and reinforcement learning from human feedback that make the model so powerful.

Artificial IntelligenceChatGPTRLHF

0 likes · 9 min read

What Is ChatGPT? Overview, Performance, and Underlying Technologies

IT Services Circle

Mar 2, 2023 · Artificial Intelligence

Understanding GPT: Word Vectors, Transformers, and Model Architectures (GPT‑2, GPT‑3)

This article provides a concise technical overview of GPT, explaining how word vectors are constructed, how the Transformer architecture with self‑attention and feed‑forward layers processes these vectors, and how GPT‑2 and GPT‑3 extend the model with decoder‑only and large‑scale designs.

AIGPTSelf-Attention

0 likes · 8 min read

Understanding GPT: Word Vectors, Transformers, and Model Architectures (GPT‑2, GPT‑3)

Top Architect

Mar 1, 2023 · Artificial Intelligence

Understanding the Internals of ChatGPT: Neural Networks, Embeddings, and Training Techniques

This article provides a comprehensive overview of how ChatGPT works, covering its probabilistic text generation, transformer architecture, embedding representations, neural network training processes, and the underlying principles that enable large language models to produce coherent and meaningful human-like language.

AIChatGPTTransformer

0 likes · 80 min read

Understanding the Internals of ChatGPT: Neural Networks, Embeddings, and Training Techniques

DataFunTalk

Feb 25, 2023 · Artificial Intelligence

Review of Deep Learning Model Evolution and Future Trends

The article reviews the historical development of deep learning models, highlights current limitations such as scaling inefficiencies, interpretability, and planning, and outlines future directions including efficient architectures, self‑supervised training, cross‑modal transformers, and the impact of AI on fields like life sciences and finance.

AI trendsFuture AISelf-supervised Learning

0 likes · 6 min read

Review of Deep Learning Model Evolution and Future Trends

DataFunTalk

Feb 20, 2023 · Artificial Intelligence

Review of Deep Learning Model Evolution and Future Trends

The article reviews the historical development of deep learning models, highlighting patterns such as scaling limits, increasing generality, interpretability challenges, planning deficiencies, and hardware constraints, and then outlines future directions including efficient architectures, enhanced capabilities, interdisciplinary applications, virtual agents, and novel AI hardware.

AI trendsSelf-supervised LearningTransformer

0 likes · 6 min read

DataFunSummit

Feb 16, 2023 · Artificial Intelligence

Understanding the Transformer Model and Self‑Attention Mechanism with a Complete PyTorch Implementation

This article introduces the Transformer architecture, explains the self‑attention mechanism with visual illustrations, and provides a full, runnable PyTorch code example that implements the encoder‑decoder structure for sequence‑to‑sequence tasks.

NLPPyTorchSelf-Attention

0 likes · 11 min read

Understanding the Transformer Model and Self‑Attention Mechanism with a Complete PyTorch Implementation

Tencent Cloud Developer

Feb 14, 2023 · Artificial Intelligence

ChatGPT: Technology, Impact, and Future Perspectives

Since its November 2022 launch, OpenAI’s ChatGPT—built on Transformer‑based generative AI—has surged to over 100 million users, demonstrated capabilities from MBA exams to software‑engineer interviews, sparked a multibillion‑dollar market with paid subscriptions and Microsoft investment, spurred rival models like Claude, and is reshaping human‑computer interaction while raising ethical concerns and promising multimodal, industry‑specific future applications.

ChatGPTTransformergenerative AI

0 likes · 15 min read

ChatGPT: Technology, Impact, and Future Perspectives

Architect's Guide

Feb 9, 2023 · Artificial Intelligence

Why ChatGPT Is So Powerful: A Technical Overview of NLP Model Evolution

This article explains why ChatGPT performs so well by tracing the evolution of natural‑language processing from rule‑based grammars through statistical n‑gram models to neural architectures like RNNs, LSTMs, attention mechanisms, Transformers, and the massive data and training methods that power modern large language models.

ChatGPTNLPTransformer

0 likes · 14 min read

Why ChatGPT Is So Powerful: A Technical Overview of NLP Model Evolution

21CTO

Feb 6, 2023 · Artificial Intelligence

Understanding the Transformer: How Attention Powers ChatGPT and Modern AI

This article breaks down the Transformer architecture behind ChatGPT, explaining its attention mechanism, embedding, positional encoding, and multi‑head self‑attention, while highlighting the model's impact on AI research, data requirements, and future innovations.

Artificial IntelligenceAttention MechanismChatGPT

0 likes · 18 min read

Understanding the Transformer: How Attention Powers ChatGPT and Modern AI

IT Architects Alliance

Feb 6, 2023 · Artificial Intelligence

Understanding the Transformer Model: A Deep Dive into “Attention Is All You Need”

This article provides a comprehensive, plain‑language walkthrough of the 2017 “Attention Is All You Need” paper, explaining the Transformer’s architecture, core mechanisms such as embedding, positional encoding and self‑attention, and discussing its broader impact on AI research and applications.

AIAttention MechanismTransformer

0 likes · 17 min read

Understanding the Transformer Model: A Deep Dive into “Attention Is All You Need”

Alimama Tech

Feb 1, 2023 · Artificial Intelligence

Video Object of Interest Segmentation (VOIS): Task, Dataset, and Dual-Path Transformer Approach

The paper presents Video Object of Interest Segmentation (VOIS), a new e‑commerce task that locates and segments video instances matching a given product image, introduces the LiveVideos dataset of 2,418 Taobao live‑stream clips, and proposes a dual‑path Swin‑Transformer with cross‑fusion modules that outperforms existing VOS/VIS baselines.

Transformerdatasetinstance segmentation

0 likes · 11 min read

Video Object of Interest Segmentation (VOIS): Task, Dataset, and Dual-Path Transformer Approach

DataFunSummit

Jan 15, 2023 · Artificial Intelligence

Intelligent Writing: AIGC Technologies, Models, Evaluation Metrics, and Real‑World Applications

This article surveys the evolution of AI‑generated content for intelligent writing, covering its definition, key technologies from RNN Seq2Seq to Transformer‑based models such as UniLM, T5, BART and GPT series, evaluation datasets and metrics, product deployments by Datagrand, and the remaining challenges and future directions.

AI WritingAIGCGPT

0 likes · 25 min read

Intelligent Writing: AIGC Technologies, Models, Evaluation Metrics, and Real‑World Applications

DataFunSummit

Jan 14, 2023 · Artificial Intelligence

Key Transformer Model Papers Across Language, Vision, Speech, and Time‑Series Domains

This article surveys the most influential Transformer‑based research papers—from the original Attention Is All You Need work to recent models such as Autoformer and FEDformer—covering breakthroughs in natural language processing, computer vision, speech recognition, and long‑term series forecasting, and provides download links for each.

AITime-Series ForecastingTransformer

0 likes · 17 min read

Key Transformer Model Papers Across Language, Vision, Speech, and Time‑Series Domains

21CTO

Jan 13, 2023 · Artificial Intelligence

How Google’s Muse Is Redefining Text‑to‑Image Generation with Parallel Decoding

Google’s new Muse model, a Transformer‑based text‑to‑image system running on TPUv4, claims to generate 256×256 images in 0.5 seconds—far faster than Imagen—while delivering unprecedented photorealism and deep language understanding through parallel decoding and large‑scale LLM‑conditioned training.

AI researchGoogle MuseLLM conditioning

0 likes · 4 min read

How Google’s Muse Is Redefining Text‑to‑Image Generation with Parallel Decoding

AntTech

Dec 19, 2022 · Artificial Intelligence

TransVCL: Attention‑Enhanced Video Copy Localization Network with Flexible Supervision

TransVCL introduces an end‑to‑end attention‑enhanced video copy localization network that leverages a custom Transformer, correlation‑Softmax similarity matrix, and temporal alignment module, combined with a semi‑supervised learning framework, achieving state‑of‑the‑art performance on VCSL and VCDB benchmarks.

AITransformerattention

0 likes · 13 min read

TransVCL: Attention‑Enhanced Video Copy Localization Network with Flexible Supervision

DataFunTalk

Dec 17, 2022 · Artificial Intelligence

Efficient Spatiotemporal Self‑Attention Transformer (Patch Shift Transformer) for Video Action Recognition

This article introduces a lightweight spatiotemporal self‑attention transformer, called Patch Shift Transformer, which achieves competitive video action recognition performance on datasets such as Kinetics‑400, Sth‑v1/v2, and Diving48 without increasing computational cost or parameters, and details its design, experiments, and speed advantages.

ECCV 2022Spatiotemporal ModelingTransformer

0 likes · 5 min read

Efficient Spatiotemporal Self‑Attention Transformer (Patch Shift Transformer) for Video Action Recognition

Rare Earth Juejin Tech Community

Dec 8, 2022 · Artificial Intelligence

ChatGPT: Development History, Technical Principles, and Future Investment Trends

This article reviews ChatGPT’s rapid rise, compares it with GPT‑3, explains the underlying transformer and reinforcement‑learning‑from‑human‑feedback technologies, outlines the evolution of natural‑language processing, and discusses emerging AI investment opportunities and future trends.

AIChatGPTNLP

0 likes · 12 min read

ChatGPT: Development History, Technical Principles, and Future Investment Trends

HelloTech

Oct 19, 2022 · Artificial Intelligence

Intelligent Creative System: Types, Quality Evaluation, Generation Models, and Optimization

The Intelligent Creative System defines advertising creatives across formats, evaluates image and text quality using reference‑based metrics and models like DeepBIQ, generates multimodal ads via GANs and Transformers, and selects optimal variants through bandit‑based CTR prediction and multimodal fusion, enabling scalable, data‑driven creative production.

AIBandit ModelGaN

0 likes · 10 min read

Intelligent Creative System: Types, Quality Evaluation, Generation Models, and Optimization

Rare Earth Juejin Tech Community

Oct 10, 2022 · Artificial Intelligence

A Beginner’s Journey into Vision Transformers (ViT) for Computer Vision Engineers

This article introduces the fundamentals of Vision Transformers (ViT) for computer‑vision developers, starting with an overview of the transformer architecture, detailed explanation of self‑attention and multi‑head attention, and step‑by‑step PyTorch code examples that illustrate query, key, value computation and attention scoring.

PyTorchSelf-AttentionTransformer

0 likes · 12 min read

A Beginner’s Journey into Vision Transformers (ViT) for Computer Vision Engineers

HaoDF Tech Team

Oct 8, 2022 · Artificial Intelligence

Exploring Transformer Technology and Its Applications in NLP, Computer Vision, and OCR at Haodf.com

This article introduces the Transformer architecture, explains its attention mechanism, details its adaptations for natural language processing, computer vision, and OCR tasks, and presents experimental results of various models such as BERT, ELECTRA, Swin Transformer, and CRNN-BCN on large-scale medical data from Haodf.com.

Model EvaluationNLPOCR

0 likes · 39 min read

Exploring Transformer Technology and Its Applications in NLP, Computer Vision, and OCR at Haodf.com

Kuaishou Audio & Video Technology

Sep 29, 2022 · Artificial Intelligence

How DeViT Revolutionizes Video Inpainting with Deformed Vision Transformers

The article introduces DeViT, a novel Deformed Vision Transformer framework for video inpainting that leverages a deformable patch homography estimator, mask‑pruned attention, and spatio‑temporal weight adaptation, achieving state‑of‑the‑art results on benchmark datasets and highlighting its potential for advanced video editing tools.

DeViTTransformerVideo Inpainting

0 likes · 10 min read

How DeViT Revolutionizes Video Inpainting with Deformed Vision Transformers

ELab Team

Sep 23, 2022 · Artificial Intelligence

Fine‑Tune a Chinese BERT Model for Cloze Tasks in 30 Minutes

This tutorial walks you through NLP fundamentals, the evolution of BERT, the concept of pre‑trained models, and a step‑by‑step guide to fine‑tune a Chinese BERT on a cloze‑style task, complete with code snippets and verification results.

BERTChineseCloze Task

0 likes · 13 min read

Fine‑Tune a Chinese BERT Model for Cloze Tasks in 30 Minutes

Alimama Tech

Sep 21, 2022 · Artificial Intelligence

EXTR: Click-Through Rate Prediction with Externalities in E-Commerce Sponsored Search

The paper introduces EXTR, a Transformer‑based CTR prediction model that jointly encodes diverse externalities from surrounding organic results and ads and infers missing ad placements via a Potential Allocation Generator, achieving superior AUC, COPC and LogLoss on Taobao data and deployment in Alibaba’s advertising system.

AdvertisingExternalitiesTransformer

0 likes · 11 min read

EXTR: Click-Through Rate Prediction with Externalities in E-Commerce Sponsored Search

Architects' Tech Alliance

Aug 31, 2022 · Artificial Intelligence

Performance Evaluation of Transformer Models on the Inspur NF5488A5 GPU Server

This article presents a detailed benchmark of four Transformer models of varying sizes trained on the high‑end Inspur NF5488A5 GPU server, compares its NVSwitch‑based interconnect with a PCIe‑based system, and analyzes the impact of model scale, tensor parallelism, and hardware bandwidth on training efficiency.

DeepSpeedGPU serverMegatron-LM

0 likes · 12 min read

Performance Evaluation of Transformer Models on the Inspur NF5488A5 GPU Server

Alibaba Cloud Developer

Jul 28, 2022 · Artificial Intelligence

Unlock Chinese Text‑to‑Image Generation with EasyNLP: Models, Code & Tutorials

This article introduces EasyNLP's Chinese text‑to‑image generation framework, explains the underlying Transformer‑VQGAN architecture, provides model specifications, showcases sample outputs, and offers step‑by‑step code and command‑line instructions for fine‑tuning and inference.

Chinese AIEasyNLPMultimodal

0 likes · 20 min read

Unlock Chinese Text‑to‑Image Generation with EasyNLP: Models, Code & Tutorials

Alibaba Cloud Big Data AI Platform

Jul 11, 2022 · Artificial Intelligence

How Structure-Aware Sparse Attention Boosts Long-Code Transformers

The SASA model, a structure‑aware sparse‑attention Transformer developed by Alibaba Cloud PAI and Prof. Gao Ming’s team, improves long‑code sequence processing by sparsifying self‑attention using top‑k frequency and AST pattern matrices, achieving higher performance and lower memory/computation costs on CodeXGLUE benchmarks.

ASTCode UnderstandingLong Sequences

0 likes · 8 min read

How Structure-Aware Sparse Attention Boosts Long-Code Transformers

DataFunTalk

Jul 9, 2022 · Artificial Intelligence

Graph Neural Networks Enter the Transformer Era – Seminar by Dr. Zheng Shuxin

The LOGS seminar on July 9, 2022 featured Dr. Zheng Shuxin from Microsoft Research presenting an overview of Transformer models, their success in NLP and CV, recent breakthroughs in applying Transformers to graph data, and future directions for graph processing.

AI SeminarMicrosoft ResearchTransformer

0 likes · 4 min read

Graph Neural Networks Enter the Transformer Era – Seminar by Dr. Zheng Shuxin

Xiaohongshu Tech REDtech

Jun 20, 2022 · Artificial Intelligence

Action Sequence Verification in Videos with CosAlignment Transformer (CAT)

The paper introduces Action Sequence Verification (ASV), a task that determines whether two videos follow the same ordered actions, provides the Chemical Sequence Verification dataset and re‑annotated COIN‑SV and Diving48‑SV sets, and proposes the CosAlignment Transformer (CAT) with intra‑step feature extraction, a Transformer‑based inter‑step encoder, and a sequence‑alignment loss that outperforms prior baselines and serves as a pre‑training model for video retrieval and classification.

Action VerificationMultimodalTransformer

0 likes · 7 min read

Action Sequence Verification in Videos with CosAlignment Transformer (CAT)

Python Programming Learning Circle

Jun 5, 2022 · Artificial Intelligence

The Rise of Hugging Face: From Emoji Logo to Leading AI Platform

From its quirky start as a teenage iPhone chatbot to becoming the central hub for open‑source transformer models, Hugging Face has grown into a fast‑rising AI platform, securing $100 million Series C funding, serving thousands of organizations, and aiming to democratize machine learning.

AIFundingHugging Face

0 likes · 7 min read

DaTaobao Tech

May 24, 2022 · Artificial Intelligence

GEN-VLKT: Simplify Association and Enhance Interaction Understanding for HOI Detection

GEN‑VLKT introduces a Guided‑Embedding Network with position‑ and instance‑guided embeddings to remove costly post‑processing and leverages CLIP‑based visual‑linguistic knowledge transfer for interaction understanding, achieving state‑of‑the‑art HOI detection performance and zero‑shot capability, now deployed in Alibaba’s Taobao services.

CLIPHOI detectionTransformer

0 likes · 7 min read

GEN-VLKT: Simplify Association and Enhance Interaction Understanding for HOI Detection

DataFunTalk

May 7, 2022 · Artificial Intelligence

Intelligent Recommendation Selling Point Generation: Architecture, Core AI Techniques, Model Development, and Product Impact

This article explains how JD's intelligent recommendation selling point system leverages NLP, BERT, Transformer and pointer‑generator models to automatically create short, personalized product highlights, describing the technical background, system architecture, model training pipeline, online/offline monitoring, and the resulting business benefits.

BERTNLPRecommendation Systems

0 likes · 13 min read

Intelligent Recommendation Selling Point Generation: Architecture, Core AI Techniques, Model Development, and Product Impact

AntTech

Apr 27, 2022 · Artificial Intelligence

Pyraformer: Low-Complexity Pyramidal Attention for Long-Range Time Series Modeling and Forecasting

The paper introduces Pyraformer, a low‑complexity pyramidal‑attention Transformer that captures multi‑scale temporal dependencies with linear time‑space complexity, achieving superior single‑step and long‑range forecasting performance on real‑world datasets while supporting green‑computing capacity management.

PyraformerTime Series ForecastingTransformer

0 likes · 14 min read

Pyraformer: Low-Complexity Pyramidal Attention for Long-Range Time Series Modeling and Forecasting

DataFunSummit

Apr 25, 2022 · Artificial Intelligence

Token‑Level Pipeline Parallelism for Transformer‑based Language Models (TeraPipe)

The article introduces a token‑level pipeline parallelism strategy that splits the sequence‑length dimension of Transformer‑based language models, explains why this approach is feasible, presents a dynamic‑programming formulation for optimal slicing, discusses engineering challenges, and evaluates its performance on large GPT models.

Large Language ModelsPipeline ParallelismToken-level

0 likes · 13 min read

Token‑Level Pipeline Parallelism for Transformer‑based Language Models (TeraPipe)

Code DAO

Apr 18, 2022 · Artificial Intelligence

Transformer‑Based Denoising AutoEncoder (TSDAE) for Job Description Embeddings (Job2Vec)

This article explains how TSDAE, a transformer‑based denoising auto‑encoder, converts noisy job description sentences into robust vector embeddings, details its training process, loss function, dataset preparation, and demonstrates using FAISS for similarity search on the resulting Job2Vec representations.

AutoencoderFAISSNLP

0 likes · 4 min read

Transformer‑Based Denoising AutoEncoder (TSDAE) for Job Description Embeddings (Job2Vec)

Kuaishou Large Model

Apr 6, 2022 · Artificial Intelligence

How Transformers Revolutionize Image Style Transfer: Introducing StyTr²

This article reviews the limitations of traditional CNN‑based image stylization, explains how Transformer architectures overcome these issues with global context and self‑attention, and presents the novel StyTr² method with content‑aware positional encoding that achieves superior, detail‑preserving style transfer results.

Transformercomputer visiondeep learning

0 likes · 8 min read

How Transformers Revolutionize Image Style Transfer: Introducing StyTr²

IEG Growth Platform Technology Team

Feb 14, 2022 · Artificial Intelligence

Multimodal Evolution and Application in Tencent Game Advertising System

This article describes the end‑to‑end multimodal modeling pipeline—covering text, image, and video understanding, model evolution from shallow to deep networks, key‑frame extraction, fine‑tuning, and multimodal fusion—used in Tencent's game ad exchange platform, along with practical deployment challenges and solutions.

This article surveys sequential recommendation methods, covering standard models such as pooling, RNN, CNN, attention, and Transformer, as well as long‑short term, multi‑interest, multi‑behavior approaches, and recent advances like contrastive learning, highlighting their impact on recommendation performance.

RNNTransformerattention

0 likes · 8 min read

Sequential Recommendation Algorithms: Overview and Techniques

JD Retail Technology

Nov 16, 2021 · Artificial Intelligence

Intelligent Online Selling Point Extraction for E‑Commerce Recommendation (IOSPE) Wins AAAI 2022 Innovation Award

The IOSPE system, which uses BERT‑based scoring, transformer‑pointer generation, and personalized distribution to automatically extract and generate selling points for millions of e‑commerce products, earned the AAAI 2022 Artificial Intelligence Innovation Application Award and has boosted click‑through rates and user dwell time across JD.com platforms.

AIBERTInnovation Award

0 likes · 6 min read

Intelligent Online Selling Point Extraction for E‑Commerce Recommendation (IOSPE) Wins AAAI 2022 Innovation Award

JD Retail Technology

Nov 16, 2021 · Artificial Intelligence

Automatic Product Copywriting for E-Commerce: The APCG System and Its AI Innovations

The APCG system, awarded the AAAI 2022 Innovation Application Prize, automatically generates e‑commerce product copy using a Transformer‑Pointer network and a pretrained sequence‑to‑sequence model, incorporates quality control, employs novel pretraining tasks, and has produced millions of descriptions that boost CTR, CVR, and GMV.

AINatural Language GenerationTransformer

0 likes · 6 min read

Automatic Product Copywriting for E-Commerce: The APCG System and Its AI Innovations

58 Tech

Oct 12, 2021 · Artificial Intelligence

Seq2Seq Approaches for Phone Number Extraction from Two‑Speaker Voice Dialogues

This article presents a practical study of extracting phone numbers from two‑speaker voice dialogues using Seq2Seq models—including LSTM, GRU with attention and feature fusion, and Transformer—detailing data characteristics, model architectures, training strategies, experimental results, and comparative analysis showing the GRU‑Attention approach achieving the best performance.

GRULSTMNLP

0 likes · 13 min read

Seq2Seq Approaches for Phone Number Extraction from Two‑Speaker Voice Dialogues