Tagged articles
Attention
88 articles · Page 1 of 1
Prefix Sliding keeps the task prefix and a sliding window of recent tokens while evicting older intermediate tokens from the KV cache, enabling up to three‑fold speedups for long‑chain inference without retraining and extending reinforcement‑learning rollouts beyond 100 k tokens.
AttentionEfficiencyKV Cache
0 likes · 11 min read

SCoPE (Sightline‑Coordinate Positional Encoding) replaces the traditional (u,v,t) pixel‑grid coordinates in video DiT models with camera‑ray coordinates injected into attention, achieving near‑zero parameter overhead while markedly improving camera control, cross‑view consistency, and revisit consistency across model scales and stylized inputs.
AttentionComputer VisionRay‑Space Positional Encoding
0 likes · 9 min read


ArchitectAug 9, 2026 · Artificial Intelligence
This article reviews the recent “Memory for Large Language Models” survey, outlining three orthogonal design axes—representation, update dynamics, and persistence—and maps them to engineering concerns such as work‑set, compressed state, long‑term items, and raw evidence, while discussing evaluation dimensions and practical implementation guidelines for agent systems.
Agent ArchitectureAttentionLLM memory
0 likes · 20 min read

The article reviews Coleman Hooper's 2026 Berkeley PhD thesis, which shows that LLM inference is increasingly limited by memory bandwidth and capacity, and proposes a four‑pronged approach—weight quantization, KV‑cache quantization, selective context loading, and multipole attention—to dramatically improve memory efficiency and throughput.
AttentionKV CacheLLM
0 likes · 12 min read

A systematic ICML 2026 study shows that sharing the K and V projection matrices in Transformers reduces KV cache size by half while incurring less than 5% perplexity degradation, offering a simple, retrain‑once solution for long‑context and edge inference.
AttentionEfficiencyKV Cache
0 likes · 10 min read

This article explains ten core AI concepts—including tokens, embeddings, attention, the Transformer architecture, large language models, hallucination, temperature, context windows, Retrieval‑Augmented Generation, and AI agents—so developers can understand model behavior, avoid common pitfalls, and build reliable AI applications.
AI FundamentalsAI agentsAttention
0 likes · 15 min read

This article walks through the full LLM pipeline—from tokenization and embedding to transformer attention, pre‑training, instruction tuning, RLHF, inference, context windows, Retrieval‑Augmented Generation and agents—explaining each component, its role, and the engineering challenges such as hallucination and safety.
AttentionEmbeddingLLM
0 likes · 17 min read

This article presents GQLA, a single‑author variant of MLA that eliminates three hardware‑related drawbacks of MLA, demonstrates how it achieves balanced compute‑memory performance on both high‑end H100 and more modest H20 GPUs, and details conversion methods (TransGQLA) and sparse extensions with concrete benchmark results.
AttentionGQLALLM
0 likes · 16 min read

The INTRA paper shows that using a decoder’s cross‑attention as an internal retrieval mechanism eliminates the need for a separate retriever, achieving state‑of‑the‑art multihop QA performance with only 164 K trainable parameters and shared pre‑encoded representations.
AttentionINTRARAG
0 likes · 8 min read

The article demystifies large language models by explaining their core task of next‑token prediction, tokenization, vector semantics, Transformer attention, massive training, hallucination risks, prompt design, and tool integration, showing how these mechanisms work together and why verification is essential.
AI HallucinationAgentAttention
0 likes · 14 min read


Geek LabsMay 6, 2026 · Artificial Intelligence
The article introduces two practical GitHub repositories—how-to-train-your-gpt, a step‑by‑step guide that builds a LLaMA‑style GPT model across 12 chapters, and dictionary-of-ai-coding, a plain‑language glossary of AI‑coding terms—showing how they together provide a complete understanding of modern LLM fundamentals and terminology.
AIAttentionGPT
0 likes · 9 min read

The article explains that AI agents’ forgetting and hallucinations stem from token‑level attention scores causing key‑value cache eviction before retrieval, then surveys KV‑cache basics, naive growth, streaming‑LLM windowing, SnapKV’s attention‑guided compression, token‑retention studies, Memory Sparse Attention, compares these methods, and discusses practical system pitfalls and design implications.
AI agentsAttentionKV Cache
0 likes · 20 min read


Geek LabsApr 20, 2026 · Artificial Intelligence
This open‑source tutorial breaks down large language model internals into 11 detailed topics—covering BPE tokenization, attention mathematics, backpropagation, transformer architecture, KV‑Cache, Paged and Flash Attention, and frontier techniques—each with numeric derivations and Python code, making it ideal for developers and interview preparation.
AttentionFlash AttentionInference Optimization
0 likes · 5 min read

This article walks engineers through the full LLM stack—from tokenization and positional encoding to transformer blocks, efficient fine‑tuning, quantization, and production‑grade inference techniques such as KV‑cache, FlashAttention, PagedAttention, continuous batching, and speculative decoding—highlighting trade‑offs, toolchains, and practical workflow steps.
AttentionLLMLoRA
0 likes · 13 min read

The article explains how KV cache eliminates redundant computations in autoregressive LLM generation, detailing the attention mechanism, the O(n²) waste of recomputing K and V, the cache‑based solution, its impact on time‑to‑first‑token, and the memory‑vs‑speed trade‑off.
AttentionInference OptimizationKV Cache
0 likes · 7 min read


SuanNiMar 29, 2026 · Artificial Intelligence
This article analyzes the AVO system, an autonomous AI agent that replaces traditional evolutionary search pipelines to iteratively improve CUDA attention kernels on NVIDIA's Blackwell B200 GPU, achieving up to 10.5% higher throughput than hand‑tuned implementations after a week of nonstop optimization.
AIAttentionCUDA
0 likes · 13 min read

This article explains how Attention Residuals (AttnRes) replace traditional residual shortcuts with layer‑wise attention, details the mathematical reformulation, design constraints, static‑Q trick, full and block variants, and presents experimental evidence of significant accuracy gains with modest overhead.
AttentionNLPRMSNorm
0 likes · 11 min read

This article analyzes the newly released Attention Residuals paper, explaining how learnable attention weighting replaces fixed residual addition to mitigate information dilution in deep LLMs, detailing the proposed Block AttnRes design, engineering trade‑offs, experimental results, and its significance for foundational model architecture.
AttentionBlock AttentionLLM
0 likes · 9 min read

This article walks through MiniMind's Attention.forward implementation, explaining why Q, K, and V are created, how tensors are reshaped for multi‑head attention, the role of masks, KV cache, GQA, and how each token aggregates information from the entire context.
AttentionGQAKV Cache
0 likes · 21 min read

NeurIF introduces a neural instrumented factorization framework that leverages company features as instruments, combines spatial and temporal attention to learn time‑varying latent factors and their loadings, achieves 1‑18% RMSE improvement over transformer baselines, and produces statistically significant long‑short portfolios that explain cross‑sectional pricing anomalies.
AttentionNeurIFasset pricing
0 likes · 15 min read

This article provides a comprehensive, step‑by‑step walkthrough of the Transformer architecture, covering input embedding, positional encoding, the mechanics of Q‑K‑V attention, scaled dot‑product formulas, multi‑head and masked attention, feed‑forward networks, residual connections, layer normalization, decoder generation, and recent attention‑optimization techniques.
AttentionFeed-Forward NetworkPositional Encoding
0 likes · 39 min read

This article explains the inherent limitations of single-head attention in Transformers, illustrates them with a linguistic example, and then details how multi-head attention works through independent projection matrices, splitting and concatenation, ultimately boosting model expressiveness, robustness, and interpretability.
AIAttentionmulti-head
0 likes · 9 min read


ArchitectDec 15, 2025 · Artificial Intelligence
This comprehensive guide explains the fundamentals of large language model (LLM) architectures, covering the original Transformer, tokenization, embeddings, positional encoding, attention mechanisms, feed‑forward networks, layer stacking, a step‑by‑step translation example, and the latest open‑source and hybrid LLM designs shaping the field.
AttentionEmbeddingLLM
0 likes · 41 min read

This article provides a comprehensive, step‑by‑step explanation of the Transformer architecture that underpins large language models, covering tokenization, embeddings, positional encoding, attention mechanisms, feed‑forward networks, layer stacking, a detailed translation example, visualized attention weights, and a survey of recent open‑source LLM designs such as DeepSeek V3, OLMo 2, and Gemma 3.
AttentionEmbeddingLLM
0 likes · 38 min read

This article explains the evolution from RNNs through Encoder‑Decoder models to Transformers, detailing self‑attention, multi‑head attention, and masked attention, and then describes what Large Language Models are, their key components, capabilities, limitations, and common applications.
AIAttentionLLM
0 likes · 9 min read

This article deeply explores the Transformer architecture by combining original paper principles with PyTorch source code, covering encoder‑decoder design, positional encoding assumptions, core parameters, residual connections, attention mechanisms, and detailed implementation snippets to help readers understand and reproduce the model.
AttentionPositional EncodingPyTorch
0 likes · 22 min read


ArchitectSep 16, 2025 · Artificial Intelligence
This article introduces the Transformer architecture, explaining its attention mechanism, encoder‑decoder design, training and inference processes, and why it surpasses RNN‑based models, while also covering common applications and variations in natural language processing.
AttentionNLPTransformer
0 likes · 13 min read

This article explains why Transformer models surpass traditional RNN‑based seq2seq architectures by introducing self‑attention, multi‑head attention, and positional encoding, detailing the inner workings of encoders, decoders, and attention mechanisms, and comparing their advantages and limitations across NLP and vision tasks.
AttentionGRULSTM
0 likes · 30 min read

A recent Meta paper introduces a rotation‑invariant 2‑simplicial attention mechanism, demonstrates its superior scaling‑law coefficients over standard dot‑product attention, and provides experimental evidence of improved token efficiency and model performance under constrained token budgets.
2-simplicialAttentionMeta
0 likes · 11 min read

This article demystifies how Transformers are trained by showing that all their linear layers have learnable weights and biases, and that the attention mechanism—including softmax and dot‑product operations—is fully differentiable and updated via standard back‑propagation.
AttentionBackpropagationPyTorch
0 likes · 7 min read

This article compares two recent papers that investigate why large reasoning models such as Llama and Qwen show degraded instruction‑following performance when using chain‑of‑thought prompting, analyzing attention patterns, training effects, and proposed mitigation strategies.
AttentionLLMchain-of-thought
0 likes · 11 min read

Introducing MambaIRv2, an image restoration backbone that replaces Mamba’s causal scanning with an attentive state‑space module, achieving single‑direction scanning, reduced parameters and computation, and superior performance on lightweight and classic super‑resolution, JPEG artifact removal, and denoising tasks, as validated by CVPR‑2025 results.
AttentionComputer VisionMambaIRv2
0 likes · 8 min read

This article presents a theoretical and experimental analysis of converting Group Query Attention (GQA) models to Multi‑Head Linear Attention (MLA) using the TransMLA method, demonstrating superior expressiveness and performance on DeepSeek‑based large language models while keeping KV‑Cache costs unchanged.
AttentionDeepSeekGQA
0 likes · 11 min read

This article explains the FlashAttention algorithm, its memory‑efficient tiling and recomputation techniques, and how enabling the flash_attn flag dramatically speeds up Qwen‑series large models while outlining hardware, software requirements and potential trade‑offs.
AttentionFlashAttentionGPU optimization
0 likes · 8 min read

From the perceptron’s inception in 1958 to modern Transformer-based models like GPT, this article traces the evolution of deep learning, explaining foundational architectures such as DNNs, CNNs, RNNs, LSTMs, attention mechanisms, and recent innovations like DeepSeek’s MLA, highlighting their principles and impact.
AttentionGPTMLA
0 likes · 19 min read

This talk presents the Ant Group team's recent work on large‑model inference memory optimization, covering GPU memory challenges, virtual memory management (VMM), the Virtual Tensor framework, LayerKV techniques, performance comparisons with Page Attention and FlashAttention, and extensive experimental results demonstrating reduced latency and higher QPS.
AttentionGPULarge Model Inference
0 likes · 25 min read


NewBeeNLPNov 18, 2024 · Artificial Intelligence
This article examines various techniques for compressing and accelerating the KV cache in transformer models—including MQA, GQA, MLA, sliding‑window and linear attention, flash attention, page and ring attention, as well as mixed‑precision training and ZeRO parallelism—providing code snippets, implementation details, and practical trade‑offs.
AttentionFlashAttentionKV Cache
0 likes · 17 min read

This article provides a comprehensive, engineer‑focused breakdown of transformer architecture—including tensor fundamentals, matrix multiplication, GPU theoretical compute, attention and FFN mechanics, quantitative parameter and FLOP analysis, performance metrics like MFU, parallelism strategies, variant optimizations, and practical exercise questions—offering clear insight into large‑model efficiency and scaling.
AttentionFFNGPU performance
0 likes · 33 min read


JD TechJun 7, 2024 · Artificial Intelligence
This article explains the fundamentals of attention mechanisms, including biological inspiration, the evolution from early visual attention to modern self‑attention in Transformers, details the scaled dot‑product calculations, positional encoding, and multi‑head attention, illustrating how these concepts enable efficient parallel processing of sequence data.
AIAttentionPositional Encoding
0 likes · 12 min read

This article provides concise technical explanations for 25 common questions about Transformer models, covering scaled dot‑product attention scaling, separate Q/K projections, feed‑forward network design, attention variants, normalization, LoRA versus full‑parameter training, KV‑cache, pre‑ and post‑norm, computational cost analysis, and advanced position‑encoding techniques.
AttentionLLMLoRA
0 likes · 25 min read

This tutorial walks through implementing a basic Denoising Diffusion Probabilistic Model in Python, explaining the forward noise schedule, reverse denoising training, and providing complete code for noise schedules, diffusion functions, residual and attention blocks, a UNet architecture, loss computation, and a training loop.
AttentionDDPMPython
0 likes · 26 min read

The paper introduces FIN (Fragment and Integrate Network), a novel spatial‑temporal model that extracts multiple sub‑sequences from ultra‑long user behavior logs, applies simplified and multi‑head attention, and fuses them with physically meaningful set operations, achieving up to 5.7% CTR lift and 7.3% RPM improvement in real‑world food‑delivery advertising.
AIAttentionCTR prediction
0 likes · 23 min read


NewBeeNLPMar 18, 2024 · Artificial Intelligence
This article provides a comprehensive technical guide on Retrieval‑Augmented Generation (RAG), open‑source large language models such as LLaMA, fine‑tuning methods, evaluation metrics, memory‑optimization tricks, and attention‑related optimizations for modern AI systems.
AttentionLLMLangChain
0 likes · 19 min read

This article explains the Transformer model, comparing it with RNNs, detailing its encoder‑decoder structure, multi‑head and scaled dot‑product attention, embedding layers, feed‑forward networks, and the final linear‑softmax output, supplemented with diagrams and code examples.
AttentionEncoder-DecoderTransformer
0 likes · 10 min read

This article provides a thorough overview of large language models, explaining the relationship between NLP and LLMs, the evolution from RNN to LSTM, the fundamentals of attention mechanisms, and the architecture and operation of Transformer models, all illustrated with clear examples and diagrams.
AttentionLSTMNLP
0 likes · 25 min read

This article introduces Social4Rec, a video recommendation algorithm that tackles user cold‑start problems by extracting and integrating social interest information through coarse‑ and fine‑grained interest extractors, attention‑based fusion, and extensive offline and online experiments demonstrating significant CTR improvements.
Attentioncold-startdeep learning
0 likes · 14 min read

This tutorial walks you through implementing a full Transformer model in PyTorch, starting from basic linear‑regression code, adding attention mechanisms, multi‑head attention, encoder‑decoder architecture, training loops, and inference, all reinforced with practical debugging tips.
AttentionNLPPyTorch
0 likes · 17 min read

This article provides a comprehensive tutorial on building the core components of a Transformer model—including multi‑head attention, layer normalization, feed‑forward networks, encoder and decoder layers—and assembles them into a complete PyTorch implementation, supplemented with explanatory diagrams and runnable code.
AttentionEncoderPyTorch
0 likes · 13 min read

The information processing theory, emerging in the 1950s‑60s, likens human cognition to computer operations, detailing how perception, attention, memory, conceptual knowledge, reasoning, and feedback mechanisms transform sensory input into mental representations and guide behavior, influencing cognitive psychology, education, and HCI.
AttentionMemorycognitive psychology
0 likes · 4 min read

This article proposes a graph‑based node representation method that combines static attribute graphs and dynamic interaction graphs with multi‑level attention to alleviate user and item cold‑start problems in recommendation systems, achieving notable AUC improvements on sparsified MovieLens datasets.
AttentionEmbeddingGraph Neural Network
0 likes · 9 min read

This article reviews the original DETR model, analyzes its strengths and weaknesses, and then examines two major follow‑up works—Deformable‑DETR and DAB‑DETR—explaining how they modify attention mechanisms, introduce deformable convolutions and dynamic anchor boxes to accelerate convergence and improve small‑object detection.
AttentionDAB-DETRDETR
0 likes · 12 min read

This article explains why ChatGPT performs so well by tracing the evolution of natural‑language processing from rule‑based grammars through statistical n‑gram models to neural architectures like RNNs, LSTMs, attention mechanisms, Transformers, and the massive data and training methods that power modern large language models.
AttentionChatGPTNLP
0 likes · 14 min read


AntTechDec 19, 2022 · Artificial Intelligence
TransVCL introduces an end‑to‑end attention‑enhanced video copy localization network that leverages a custom Transformer, correlation‑Softmax similarity matrix, and temporal alignment module, combined with a semi‑supervised learning framework, achieving state‑of‑the‑art performance on VCSL and VCDB benchmarks.
AIAttentionTransformer
0 likes · 13 min read

The paper proposes graph‑based deep recall models that enrich sparse user behavior sequences in video recommendation by integrating content knowledge graphs and adaptive attention mechanisms, demonstrating that variants such as GADM, SGGA, and SGGGA significantly boost click‑through rates in online experiments.
AttentionGraph Neural Networksknowledge graph
0 likes · 11 min read

This talk presents Alibaba's travel intent prediction system, detailing the unique challenges of low‑frequency, multi‑source travel behavior, the multi‑granular CNN and time‑attention model architecture, experimental comparisons with baselines, and how integrated user interest modeling improves recommendation performance.
Attentiondeep learningmachine learning
0 likes · 11 min read

This presentation introduces Tencent Angel Graph's NDLS and GAMLP techniques that address GNN scalability and flexibility by adaptively selecting propagation depth per node, employing node‑wise feature and label propagation with attention mechanisms, and demonstrating superior performance on large‑scale and sparse graph benchmarks.
AttentionGAMLPNode Adaptive
0 likes · 16 min read


Code DAODec 25, 2021 · Artificial Intelligence
This article walks through building an image‑captioning system using a TensorFlow 2.0 encoder‑decoder with Bahdanau attention, covering dataset preparation, feature extraction with InceptionV3, model architecture, training with teacher forcing, and inference on the Flickr8K dataset.
AttentionEncoder-DecoderFlickr8k
0 likes · 20 min read

SMAD is a scalable graph‑based ad retrieval framework for e‑commerce search that builds a heterogeneous Query‑Item‑Ad graph, learns multi‑view embeddings with a parallel deep neural network and attention, employs category‑aware sampling for efficient distributed training, and delivers significant gains in offline relevance and online CTR, RPM, and PVR.
Attentionad retrievaldistributed training
0 likes · 17 min read


Code DAODec 7, 2021 · Artificial Intelligence
This article surveys deep‑learning image captioning, detailing the image encoder, sequence decoder, attention mechanisms and multimodal designs, comparing encoder‑decoder, detection‑backbone, transformer and dense captioning architectures, and explaining generation strategies and BLEU evaluation.
AttentionBLEUCNN
0 likes · 9 min read

This article surveys sequential recommendation methods, covering standard models such as pooling, RNN, CNN, attention, and Transformer, as well as long‑short term, multi‑interest, multi‑behavior approaches, and recent advances like contrastive learning, highlighting their impact on recommendation performance.
AttentionRNNTransformer
0 likes · 8 min read

The project presents a lightweight MobileNet‑FPN architecture enhanced with a modified spatial‑attention module that evaluates corner‑based self‑similarity to classify sneaker photo backgrounds, achieving 96% test accuracy—exceeding baseline CNN performance—and meeting business targets of over 80% hint accuracy and 90% mandatory enforcement.
AttentionCNNComputer Vision
0 likes · 12 min read

This article explains how a financial company leverages deep learning sequence models, including embedding, attention, and transformer techniques, to automatically extract features from massive time‑series data, improve risk model performance, and build a reusable, end‑to‑end system framework.
AIAttentionEmbedding
0 likes · 8 min read


58 TechOct 12, 2021 · Artificial Intelligence
This article presents a practical study of extracting phone numbers from two‑speaker voice dialogues using Seq2Seq models—including LSTM, GRU with attention and feature fusion, and Transformer—detailing data characteristics, model architectures, training strategies, experimental results, and comparative analysis showing the GRU‑Attention approach achieving the best performance.
AttentionGRULSTM
0 likes · 13 min read

This article explains how attention mechanisms are applied to text recognition, covering the basic CNN‑Seq2Seq‑Attention architecture, multi‑scale attention extensions, and a 2D attentional irregular scene text recognizer with detailed network components, training loss, and experimental results.
AttentionCNNComputer Vision
0 likes · 8 min read

This article presents a comprehensive overview of recent advancements in ranking algorithms for the Good Goods recommendation scenario, covering long‑sequence modeling, category‑retrieval attention, multi‑objective ranking, model structure optimizations, loss functions, and LTR techniques, along with experimental results and practical insights.
AttentionLTRRanking
0 likes · 13 min read

This article explains the BERT model by first reviewing the Encoder-Decoder framework, then detailing the attention mechanism—including self-attention and multi-head attention—before describing the full Transformer architecture and finally outlining BERT’s encoder-only design, training stages, and fine-tuning applications.
AttentionBERTEncoder-Decoder
0 likes · 15 min read

This article introduces a personalized re‑ranking model for recommendation systems, explaining the limitations of traditional point‑wise ranking, describing the PRM architecture with input, encoding, and output layers using multi‑head attention and pre‑trained personalization features, and presenting experimental results and future extensions.
AttentionCTRPersonalization
0 likes · 7 min read

User behavior sequence modeling, crucial for search and recommendation advertising ranking, has evolved from simple pooling to attention, RNN, capsule, and Transformer architectures, with industrial applications across e‑commerce, social, video, and music platforms, and future directions include time‑aware, multi‑dimensional, and self‑supervised approaches.
AttentionSequence ModelingTransformer
0 likes · 24 min read

This article analyzes the RealFormer modification to the Transformer architecture, details its implementation in BERT, and presents extensive experiments showing that while RealFormer can boost performance on low‑label‑count classification tasks, its benefits diminish or disappear as the number of classes grows.
AttentionBERTPretraining
0 likes · 12 min read

This article explains key concepts of neural machine translation, covering Seq2Seq encoder‑decoder models, beam search strategies, BLEU evaluation, various attention mechanisms, and the enhancements introduced in Google's Neural Machine Translation system to improve speed, OOV handling, and translation quality.
AttentionBLEUGNMT
0 likes · 11 min read

This article introduces a hierarchical graph neural network model that jointly captures multi‑scale temporal patterns and global spatial context for urban traffic flow prediction, demonstrates its superiority over existing methods on multiple public datasets, and validates each component through extensive ablation studies.
AttentionGraph Neural Networkdeep learning
0 likes · 8 min read


Didi TechMay 25, 2020 · Artificial Intelligence
This article provides a comprehensive technical overview of modern speech recognition, covering Didi’s driver‑assistant and smart‑customer‑service applications, fundamental ASR concepts, classic GMM‑HMM methods, deep‑learning breakthroughs such as DNN‑HMM, CTC, attention‑based and transformer models, practical training tricks, signal‑processing steps, and multimodal fusion techniques.
ASRAttentionCTC
0 likes · 16 min read

This article surveys the application of attention mechanisms in deep learning recommendation systems, reviewing models such as AFM, DIN, DIEN, DSIN, Behavior Sequence Transformer, Deep Spatio‑Temporal Networks, and ATRank, and discusses their architectures, attention types, advantages, and limitations.
AttentionCTR predictiondeep learning
0 likes · 10 min read

This article details the evolution of Feizhu's "Guess You Like" ranking system, moving from a linear FTRL model to several end‑to‑end deep learning versions—including PALM, FB‑PALM, and GLA—highlighting technical challenges, architectural changes, and measurable performance gains.
AIAttentionModel Iteration
0 likes · 15 min read

This article presents a comprehensive overview of modern speech recognition technology, covering basic ASR concepts, classic acoustic and language models, deep‑learning approaches such as DNN‑HMM, CTC, attention‑based and transformer models, multimodal fusion, signal‑processing pipelines, and practical deployment considerations at Didi.
ASRAttentionCTC
0 likes · 15 min read

This article provides an in-depth overview of sequential recommendation, defining the problem, discussing data preparation, and reviewing various neural architectures—including MLP, CNN, RNN, Temporal CNN, self‑attention, and reinforcement‑learning approaches—while offering practical guidance on model selection and implementation.
AttentionCNNRNN
0 likes · 36 min read

This talk presents a real-time attention-based look‑alike model (RALM) designed to address the long‑tail problem in recommendation systems by efficiently expanding seed users, leveraging user representation learning, attention mechanisms, and clustering to deliver timely, diverse content without retraining the model.
AttentionLong TailUser Representation
0 likes · 24 min read

This article systematically reviews the history, core concepts, variants, and practical implementations of attention mechanisms—from early additive and multiplicative forms to self‑attention, multi‑head attention, and recent transformer‑based models—highlighting why attention has become fundamental in modern AI research.
AttentionNLPSelf-Attention
0 likes · 16 min read

This article provides a comprehensive, beginner‑friendly overview of attention mechanisms—from their origins in early neural machine translation papers to modern self‑attention, multi‑head attention, and transformer variants—explaining core concepts, common variants, and why attention has become essential across NLP and vision tasks.
AttentionNLP
0 likes · 18 min read


HomeTechAug 7, 2019 · Artificial Intelligence
This article presents a comprehensive study of near‑duplicate video retrieval, covering the definition of near‑duplicate videos, motivations for deduplication, challenges, a two‑stage offline/online processing framework, keyframe and VGG16‑based feature extraction, metric‑learning loss functions, training procedures, dataset preparation, evaluation metrics, and model enhancements using LSTM and attention mechanisms.
AttentionLSTMVGG16
0 likes · 12 min read

This article presents a comprehensive solution to the absolute semantic recognition competition, detailing the problem background, dataset, evaluation metrics, feature engineering, model architecture—including Attention, Capsule, Bi‑GRU, and BERT—and analysis of results and lessons learned.
AttentionBERTCapsule Networks
0 likes · 11 min read

This article reviews the history of natural language processing, explains key deep‑learning models such as NNLM, Word2vec, CNN, RNN, attention mechanisms, and Transformers, and discusses their applications, future trends, and practical considerations in NLP tasks.
AttentionNLPTransformer
0 likes · 38 min read

This article reviews the current state of machine translation, explores the challenges of ambiguity, quality, and domain specificity, and presents human‑in‑the‑loop translation techniques—including attention‑enhanced models, transformer architectures, and online learning—while discussing practical productization and deployment considerations.
AI productizationAttentionTransformer
0 likes · 16 min read

This article introduces Advanced LSTM (A‑LSTM), which linearly combines multiple past hidden states to overcome traditional LSTM's one‑step dependency, and demonstrates its application in utterance‑level speech emotion recognition, achieving a 5.5% accuracy improvement through attention‑based weighted‑pooling RNNs and auxiliary speaker and gender tasks.
A-LSTMAttentionLSTM
0 likes · 8 min read


AntTechAug 16, 2018 · Artificial Intelligence
This article reviews deep‑learning‑based text classification techniques—including TextCNN, BiGRU, Capsule Networks, Attention mechanisms, and the novel cw2vec embedding—applied to Alipay complaint fraud data, presents experimental comparisons, and discusses their advantages, challenges, and future directions.
AlipayAttentioncapsule network
0 likes · 18 min read


Didi TechJun 1, 2018 · Artificial Intelligence
Didi’s attention‑based end‑to‑end Mandarin speech recognizer, built on the Listen‑Attend‑Spell architecture and modeling roughly 5,000 common characters, delivers 15‑25% relative accuracy gains over its prior LSTM‑CTC system while cutting model size, latency and server requirements and simplifying training by eliminating separate acoustic, pronunciation and language components.
AttentionEnd-to-EndLAS
0 likes · 14 min read

This article explains the Seq2Seq encoder‑decoder framework, its benefits for various sequence modeling tasks, and compares common decoding strategies such as greedy search and beam search, while also introducing attention and other enhancements for improved performance.
AttentionEncoder-Decoderbeam search
0 likes · 9 min read

The article explains how emotional design leverages usefulness, usability, and delight to capture attention, influence emotions, and drive user actions, ultimately forming lasting relationships between users and products through the dimensions of value and arousal.
AttentionEmotional DesignUser Experience
0 likes · 16 min read
