NLP Basics: Word Embeddings, Word2Vec, and Hand‑crafted RNN Implementation in PyTorch

This article introduces word‑level representations—from one‑hot encoding to dense word embeddings via Word2Vec—explains cosine similarity, then walks through the structure, limitations, and PyTorch implementation of a vanilla RNN, including a custom forward function and verification against the library API.

Cosine SimilarityNLPPyTorch

0 likes · 19 min read

NLP Basics: Word Embeddings, Word2Vec, and Hand‑crafted RNN Implementation in PyTorch

IT Services Circle

Mar 2, 2023 · Artificial Intelligence

Understanding GPT: Word Vectors, Transformers, and Model Architectures (GPT‑2, GPT‑3)

This article provides a concise technical overview of GPT, explaining how word vectors are constructed, how the Transformer architecture with self‑attention and feed‑forward layers processes these vectors, and how GPT‑2 and GPT‑3 extend the model with decoder‑only and large‑scale designs.

AIGPTSelf-Attention

0 likes · 8 min read

Understanding GPT: Word Vectors, Transformers, and Model Architectures (GPT‑2, GPT‑3)

政采云技术

Jul 5, 2022 · Artificial Intelligence

Overview of Natural Language Processing Techniques and Their Evolution

This article provides a comprehensive overview of natural language processing, covering its definition, historical development from one‑hot encoding to modern models such as word2vec, ELMo, GPT, and BERT, and discusses the advantages, limitations, and key concepts of each technique.

NLPWord Embeddingartificial intelligence

0 likes · 23 min read

Overview of Natural Language Processing Techniques and Their Evolution

Tencent Advertising Technology

Jul 30, 2020 · Artificial Intelligence

Tencent Advertising Algorithm Competition: Embedding Team's Transformer-Based Solution and Tips

The article details the Tencent Advertising Algorithm Competition semifinal, presenting the Embedding team's pure Transformer model with joint ID word embeddings, data preprocessing, training hyperparameters, model architecture, and practical tips for improving age/gender prediction from click sequences.

Competition SolutionTencent AdvertisingWord Embedding

0 likes · 7 min read

Tencent Advertising Algorithm Competition: Embedding Team's Transformer-Based Solution and Tips