From the 1943 perceptron concept to modern Transformer-based large language models, this article traces the evolution of neural network techniques in NLP, highlighting key milestones such as early perceptrons, the 1986 back‑propagation breakthrough, statistical methods, LSTM, word2vec, multitask learning, and the rise of GPT.
The article explains how embedding techniques encode semantic information into numeric vectors, covering Word2Vec and GloVe fundamentals, BERT anisotropy, SimCSE contrastive learning, alignment and uniformity metrics, ANN index structures such as HNSW, IVF and PQ, Matryoshka representation learning, practical deployment challenges, and evaluation best practices.
This article provides a comprehensive overview of natural language processing, tracing its evolution from early symbolic and statistical stages through deep learning breakthroughs, detailing sequence models, key NLP tasks, text representation methods, and the development of modern architectures like RNN, LSTM, GRU, Transformer, and GPT series.
This tutorial explains how to transform raw text into tensor representations using one‑hot encoding, Word2Vec, and FastText, provides step‑by‑step code examples, discusses their advantages and drawbacks, and shows how to visualize embeddings with TensorBoard.
This article introduces word‑level representations—from one‑hot encoding to dense word embeddings via Word2Vec—explains cosine similarity, then walks through the structure, limitations, and PyTorch implementation of a vanilla RNN, including a custom forward function and verification against the library API.
This article explains the Word2Vec algorithm, its CBOW and Skip‑Gram architectures, cosine similarity mathematics, training process with negative sampling, and provides a concise Python example using the gensim library.
This article walks through a Python solution for semantic similarity using Word2Vec, showing the problem context, required data preprocessing, the exact code implementation, and the resulting output, while also offering practical tips for asking technical questions in community groups.
This article traces the history of deep‑learning pre‑training techniques, comparing the parallel developments in natural‑language processing and computer vision—from early word2vec and bag‑of‑words models through ELMo and BERT to recent transformer‑based vision models like iGPT, ViT, BEiT and MAE—highlighting key innovations, challenges, and the convergence of the two fields.
This article traces the evolution of deep‑learning pre‑training techniques, starting with word2vec in NLP, moving through ELMo and BERT, then shifting to computer‑vision models such as iGPT, ViT, BEiT, and MAE, highlighting key innovations, challenges, and the convergence of NLP and CV paradigms.
This article explains practical techniques for improving NLP model accuracy on massive corpora, covering challenges of multi‑field text, word‑embedding choices, a fasttext‑based regression demo with book‑review data, feature engineering tricks, and a comparison with tf‑idf + LASSO.
This article details how 58.com improved its home‑page recommendation system by introducing vectorized recall with Word2Vec, optimizing negative sampling, deploying FAISS for fast nearest‑neighbor search, and later adopting a dual‑tower deep learning model with user interest features, achieving higher click‑through and conversion rates.
This article provides a comprehensive overview of natural language processing, tracing its origins from Turing's seminal test to modern techniques like regular expressions, word order importance, word embeddings, Word2vec, GloVe, and knowledge‑ and retrieval‑based chatbot methods.
The article details the Tencent Advertising Algorithm competition final, explains the chizhu team's approach of converting ad IDs into word sequences for text classification using large‑scale word2vec embeddings and a dual BiLSTM architecture, presents custom loss functions, training tricks, and shares full Python model code, achieving an overall rank of 11.
This article details the end‑to‑end process of extracting and leveraging tags for used‑car listings, covering data collection, segmentation, NLP‑based tokenization, word‑vector generation, tag‑library construction, and online retrieval flow to improve personalized recall and CTR.
This article provides a comprehensive overview of embedding techniques, explaining their role in deep learning recommendation systems, detailing Word2Vec and its Skip‑gram model with negative sampling and hierarchical softmax, and extending the discussion to Item2Vec and dual‑tower architectures for item representation.
This article reviews the evolution of word and sentence embeddings, covering foundational theories like vector semantics and distributional hypothesis, practical models such as Word2Vec, GloVe, fastText, Skip‑Thought, Quick‑Thought, and evaluation techniques, while offering implementation tips and real‑world use cases.
This article explains the fundamentals of the Word2Vec SGNS model, details its hyper‑parameters and training tricks, and demonstrates how customized embeddings are built for rental‑listing and news‑article recommendation, covering data preparation, objective‑function redesign, evaluation, and deployment in both recall and ranking stages.
This article demonstrates how to combine Jieba segmentation, Word2Vec embeddings, and NetworkX graph visualization to extract and analyze character relationships from the Chinese novel "Heavenly Sword and Dragon Slaying," detailing data preparation, model training, entity matrix construction, and network graph generation.
This article explains why deep learning has become essential for modern recommendation systems, describing its advantages such as automatic feature extraction, noise robustness, sequential modeling with RNNs, and improved user‑item representation, and reviews major deep‑learning‑based recommendation models and techniques.
At Qunar, the data center algorithm team developed an automatic ticket classification system that combines Support Vector Machine with word2vec embeddings to handle high‑dimensional, low‑sample text data, achieving 89% accuracy and 80% recall while outlining the full machine‑learning pipeline from feature extraction to deployment.
This article details how Industrial and Commercial Bank of China (ICBC) applies text mining and natural language processing to analyze both internal call‑center records and external online discussions, building ontologies and models that turn massive unstructured feedback into actionable insights for improving service quality and reducing costs.
