Introduction to Search Engine Algorithm Systems: Ranking and Intent Recognition

The presentation outlines the development of search engine algorithms, starting with early simple models like Bayes, LR, SVM, and decision trees, progressing through ensemble methods (GBDT, RF) and deep learning models (CNN, RNN, Wide & Deep) that are now widely used in industry.

It describes a three‑stage framework: an initial retrieval stage delivering a few results, an enriched platform stage providing detailed data and personalized results, and a knowledge‑graph/precise‑QA stage that leverages entity recognition and relationship mining.

The system architecture is divided into offline processing (crawling, classification, clustering, tagging, entity and relation extraction) and online serving modules that perform query analysis, intent recognition, and ranking, with Sogou’s framework used as an example.

Query analysis evolves from rule‑based keyword extraction to classifiers (Bayes, LR, SVM) and finally neural networks, while article ranking relies on Learning‑to‑Rank (LTR) techniques such as LambdaMart, listwise/pairwise approaches, and feature‑rich models like FM and CDSSM that combine linear, tree‑based, and deep features.

Various click‑through models are discussed, including simple click models, DBN (allowing multiple clicks), and UBM (supporting jumps), followed by newer ideas like Ubias LTR, reinforcement‑learning‑based ranking, and adversarial IRGAN for recommendation.

Additional models such as FastText, embedding‑based softmax, and hybrid CNN/RNN architectures for query representation are introduced, highlighting their speed and scalability advantages.

The article concludes with references to recent papers and encourages further exploration of the presented algorithms.