Archie, created in 1989 by Peter Deutsch at McGill University, was the first internet search tool that indexed FTP sites, while WAIS, developed by Brewster Kahle at Thinking Machines, extended searchable databases worldwide, both highlighting early challenges in managing large communication traffic and user-friendly interfaces.
This article explains the fundamentals of full‑text search, describing how Lucene builds and uses inverted indexes, the steps of tokenization, linguistic processing, term weighting, and relevance scoring, and illustrates these concepts with examples, tables, and diagrams.
This article explains the core of search engine technology by describing what an inverted index is, how it is built using single‑pass memory and multi‑way merge methods, how indexes can be partitioned and incrementally updated, and how Hadoop can be used for large‑scale indexing.
This article explains the principles, algorithms, and practical challenges of search autocomplete (query suggestion), covering popularity‑based models, time‑sensitive methods, user‑aware and context‑aware approaches, data pipelines, indexing, ranking, personalization, and evaluation techniques used in e‑commerce search systems.
This article explains how modern search engines identify precise and broad user intents, examines real‑world query examples, and details extension modules such as synonym, pinyin, and correction that enhance query understanding using algorithms like Aho‑Corasick, Hidden Markov Models, and Levenshtein distance.
This article summarizes two SIGIR 2017 papers: one introduces a probabilistic latent‑class model for shopping‑pair push recommendations that improves e‑commerce click‑through rates by leveraging co‑purchase and view‑then‑purchase graphs, and the other presents IRGAN, a GAN‑based framework that unifies generative and discriminative information‑retrieval models, achieving state‑of‑the‑art results across web search, recommendation, and QA tasks.
The second “Frontiers and Future of Information Retrieval” forum, co‑hosted by the Chinese Computer Society, Alibaba and academic committees, showcased how massive, structured e‑commerce data and AI algorithms are revolutionizing search, customer service, and research collaborations across the industry.
This article explains the fundamentals of search engine architecture, covering text collection, indexing, ranking and evaluation, and then traces Twitter's internal search evolution from MySQL full‑text search to the Earlybird index server, Blender aggregation, and smart memory‑SSD strategies.
The article introduces a four‑dimensional evaluation framework for retrieval systems—Fast, Accurate, Complete, and New—explaining how each metric is measured, why it matters to users, and how crowdsourced testing across devices and networks can provide objective quality assessments.
This article explains how Google’s PageRank algorithm uses the web’s link structure, Markov processes, and stochastic matrix adjustments—including damping factor α—to overcome ranking challenges and provide a mathematically sound method for ordering search results.
This article provides a comprehensive overview of search engine fundamentals—including inverted indexing, tokenization, ranking, high‑concurrency infrastructure, caching, crawling strategies, query understanding, keyword rewriting, personalization, and knowledge‑base construction—highlighting the technical challenges that make modern search engines like Google superior to simpler implementations.
This article provides a comprehensive overview of search engine evolution, core technologies such as crawling, indexing, retrieval and link analysis, platform foundations including cloud storage and computing, and techniques for improving search results through anti‑spam, user‑intent analysis, deduplication and caching.
