This article explains the architecture and key algorithms behind a specialized exam‑question search engine, covering query parsing, BM25‑based relevance scoring, text‑image hybrid retrieval, Learning‑to‑Rank models, and practical optimizations for long queries and large K‑12 datasets.
This article presents a comprehensive overview of Meituan's query understanding system, detailing how entity recognition, entity linking, query rewriting, and intent detection are designed and deployed to improve O2O search relevance across diverse local services.
This comprehensive guide explains Elasticsearch’s role in handling massive data, covering its architecture, core concepts like shards and replicas, indexing and search processes, performance considerations, common pitfalls, and practical interview insights for developers and engineers.
This comprehensive guide explains Elasticsearch’s fundamentals, including its distributed architecture, indexing process, shard and replica mechanisms, query execution, near‑real‑time search, segment management, and practical optimization techniques, providing developers and engineers with the knowledge needed to design, operate, and troubleshoot large‑scale search clusters.
This article provides a comprehensive overview of Elasticsearch, covering its distributed architecture, fundamental concepts such as nodes, shards, and indices, as well as practical guidance on index design, bulk writing, query‑fetch workflow, scroll queries, aliases, and basic optimization tips.
Baidu’s Wànxiàng system processes billions of images and videos daily by extracting low‑ and high‑level features, linking related media, and aggregating semantic attributes in a scalable, timely architecture that leverages thousands of CPU, GPU, and FPGA cores to power accurate, low‑latency rich‑media search and recommendation.
This article presents multiple real‑world Elasticsearch deployments across Chinese tech giants, detailing how they handle massive query loads, scale clusters, integrate with Kafka and Kibana, and apply the Elastic Stack for search, analytics, and security monitoring.
This article explains how the 58 search engine achieves millisecond‑level real‑time inverted‑index updates by redesigning the underlying data structures, combining static and dynamic indexing, using a chain‑array hybrid, segment merging strategies, and lock‑free read‑write concurrency while maintaining search performance.
This article explains the architecture and core concepts of Elasticsearch and Lucene, outlines the requirements for cross‑month and high‑speed queries on massive datasets, and provides detailed index and search performance tuning techniques—including bulk writes, shard routing, doc‑values management, and pagination strategies—to achieve sub‑second response times on billions of records.
Apache Lucene 8.8.1, a high‑performance Java full‑text search engine library, has been released, fixing bugs from 8.8.0 and including optimizations, while the previous 8.8.0 version introduced features such as LatLonPoint spatial queries, configurable Doc value compression, and FeatureField’s newLinearQuery scoring.
This article recounts how Shay Banon’s early cooking‑app project led to the creation of Compass, the evolution into Elasticsearch, and the pivotal role of Apache Lucene, distributed testing, and visionary design in building today’s powerful search platform.
This article explains the concept of inverted indexes, their role in full‑text search, and provides a detailed overview of how Apache Lucene implements inverted indexing, including term dictionaries, posting lists, query processing, and numeric handling with BKDTree.
This article recounts how Shay Banon's early cooking‑app project led to the creation of Compass, the evolution of Apache Lucene, and ultimately the development of Elasticsearch—a powerful, distributed search platform built with extensive testing infrastructure and inspired by futuristic data‑interaction concepts.
This article introduces Elasticsearch’s core concepts—including clusters, node roles, documents, mappings, and shards—and walks through the complete indexing workflow from client request to replica synchronization, highlighting key settings, routing calculations, and the role of refresh and flush operations.
The article details WeChat Search’s practical distributed architecture—using a Chubby‑elected leader for shard‑to‑node mapping, hash‑based sharding with dynamic rebalancing, a Lambda‑style batch and near‑real‑time indexing pipeline, relaxed monotonic consistency, and group‑based searcher scaling—to illustrate trade‑offs and lessons for building scalable, reliable search services.
This article explains how Elasticsearch processes text before indexing by describing the analysis pipeline, built‑in and custom analyzers, tokenizers, token filters, n‑gram techniques, the analysis API, and the IK Chinese tokenizer plugin, providing practical curl examples throughout.
The article surveys core search‑engine technology—document hierarchy, flat and vertical inverted indexes, query operators for building and merging score lists, and ranking models from Boolean and BM25 to language‑model approaches like Indri—providing a foundational overview of information retrieval.
This article explains Elasticsearch's logical and physical design, how documents are structured and indexed, the role of shards and replicas, and provides practical examples of indexing, searching, aggregations, and retrieving documents using RESTful APIs.
This article provides a detailed collection of basic Elasticsearch interview questions and answers, covering core concepts such as cluster and node architecture, indexing, mapping, search APIs, analyzers, tokenizers, aggregation, security, and operational tools, along with practical code snippets for common tasks.
This article introduces Elasticsearch as a Lucene‑based distributed search engine and demonstrates how to use curl commands to create, read, update, and delete documents and indices, providing step‑by‑step examples with command‑line output and screenshots.
The article offers a thorough, step‑by‑step guide to Elasticsearch, explaining how to check cluster health, create and manage indices, define mappings and field types, use dynamic mapping, and perform maintenance tasks such as shrink, split, rollover, and cache management, all illustrated with concrete API examples.
This article explains Elasticsearch's foundation on Lucene, compares its concepts to MySQL, describes inverted indexing, and provides a step‑by‑step guide for installing, configuring, and using the basemkhirat/elasticsearch Laravel plugin with code examples and tips for Chinese analysis.
The Hulu AI Class event showcased a series of technical talks covering large‑scale graph neural network optimizations, multi‑factor video ad placement algorithms, recommendation and search engine techniques, machine‑learning‑driven video codec improvements, and advanced content‑embedding methods, highlighting practical engineering experiences from Hulu’s Beijing office.
This article provides a comprehensive overview of general search engines, covering their classification, core workflow, key modules such as web crawlers, content processing, storage, user query handling, ranking strategies like TF‑IDF and PageRank, as well as anti‑cheat measures and user intent understanding.
The article explains cosine similarity from basic geometry and vector math, derives its formula, and shows how it powers precision marketing, image classification, and text retrieval, while also detailing its industrial implementation in Lucene’s vector space model.
A Beijing Internet Court case ruled that Baidu must pay nominal damages after failing to remove a user’s ID photo, uploaded to a defunct alumni site, which the search engine indexed and displayed at the top of results, highlighting legal responsibilities for personal data handling.
This article provides a comprehensive introduction to Elasticsearch, covering data types, the role of Lucene, cluster architecture, node roles, discovery mechanisms, shard and replica management, mapping, installation, health monitoring, indexing workflow, storage internals, refresh and translog processes, segment merging, and practical performance and JVM tuning tips.
This article reviews how major Chinese companies such as JD.com, Ctrip, Quark, 58.com, and Didi have adopted Elasticsearch for large‑scale search, log analysis, and real‑time analytics, detailing their cluster architectures, scaling strategies, and operational practices.
This article provides a comprehensive, top‑down and bottom‑up explanation of Elasticsearch’s internal architecture, covering clusters, nodes, shards, Lucene segments, inverted indexes, stored fields, document values, caching, merging, routing, scaling, and query processing, while addressing common performance questions.
This article explains the internal workings of Elasticsearch, covering its cloud‑based cluster architecture, Lucene‑based indexing structures such as segments, shards, inverted indexes, stored fields and doc values, as well as search processing, caching, merging, routing, and scaling strategies.
This article provides a comprehensive guide to improving Elasticsearch performance and stability by covering configuration file tweaks, system‑level settings, and usage‑level optimizations such as hot‑thread analysis, pending tasks, field storage, translog handling, refresh intervals, shard management, and best practices for routing and alias usage.
This guide walks through essential Elasticsearch operations, covering cluster health checks, creating and querying indices, managing settings, templates, aliases, index shrink, split, rollover, and detailed mapping definitions, providing practical code examples for each step.
This article consolidates practical Elasticsearch tuning techniques—including configuration file adjustments, system‑level optimizations, and usage‑level settings—to improve cluster performance, stability, and resource efficiency for production environments.
Elasticsearch 7.9.0 introduces critical security patches for field‑level leakage, updates script cache limits, refines field capabilities, improves snapshot restore throttling, expands thread‑pool write queue, deprecates dangling indices, and addresses known issues like mapping errors in machine‑learning indices.
This article explains Elasticsearch’s write, read, and search workflows, detailing the roles of coordinating nodes, primary and replica shards, refresh and commit cycles, translog handling, and the underlying Lucene inverted index mechanism.
This article introduces Elasticsearch as a distributed open‑source search and analytics engine, explains its core concepts and architecture, compares it with relational databases, details installation steps, configuration, indexing, analyzers, query DSL, pagination, sorting, and provides practical examples for building search functionality.
ElasticSearch achieves distributed search by organizing data into indices, types, mappings, documents, and fields, splitting indices into primary and replica shards across multiple nodes, with automatic master election and shard allocation, enabling horizontal scaling, high availability, and improved performance for large‑scale data workloads.
This article explains Elasticsearch's internal mechanisms for indexing, querying, and retrieving data, covering the roles of coordinating nodes, primary and replica shards, the refresh and commit cycles, near‑real‑time search, and the underlying Lucene inverted index.
Elasticsearch 7.8.1 introduces a range of new capabilities such as literal SUM/MIN/MAX/AVG in SQL, enhanced authorization for apm_user, updated index creation logging, composable template renaming, additional machine‑learning aggregations, snapshot/restore optimizations, and an improved update API, all aimed at boosting search and analytics performance.
Meituan’s AI‑driven search combines the Augur online inference framework—offering stateless, distributed feature operators, transformers, and a DSL for rapid, high‑throughput model scoring—with the Poker platform for model training, versioning, and experimentation, together accelerating iteration, improving performance, and enabling advanced model‑as‑feature ensembles.
This article details 58.com’s extensive use of Elasticsearch for search, analytics, and log processing, covering cluster optimization challenges, typical issues like disk exhaustion and write slowdown, practical solutions, development standards, ELKB architecture, real‑time log and MySQL slow‑log applications, platform‑as‑a‑service construction, and future roadmap plans.
This article explains the core components of a search engine—document processing, query processing, and matching—detailing each step from indexing to ranking, and discusses the document features that influence relevance, providing a comprehensive overview of information retrieval fundamentals.
This article explains Elasticsearch’s cluster and data layer architecture, detailing nodes, indices, shards, replicas, deployment models, and the trade‑offs of different distributed system designs, helping readers understand how Elasticsearch achieves scalability, reliability, and query performance.
This article provides a comprehensive tutorial on installing a single‑node Elasticsearch 7.x cluster, configuring its key settings, setting up the Elasticsearch‑head web UI, and includes a reference 6.x configuration file for production environments.
This article reviews how leading Chinese internet companies such as JD.com, Ctrip, Qunar, 58.com, and Didi have adopted Elasticsearch for large‑scale order search, log analysis, real‑time monitoring, and security, describing the evolution of cluster architectures, shard strategies, multi‑cluster pipelines, and performance optimizations.
This article explains how Elasticsearch uses Analyzer components—character filters, tokenizers, and token filters—to perform text analysis, reviews the built‑in analyzers such as standard, simple, stop, whitespace, keyword, pattern, language, ICU and IK, and provides practical _analyze API examples with code snippets and result screenshots.
This article examines Elasticsearch query performance by explaining Lucene's underlying data structures, describing how composite queries are merged, and presenting benchmark numbers for various query types such as term, range, and combined queries, highlighting optimization techniques and practical conclusions.
This article explains the differences between Master‑Slave and P2P ring architectures, introduces Elasticsearch’s core concepts, internal components, master election, shard routing, indexing and search processes, and discusses how the system avoids split‑brain scenarios and ensures high availability.
The article surveys how leading Chinese tech firms such as JD Daojia, Ctrip, Qunar, 58.com, and Didi have adopted Elasticsearch for large‑scale search, real‑time analytics, and security, detailing their evolving cluster architectures, shard strategies, data volumes, and supporting services.
Query understanding (QU) involves lexical, syntactic, and semantic analysis of user queries to enable effective search recall and ranking, covering modules such as preprocessing, correction, expansion, segmentation, intent detection, term importance, and guidance, with detailed discussion of algorithms, models, and system architecture.
This article offers a detailed, experience‑driven comparison of Elasticsearch against its main rivals—Lucene, Solr, relational databases, OpenTSDB, HBase, MongoDB, ClickHouse, and Druid—highlighting where Elasticsearch excels, where it falls short, and practical guidance for choosing the right data solution.
This article provides a detailed comparison of Solr 7.3 and Elasticsearch 7.4, covering their shared Lucene foundation, distributed architecture, feature sets, configuration approaches, storage options, shard management, and ecosystem differences to help readers choose the right search platform for their needs.
This comprehensive guide introduces Elasticsearch fundamentals, its features and use cases, then walks through integrating it with SpringBoot, configuring Maven dependencies, performing index and document operations, and demonstrates a variety of query types and aggregations using both RESTful APIs and Java code examples.
This article introduces Elasticsearch by explaining its core concepts such as indices, types, documents, mappings, and Query DSL, demonstrates common use cases, and provides step‑by‑step instructions for creating, updating, viewing, and deleting indices and documents using RESTful APIs, curl commands, and Docker‑compose deployment.
This article outlines a step‑by‑step procedure for safely taking an Elasticsearch node offline—checking master‑eligible settings, adjusting minimum_master_nodes, excluding the node from routing, waiting for shard relocation, stopping the service, and restoring the cluster routing—ensuring no data loss or service interruption.
This article explains what an inverted index is, why it’s essential for full‑text search, how it is built and queried, and common token transformations such as stop‑word removal, lemmatization, and stemming.
This article compares Elasticsearch and Solr, examining their history, community, licensing, core technologies, APIs, scalability, vendor support, ecosystem, performance, management tools, and visualization options to help organizations decide which open‑source search engine best fits their big‑data and search requirements.
This article introduces Elasticsearch as a distributed search and analytics engine, outlines its common application scenarios, provides step‑by‑step installation commands, explains core concepts such as documents and indices, and demonstrates basic indexing, retrieval, bulk processing, and aggregation operations.
Elasticsearch is a distributed, open‑source search and analytics engine built on Apache Lucene, offering a simple REST API, scalability, and a rich ecosystem—including Kibana, Beats, and Logstash—that enables storage, querying, and aggregation of text, numeric, and geospatial data for diverse use cases.
This article explains the architecture of Elasticsearch and Lucene, outlines common performance bottlenecks, and provides concrete indexing and search optimization techniques—including bulk writes, shard routing, doc values tuning, and pagination strategies—to achieve sub‑second query responses on billions of records.
This article explains how search engines collect, analyze, and rank web pages by describing the spider system, HTML and layout tree construction, feature extraction, and machine‑learning based classification methods used to understand page content and improve result relevance.
TurboSearch is Tencent AI Lab's next-generation large-scale search system, delivering distributed massive indexing, high-performance parallel retrieval, multi-granularity and multi-modal vector indexing, private Docker deployment, integrated NLP query analysis, extensible plugins, and robust operations for massive data and diverse search scenarios.
This article explains how query understanding (QP) transforms user search queries into structured semantic blocks and intent categories using rule‑based NLP, entity recognition, and post‑processing, and describes its taxonomy, implementation details, and practical impact on search engine results.
This article traces the evolution of web crawlers—from early Perl scripts to modern ByteDance agents—explaining their role in search engines, business models, anti‑crawling measures, and the impact on content creation and competition.
This article explains how 360 Search handles billions of daily crawls and hundred‑billion‑scale indexing by describing its overall architecture, core modules such as offline indexing and online retrieval, query analysis, relevance scoring, and the engineering techniques that enable efficient large‑scale web search.
This article describes how an e‑commerce platform built a big‑data pipeline—including TTLog, Lindorm, BCP, MetaQ, Blink and Xflush—to detect and verify search advertising placement errors in real time, covering background, implementation details, core challenges, and future optimization directions.
The article details JD.com's internally developed distributed search engine and recommendation platform, their new architectures, deep‑learning‑driven ranking and recall models, component‑based deployment, extensive performance testing, and coordinated operations that powered the massive 11.11 shopping event.
This article explains the fundamental architecture of Elasticsearch and its underlying Lucene engine, detailing the data model, index hierarchy, and the step‑by‑step write, refresh, flush, and merge processes that enable near‑real‑time search and data durability.
To avoid building separate search services for each Xianyu business, the team created a unified, generic search architecture based on Alibaba’s HA3 engine and a control layer that automates data dumping, indexing, query translation, and result ranking across five subsystems, enabling new services to be onboarded in minutes instead of weeks.
ByteDance has entered the Chinese search market with its new mobile‑first “Toutiao Search”, assembling a team from top tech firms and leveraging AI, NLP and computer‑vision technologies, sparking a fresh rivalry with Baidu’s long‑standing dominance.
This article provides a comprehensive overview of Elasticsearch, covering its role in handling structured and unstructured data, core concepts such as Lucene, inverted indexes, clusters, shards, replicas, mapping, indexing processes, storage mechanisms, and practical performance tuning tips for deployment.
This article explains how 360 Search processes billions of web pages daily, detailing its backend architecture, offline indexing, online retrieval, index organization, and relevance models that enable efficient search over a hundred‑billion‑scale web corpus.
This article explains the architecture of Elasticsearch clusters, covering core concepts such as nodes, indices, shards, replicas, indexing flow, mixed and tiered deployment models, data‑layer storage, and compares these designs with other distributed data system architectures.
This article narrates the origin and evolution of Elasticsearch, from its Lucene roots through Compass to the modern ELK Stack, illustrating how it simplifies full‑text search, log analysis, and real‑time monitoring for developers and operations teams.
This article provides a comprehensive tutorial for installing Elasticsearch 7.x in single‑node mode, configuring its key settings, deploying the Elasticsearch‑head web UI via Tomcat, and includes reference configuration files for Elasticsearch 6.x, complete with command‑line examples and code snippets.
This article describes the design and implementation of a spelling‑correction module for 58.com’s search engine, covering common query errors, rule‑based and statistical language‑model approaches, offline dictionary generation, n‑gram and Viterbi decoding, online workflow, and practical examples.
This article compiles common Elasticsearch interview questions and detailed answers covering cluster architecture, inverted index fundamentals, index design, write/query optimizations, master election, document indexing flow, search process, Linux tuning, and Lucene internals, providing practical guidance for candidates.
Open Distro for Elasticsearch is an AWS‑backed, fully open‑source distribution of Elasticsearch that adds enterprise‑grade security, alerting, SQL querying, and performance analysis features, includes Kibana for visualization, and follows a clear version‑release roadmap.
This tutorial walks you through installing Elasticsearch, creating indices, mapping schemas, inserting, updating, deleting, and searching documents using both cURL commands and the official Python client, illustrated with practical code examples and screenshots.
This article examines the unique characteristics of log data, outlines the challenges of using Elasticsearch at scale, and presents practical optimization techniques—including ingestion, mapping, time‑range search, metadata loading, and a custom C++ engine—to dramatically improve performance, stability, and cost efficiency.
This article walks through the fundamentals of search engines, explaining inverted indexes, the explosion of index size, core Elasticsearch concepts, its distributed architecture, and how it powers the ELK stack for log analysis, all illustrated with clear diagrams and examples.
The article explains how Xianyu guarantees high‑availability of its core Ha3‑based search engine through independent gateway deployment, multi‑datacenter disaster recovery, traffic isolation, comprehensive monitoring, pressure testing, gray releases, and automated/manual failover, enabling rapid issue detection, recovery, and continuous service stability.
This article reviews major in‑memory inverted index compression techniques such as PForDelta, PEF, and MILC, explains their principles and trade‑offs, and details practical optimizations applied at 58.com to achieve query performance comparable to uncompressed indexes while reducing memory usage by about 35 percent.
This article compares the two leading open‑source search engines, Solr and Elasticsearch, examining their architectures, features, deployment ease, scalability, community support, and ideal use cases to help you decide which solution best matches your application requirements.
This article details how JD.com’s order center migrated its Elasticsearch cluster through multiple stages—from an initial unoptimized deployment to a real‑time dual‑cluster backup solution—addressing scalability, reliability, shard tuning, version upgrades, and data synchronization strategies to support billions of documents and hundreds of millions of daily queries.
This article presents a comprehensive overview of constructing a search engine using Elasticsearch, covering architecture components, data read/write mechanisms, shard management, caching strategies, and real‑world case studies that illustrate performance tuning, isolation, and deployment best practices.
The article chronicles Elastic’s journey from a small open‑source search tool to a NYSE‑listed company, explaining Elasticsearch’s technical foundations, its real‑world applications, and what the IPO means for developers and the broader search‑technology ecosystem.
Elastic's NYSE debut, its rapid stock surge, the origins and technical strengths of Elasticsearch, and what the company's public listing means for developers and tech entrepreneurs are explored in detail, highlighting the journey from a personal recipe‑search tool to a global data‑search platform.
This article examines the high‑real‑time requirements of incremental data ingestion for search engines, compares three update schemes, and details how adopting a Kafka subscription‑based message‑queue approach dramatically improves latency and flexibility for the Nuomi search framework.
This article explains Elasticsearch mapping, covering dynamic mapping, field datatypes, meta‑fields, mapping parameters, and dynamic templates, while emphasizing the importance of careful schema design and the need for reindexing when changes are required.
This reflective essay recounts the author’s two‑and‑a‑half‑year stint at Infoseek, the strategic missteps around free‑mail branding, the transition to GO.COM, and the evolution of search‑engine ranking principles—from early link‑analysis patents to the so‑called confidence‑based bidding model.
This tutorial walks you through installing Elasticsearch, creating indices, inserting and updating documents, performing searches via REST API, and integrating Elasticsearch into Python applications using both raw HTTP requests and the official Python client, illustrated with practical examples and screenshots.
This article explains how 58.com’s in‑house C++ search kernel ESearch was architected to provide second‑level real‑time indexing, high‑concurrency low‑latency querying, flexible ranking models, and efficient storage structures for billions of daily queries across massive classified data.
This article explains the core of search engine technology by describing what an inverted index is, how it is built using single‑pass memory and multi‑way merge methods, how indexes can be partitioned and incrementally updated, and how Hadoop can be used for large‑scale indexing.
This article introduces Apache Solr and Elasticsearch, explains their shared Lucene foundation, compares their scalability, deployment ease, and key features, and highlights why Elasticsearch is often considered more efficient for large‑scale, multi‑tenant search applications.
This article explains Elasticsearch’s core mechanisms, covering the cluster startup sequence, node discovery and election, handling of failed nodes, cluster management APIs, and the detailed read/write processes including coordinating nodes, shard allocation, memory buffering, transaction logs, and query‑then‑fetch execution.
This article explains how 360’s Hulk private cloud platform deploys Elasticsearch with a dedicated master architecture, load‑balancing, per‑business isolated clusters, SearchGuard security, dynamic tokenization, self‑service user features, and advanced monitoring to achieve high‑performance, scalable search services.
This article explains the core challenges of search engines, the origins and mechanics of the PageRank algorithm, its handling of dead ends and spider traps, extensions like Topic‑Sensitive PageRank, and the various link‑spam attacks and countermeasures such as Spam Farms and TrustRank.
This guide explains how to set up a Go environment, install the open‑source Riot search engine, understand its goroutine‑based architecture for indexing and ranking, and provides a concise code example demonstrating document indexing and query execution for content‑driven websites.
The talk demonstrates how Meipai, a leading Chinese short‑video community, leverages large‑scale text mining and machine‑learning techniques—ranging from anti‑spam filtering to AI‑enhanced search—to enrich captions, comments, and metadata, improve user experience, and inspire further research on text data in video platforms.
