This article provides a comprehensive overview of Elasticsearch, covering data types, Lucene fundamentals, cluster architecture, shard allocation, indexing mechanisms, storage strategies, refresh and translog processes, segment merging, performance tuning, and JVM optimization for building scalable, near‑real‑time search solutions.
This article explains the end‑to‑end pipeline of query understanding for e‑commerce search, covering preprocessing, segmentation, spell correction, normalization, and expansion, and discusses both academic research and industry implementations with examples and references.
This article provides a comprehensive introduction to Elasticsearch, covering data types, Lucene fundamentals, cluster architecture, node roles, shard and replica mechanisms, mapping, basic usage, installation steps, health monitoring, indexing workflow, storage strategies, refresh and translog handling, segment merging, and practical performance tuning tips.
This guide explains how to deploy an Elasticsearch cluster with hot, warm, and cold nodes, configure Index Lifecycle Management policies, adjust polling intervals, create indices with rollover aliases, add documents, and observe automatic phase transitions and deletions, all using Docker Compose.
The article details Xianyu’s end‑to‑end product‑search architecture—covering tokenization, indexing, online request flow, offline index building, multi‑datacenter active‑active deployment, and supporting ad and debugging systems—and explains how expanding searcher capacity, separating query engines, grading updates, and diffusing auxiliary‑table writes together reduced latency from hours to near‑zero, enabling real‑time search.
This article explains the fundamentals of search, why traditional databases struggle with large‑scale text queries, introduces full‑text search and inverted indexes, describes Lucene as the core library, and details ElasticSearch's distributed architecture, features, and common use cases.
This article presents a comprehensive guide to optimizing Elasticsearch for massive datasets, covering Lucene fundamentals, index and shard architecture, practical performance‑tuning techniques, and real‑world case studies that achieve sub‑second query responses on billions of records.
This article explains the role of Youzan’s Query Parser (QP) in search, walks through its overall and layered architecture, details each algorithmic plugin—from preprocessing to synonym handling—and shows concrete code examples and results that improve search relevance across multiple retail scenarios.
This article explains what search is, why traditional databases struggle with full‑text queries, introduces the concepts of inverted indexes and Lucene, and shows how ElasticSearch combines distributed architecture, real‑time analytics, and powerful search features to solve these problems.
This article explains how Elasticsearch leverages inverted indexes, term dictionaries, and compression techniques such as Frame‑of‑Reference and Roaring Bitmaps to enable rapid full‑text search, detailing the underlying data structures, query processing, and practical indexing tips for efficient backend search implementations.
This article explains how Elasticsearch uses inverted indexes, term dictionaries, and advanced compression techniques such as Frame‑of‑Reference and Roaring Bitmaps to achieve rapid search performance while minimizing memory and disk usage, and it also covers practical indexing tips for production use.
This tutorial walks through installing Elasticsearch 8.2, configuring the PHP client via Composer, creating a MySQL articles table, indexing its rows into Elasticsearch, and performing basic CRUD operations such as retrieving, deleting, and searching documents using PHP code.
This comprehensive guide explains Elasticsearch’s role in searching structured and unstructured data, covers Lucene’s inverted index, details cluster components, shard and replica mechanics, mapping types, installation steps, indexing workflow, storage strategies, and practical performance optimizations for real‑world deployments.
This article introduces the lightweight pure‑Python search library Whoosh, outlines its key features, demonstrates how to define a schema, build an index from a CSV of Chinese poems, and perform full‑text queries with example code, making it ideal for small search projects.
This article provides a comprehensive introduction to Elasticsearch, covering data types, Lucene fundamentals, cluster architecture, node roles, shard and replica mechanisms, mapping, installation, health monitoring, indexing principles, storage strategies, refresh and translog handling, segment merging, performance tuning, and JVM optimization for large‑scale search applications.
This article provides a detailed overview of Elasticsearch, covering its data model, Lucene foundation, cluster architecture, shard and replica mechanisms, index mapping, installation steps, health monitoring, write and storage processes, segment management, and performance tuning techniques for large‑scale search applications.
Elasticsearch 8.3.0, a distributed full‑text search engine built on Lucene, introduces several known issues, deprecations, performance enhancements across aggregations, authentication, authorization, autoscaling, and adds new geo, health, mapping, SQL, search, and TSDB capabilities.
This article introduces the lightweight Python search library Whoosh, explains its features, demonstrates how to define schemas, create indexes from a CSV dataset, and perform full‑text queries with example code, including searching.
This article introduces the intelligent question‑answering technology used in WeChat Search, covering background, knowledge‑graph‑based QA, document‑based QA, technical pipelines, key modules such as entity linking and relation recognition, and future research directions.
This article explains how modern video platforms combine traditional keyword retrieval with deep‑learning‑based vector retrieval, detailing model architectures, attention mechanisms, personalization features, offline experiments, and online A/B results that show significant improvements in recall, relevance, and user experience.
This article provides a comprehensive tutorial on why to use Elasticsearch, how to pull the Docker image, set up data directories, configure cluster and node settings, adjust system limits, launch three Elasticsearch containers, and verify the cluster using REST APIs and the elasticsearch‑head UI.
This article provides a comprehensive introduction to Elasticsearch, covering data types, Lucene fundamentals, cluster architecture, node roles, shard and replica mechanisms, mapping, basic usage, health monitoring, indexing workflow, storage strategies, and practical performance tuning techniques.
In a shared Elasticsearch cluster, the team used slow‑log analysis to pinpoint costly queries caused by unnecessary fuzzy matches and integer‑mapped low‑cardinality fields, then optimized them by converting matches to filters and remapping those fields to keyword, re‑indexing, which cut latency from over 100 ms to under 10 ms and eliminated slow‑query alerts.
This article provides a comprehensive overview of Elasticsearch, explaining its underlying Lucene technology, data modeling, cluster architecture, shard and replica mechanisms, indexing workflow, storage strategies, refresh and translog processes, as well as practical performance and JVM tuning tips for building scalable, near‑real‑time search solutions.
This article details Ctrip's experience migrating its high‑traffic hotel search service to AWS, covering microservice architecture challenges, API‑level deployment, persistence strategies with Redis, MySQL, XPipe and DRC, file sharing via S3, and lessons learned for future cloud‑native designs.
Elasticsearch 8.0 introduces major updates including 7.x REST API compatibility headers, default-enabled security with registration tokens, system index protection, a preview KNN search API using dense_vector, storage‑saving field mappings, faster geo indexing, PyTorch model support, and numerous deprecations and configuration changes across aggregations, allocation, analysis, authentication, and core infrastructure.
This article provides a comprehensive overview of Elasticsearch, covering data types, Lucene fundamentals, cluster discovery, node roles, shard and replica management, mapping, installation, health monitoring, indexing mechanics, storage strategies, refresh and translog processes, segment merging, and practical performance optimizations for production deployments.
This comprehensive article explains what Elasticsearch is, how it builds on Lucene to provide distributed real‑time search and analytics, covering data types, cluster components, shard routing, indexing pipelines, storage formats, segment merging, and practical performance‑tuning tips for production deployments.
This article explains how to build a site‑wide search engine using Java, covering crawling concepts, forward and inverted indexing, module design, tokenization methods, and detailed code examples for file enumeration, HTML parsing, and index generation.
This comprehensive guide explains Elasticsearch fundamentals, including data types, Lucene and inverted indexes, cluster and node roles, shard and replica mechanisms, mapping, installation steps, health monitoring, write and storage processes, segment merging, and practical performance tuning tips for large‑scale search deployments.
Learn how to install MeiliSearch via Docker, add documents using the PHP SDK, and perform both basic and custom searches, including typo tolerance and highlighted results, with step‑by‑step commands and code examples.
This guide introduces RedisMod’s enhanced modules, shows how to install Redis with Docker, demonstrates native JSON storage using RedisJSON, walks through building and querying a full‑text search index with RediSearch—including Chinese support—and compares its performance against Elasticsearch.
This comprehensive guide explains Elasticsearch fundamentals—including documents, indices, nodes, clusters, REST and Document APIs, query DSL, relevance scoring, distributed architecture, real‑time indexing, search execution, pagination, scroll, aggregations, data modeling, mapping options, parent/child relationships, reindexing, and practical cluster and write/read performance optimizations.
Elasticsearch 8.0 introduces major updates including 7.x REST API compatibility headers, default-enabled security with registration tokens, known issues on ARM/macOS, a preview KNN search API using dense_vector, storage reductions for keyword and text fields, faster geo indexing, PyTorch model support, and numerous other enhancements across aggregations, allocation, analysis, authentication, and core infrastructure.
Elasticsearch 8.0 introduces major updates including 7.x REST API compatibility headers, default‑enabled security with registration tokens, protected system indices, a preview KNN search API, storage‑efficient keyword/match_only_text/text fields, faster indexing for geo_point and geo_shape, PyTorch model support, and numerous deprecations and enhancements across aggregations, allocation, analysis, authentication, cluster coordination, and engine components.
Elasticsearch 8.0 introduces major updates including 7.x REST API compatibility headers, default-enabled security with enrollment tokens, protected system indices, a preview KNN search API, storage‑saving field encodings, faster geo indexing, and numerous deprecations and enhancements across aggregations, authentication, cluster coordination, and packaging.
Elasticsearch 8.0 introduces 7.x REST API compatibility headers, default‑on security features with automatic enrollment tokens, tighter protection of system indices, a preview KNN search API, storage‑saving field encodings, faster geo‑point and geo‑shape indexing, PyTorch model support, and a long list of deprecations and internal improvements.
This comprehensive guide explains Elasticsearch’s core concepts—including its distributed architecture, indexing process, shard routing, refresh and translog mechanisms, segment merging, and performance tuning—while providing practical examples and configuration tips for building scalable, near‑real‑time search solutions.
The article provides a comprehensive overview of Elasticsearch 8.0, highlighting major updates such as 7.x REST API compatibility headers, default-enabled security, system‑index protection, a new KNN search API, storage and indexing optimizations, PyTorch model support, and numerous deprecations and feature removals across the stack.
Elasticsearch 8.0 introduces major changes such as 7.x REST API compatibility headers, default‑enabled security with registration tokens, protected system indices, a technical preview of KNN search, storage‑saving field encodings, faster geo‑point indexing, PyTorch model support for NLP, and numerous deprecations and improvements across aggregations, allocation, analysis, authentication, cluster coordination, and packaging.
This article examines a real‑world Elasticsearch cluster handling hundreds of terabytes, explains why high‑cardinality aggregations can be slower, and shows how setting execution_hint=map and tuning doc_values dramatically improves aggregation performance for ultra‑high‑concurrency workloads.
This article explains why a high‑traffic platform migrated from SQL Server to Elasticsearch, covering the technical motivations, performance characteristics, indexing strategies, data synchronization methods, API design, and operational monitoring to achieve a scalable, low‑latency search solution.
This article shares a practical experience of replacing a massive SQL Server data store with Elasticsearch, covering the motivations, architectural design, performance advantages, indexing mechanisms, data synchronization strategies, implementation details, and operational considerations for handling tens of billions of records.
This article details the business background, technical advantages, architecture, indexing mechanisms, clustering, data synchronization strategies, API design, and performance monitoring of Elasticsearch, illustrating how it replaces costly SQL LIKE queries with a scalable, high‑performance search solution for massive user activity data.
Elasticsearch is a distributed, schemaless search engine built on Lucene that stores JSON documents in sharded indexes, uses immutable segments and merges, provides a flexible Query DSL with aggregations and relevance scoring, and executes distributed query‑then‑fetch searches with features like scrolling, optimistic locking, and zero‑downtime reindexing.
This article explains how Elasticsearch uses inverted indexes, term dictionaries, and compression techniques like FOR and Roaring Bitmaps to enable rapid full‑text search, contrasting its approach with traditional relational databases and offering practical indexing tips for large‑scale applications.
Benchmark tests reveal that RedisJSON, powered by RediSearch, dramatically outperforms MongoDB and ElasticSearch across isolated reads and writes, mixed workloads, and latency metrics, delivering up to 500‑fold speed gains, lower latency, and higher throughput, making it a compelling choice for modern data‑intensive applications.
Recent benchmark tests show that RedisJSON, powered by RediSearch, dramatically outperforms MongoDB and ElasticSearch in isolated writes, isolated reads, and mixed workloads, delivering up to 12.7× faster reads, 5.4× faster writes, and significantly lower latency across various operation types.
This article explains Elasticsearch’s cluster and data layer architecture, covering nodes, indices, shards, replicas, mixed and tiered deployment models, storage options, and the trade‑offs of different distributed system designs for scalable systems.
Apache Lucene 9.0, a high‑performance Java full‑text search library, introduces high‑dimensional vector indexing, new language analyzers, faster faceting and sorting, updated file formats, and several performance optimizations, providing developers with a richer, more efficient search toolkit.
This article explains how Elasticsearch leverages Lucene's inverted index, term dictionaries, and posting list compression methods such as FST and Frame‑of‑Reference to achieve fast, accurate, and tolerant search performance, while also discussing practical implementation details and examples.
This article explains how Elasticsearch leverages Lucene’s inverted index, detailing the structure of term dictionaries, postings lists, compression techniques like Frame‑of‑Reference and Roaring Bitmaps, and query optimizations such as filter caches and skip‑list intersections to achieve fast, memory‑efficient search.
This article explains how Elasticsearch uses inverted indexes, term dictionaries, and compression techniques such as Frame‑of‑Reference and Roaring Bitmaps to deliver rapid full‑text search, efficient storage, and fast union queries, while also offering practical indexing tips for production use.
This article explains how to integrate Elasticsearch into a Spring Boot project, covering version compatibility, supported access methods, dependency setup, configuration, entity mapping with annotations, repository creation, and unit testing to verify successful indexing and retrieval.
This article provides a detailed guide on Elasticsearch index design, covering index settings, shard and replica planning, mapping strategies, complex types, lifecycle management, template usage, and practical best‑practice recommendations for large‑scale log data clusters.
This article traces the rise of Elasticsearch as the leading full‑text search engine, explains the licensing changes that sparked conflict with cloud providers, outlines the emergence of OpenSearch as a fork, and compares the strategic choices available to users and vendors today.
This article explains how Elasticsearch uses inverted indexes, term dictionaries, and advanced compression techniques like Frame of Reference and Roaring Bitmaps to enable rapid, scalable search over massive datasets, contrasting its approach with traditional relational database queries and detailing practical optimization tips.
This article presents a comprehensive overview of JD Daojia's query understanding system for e‑commerce search, detailing its overall architecture, core modules such as tokenization, term weighting, query rewriting, intent detection, the algorithms employed, experimental evaluations, and future directions.
This article explains how Elasticsearch uses inverted indexes, term dictionaries, and postings lists—along with compression techniques like Frame of Reference and Roaring Bitmaps—to achieve fast, memory‑efficient search queries, and provides practical tips for optimizing indexing and query performance.
This article explains how Elasticsearch uses inverted index structures—including term dictionaries, term indexes, and postings lists—combined with compression methods like Frame‑of‑Reference and Roaring Bitmaps to achieve fast search, efficient storage, and effective union queries compared to traditional relational databases.
This article presents the architecture, techniques, and practical solutions behind intelligent question answering in QQ Browser's search engine, covering knowledge‑graph based QA (KBQA), machine‑reading‑comprehension QA (DeepQA), and information‑retrieval QA (IRQA), and discusses system design, model optimization, and future directions.
This article explains Elasticsearch's core mechanisms for indexing, querying, and searching data, covering the roles of coordinating nodes, primary and replica shards, refresh cycles, translog, commit/flush processes, and the underlying Lucene inverted index.
This article extends previous Elasticsearch cheat sheets with practical commands and step‑by‑step instructions for shard allocation, replica adjustment, cluster settings, slow‑log configuration, mapping routing, force merge, bulk writes, refresh intervals, translog durability, heap sizing, disk‑space monitoring, and troubleshooting strategies.
This article explains how Elasticsearch uses inverted indexes, term dictionaries, postings lists, and compression techniques like Frame‑of‑Reference and Roaring Bitmaps to deliver rapid, memory‑efficient search, and provides practical tips for indexing and query optimization.
This article explains Elasticsearch's core concepts—nodes, indices, shards, and replicas—describes the indexing flow, compares mixed and layered deployment models, and discusses data‑layer architectures, helping readers choose the appropriate cluster design for reliability and scalability.
This article compares Elasticsearch and ClickHouse in terms of architecture, query capabilities, and performance for log analytics, presenting test setups, Docker‑compose configurations, query examples, and benchmark results that show ClickHouse generally outperforms Elasticsearch in most basic query scenarios.
This article explains how Elasticsearch uses inverted indexes, term dictionaries, FST structures, and compression techniques like FOR and Roaring Bitmaps to dramatically speed up search queries over massive datasets while minimizing memory and disk usage.
This article provides a comprehensive overview of Elasticsearch, covering its fundamental architecture, key concepts such as indices, shards and replicas, the complete write and search workflows, consistency mechanisms, master node election, and practical performance‑tuning recommendations for large‑scale deployments.
This article explains how Elasticsearch achieves rapid search by using inverted indexes, detailing the structure of posting lists, term dictionaries, compression methods like Frame‑of‑Reference and Roaring Bitmaps, and how these techniques enable efficient union queries and filter caching.
This article explains how Elasticsearch leverages Lucene's inverted index, term dictionary, term index, and compression methods such as Frame‑of‑Reference and Roaring Bitmaps to enable rapid search, efficient storage, and fast set operations for large‑scale data retrieval.
This article explains how Elasticsearch achieves fast data retrieval by comparing it with traditional relational databases, detailing search engine fundamentals, the structure of Lucene's inverted index—including term dictionaries, postings lists, and term indexes—and the compression techniques such as Frame of Reference and Roaring Bitmaps that optimize storage and query performance.
The article provides a comprehensive overview of Baidu's Wanxiang system, detailing how it tackles the challenges of massive image and video data processing, feature extraction, cross‑media indexing, and real‑time retrieval to support modern search engine products.
This article provides a comprehensive introduction to Elasticsearch, covering its underlying Lucene-based inverted index, data types, shard routing, cluster roles, discovery mechanisms, refresh and translog handling, segment merging, and practical performance and JVM tuning tips for building scalable, near‑real‑time search systems.
This article explains why MySQL queries become slow, how proper indexing—including B+‑tree, left‑most prefix, index push‑down, and covering indexes—can improve performance, outlines common causes of index failure, and then introduces ElasticSearch search capabilities and HBase column‑family storage as complementary solutions for large‑scale data handling.
This guide shows how to set up the open‑source Searx search engine using a Docker image, walk through the essential Docker commands, explore its Python source code, and explains how to customize the response handling for building your own lightweight search engine.
This article shows how to quickly set up a searchable engine using the open‑source Searx Docker image, provides the necessary Docker commands, examines core Python code that aggregates query results, and suggests customizing responses with your own data sources.
The article introduces Apache Lucene 7.3.1, explains its core architecture and index hierarchy, details the two‑phase indexing and search workflow with code examples for document addition, deletion, merging, and query execution, and highlights its suitability for small‑to‑medium projects versus distributed alternatives.
This tutorial explains how to set up a single‑node Elasticsearch instance with Docker, integrate it into a Spring Boot application using Spring Data Elasticsearch Repositories, configure the necessary dependencies and settings, and perform basic CRUD operations and visualization with Kibana.
This article explains how Elasticsearch leverages inverted indexes, term dictionaries, and advanced compression techniques like Frame of Reference and Roaring Bitmaps to enable rapid full‑text search, covering the underlying concepts, data structures, and query optimizations essential for high‑performance search applications.
This article explains how Elasticsearch achieves fast search by using inverted indexes, detailing the structure of posting lists, term dictionaries, term indexes, and compression methods such as Frame of Reference and Roaring Bitmaps, as well as techniques for efficient union and intersection queries.
This article explains why MySQL queries become slow, how index design and common pitfalls affect performance, introduces MDL locks and large‑table strategies, then compares Elasticsearch and HBase as complementary storage and search solutions, providing practical code examples and best‑practice recommendations.
This article examines common performance pitfalls in Elasticsearch—including slow queries, cluster architecture bottlenecks, and business‑scenario challenges—and provides practical guidance such as caching key fields, data pre‑heating, hot‑cold separation, avoiding joins, and using tribe nodes to improve accuracy and response time.
This comprehensive guide walks through Elasticsearch fundamentals, including its architecture, core concepts like indices, shards, and replicas, the write and update processes, search workflow, consistency mechanisms, master election, performance tuning, and strategies for deep pagination and scroll searches.
This article explains the architecture and key algorithms behind a specialized exam‑question search engine, covering query parsing, BM25‑based relevance scoring, text‑image hybrid retrieval, Learning‑to‑Rank models, and practical optimizations for long queries and large K‑12 datasets.
This article presents a comprehensive overview of Meituan's query understanding system, detailing how entity recognition, entity linking, query rewriting, and intent detection are designed and deployed to improve O2O search relevance across diverse local services.
This comprehensive guide explains Elasticsearch’s role in handling massive data, covering its architecture, core concepts like shards and replicas, indexing and search processes, performance considerations, common pitfalls, and practical interview insights for developers and engineers.
This comprehensive guide explains Elasticsearch’s fundamentals, including its distributed architecture, indexing process, shard and replica mechanisms, query execution, near‑real‑time search, segment management, and practical optimization techniques, providing developers and engineers with the knowledge needed to design, operate, and troubleshoot large‑scale search clusters.
This article provides a comprehensive overview of Elasticsearch, covering its distributed architecture, fundamental concepts such as nodes, shards, and indices, as well as practical guidance on index design, bulk writing, query‑fetch workflow, scroll queries, aliases, and basic optimization tips.
Baidu’s Wànxiàng system processes billions of images and videos daily by extracting low‑ and high‑level features, linking related media, and aggregating semantic attributes in a scalable, timely architecture that leverages thousands of CPU, GPU, and FPGA cores to power accurate, low‑latency rich‑media search and recommendation.
This article presents multiple real‑world Elasticsearch deployments across Chinese tech giants, detailing how they handle massive query loads, scale clusters, integrate with Kafka and Kibana, and apply the Elastic Stack for search, analytics, and security monitoring.
This article explains how the 58 search engine achieves millisecond‑level real‑time inverted‑index updates by redesigning the underlying data structures, combining static and dynamic indexing, using a chain‑array hybrid, segment merging strategies, and lock‑free read‑write concurrency while maintaining search performance.
This article explains the architecture and core concepts of Elasticsearch and Lucene, outlines the requirements for cross‑month and high‑speed queries on massive datasets, and provides detailed index and search performance tuning techniques—including bulk writes, shard routing, doc‑values management, and pagination strategies—to achieve sub‑second response times on billions of records.
Apache Lucene 8.8.1, a high‑performance Java full‑text search engine library, has been released, fixing bugs from 8.8.0 and including optimizations, while the previous 8.8.0 version introduced features such as LatLonPoint spatial queries, configurable Doc value compression, and FeatureField’s newLinearQuery scoring.
This article recounts how Shay Banon’s early cooking‑app project led to the creation of Compass, the evolution into Elasticsearch, and the pivotal role of Apache Lucene, distributed testing, and visionary design in building today’s powerful search platform.
This article explains the concept of inverted indexes, their role in full‑text search, and provides a detailed overview of how Apache Lucene implements inverted indexing, including term dictionaries, posting lists, query processing, and numeric handling with BKDTree.
This article recounts how Shay Banon's early cooking‑app project led to the creation of Compass, the evolution of Apache Lucene, and ultimately the development of Elasticsearch—a powerful, distributed search platform built with extensive testing infrastructure and inspired by futuristic data‑interaction concepts.
This article introduces Elasticsearch’s core concepts—including clusters, node roles, documents, mappings, and shards—and walks through the complete indexing workflow from client request to replica synchronization, highlighting key settings, routing calculations, and the role of refresh and flush operations.
The article details WeChat Search’s practical distributed architecture—using a Chubby‑elected leader for shard‑to‑node mapping, hash‑based sharding with dynamic rebalancing, a Lambda‑style batch and near‑real‑time indexing pipeline, relaxed monotonic consistency, and group‑based searcher scaling—to illustrate trade‑offs and lessons for building scalable, reliable search services.
This article explains how Elasticsearch processes text before indexing by describing the analysis pipeline, built‑in and custom analyzers, tokenizers, token filters, n‑gram techniques, the analysis API, and the IK Chinese tokenizer plugin, providing practical curl examples throughout.
The article surveys core search‑engine technology—document hierarchy, flat and vertical inverted indexes, query operators for building and merging score lists, and ranking models from Boolean and BM25 to language‑model approaches like Indri—providing a foundational overview of information retrieval.
