Tagged articles
Lucene
99 articles · Page 1 of 1
The author migrates a Spring AI Agent with 30+ business tools to Spring AI 2.0's Tool Search, which uses Lucene indexing to let the model search for relevant tools before calling them, reducing context overload and improving selection accuracy, while sharing best practices for tool descriptions and safe write-operation handling.
AI AgentAgent ArchitectureJava
0 likes · 15 min read

This article explains why running Elasticsearch in production is far more complex than a simple API tutorial, covering Lucene fundamentals, mapping design, shard planning, write‑path architecture, query optimization, hot‑warm‑cold tiering, ILM policies, capacity planning, monitoring, incident handling, and a step‑by‑step evolution roadmap for building a reliable, scalable search system.
ElasticsearchIndex ModelingLucene
0 likes · 54 min read

This article explains why traditional database search struggles with large data sets, introduces Lucene as a high‑performance full‑text engine, and shows step‑by‑step how to integrate it into Spring Boot 3 using Hibernate Search, custom analyzers, entity mapping, repository extensions, and test cases.
Hibernate SearchJavaLucene
0 likes · 11 min read

Elasticsearch 9.0, released on April 15, 2025, builds on Lucene 10.1.0 to deliver major performance gains, introduces Better Binary Quantization, Elastic Distributions of OpenTelemetry, LLM observability, AI‑driven attack discovery, enhanced ES|QL, and is available via Elastic Cloud with deployment tips and examples.
AIElasticsearchLucene
0 likes · 7 min read

The article explains why Elasticsearch's _count and _stats APIs can return vastly different document totals, especially when nested fields are involved, and provides step‑by‑step analysis, code examples, and practical solutions such as index refresh and data‑model adjustments.
COUNTElasticsearchLucene
0 likes · 7 min read


JavaEdgeDec 22, 2024 · Backend Development
LinkedIn’s message search system stores messages in RocksDB, builds Lucene inverted indexes on demand, partitions them by user, keeps indexes in memory, and uses a coordinator with D2/Zookeeper for node routing, enabling rapid, cost‑effective searches while minimizing write overhead.
BackendLinkedInLucene
0 likes · 9 min read

The article explains how Elasticsearch implements efficient search using an inverted index, detailing its components—posting list, term dictionary, and term index—illustrating the query flow with concrete examples, compression techniques, and the role of Lucene.
ElasticsearchLuceneTerm Index
0 likes · 11 min read

The article explains Elasticsearch's on‑heap and off‑heap memory design, details each memory pool and its management via LRU, describes Lucene's role in off‑heap storage, and provides practical tips for configuring JVM heap size, cache strategies, and monitoring to achieve stable, high‑performance search.
CachingElasticsearchJVM
0 likes · 8 min read

This article explains Elasticsearch's distributed architecture, shard and replica model, the detailed write path—including buffer, transaction log, refresh and flush operations—and the read path with coordinating nodes, Lucene query execution, aggregation, caching, and performance optimizations.
ElasticsearchLuceneRead Process
0 likes · 14 min read

This article explains Elasticsearch’s overall structure, the underlying Lucene index layout, the indexing and search processing flow, and how built‑in analyzers work, illustrating each concept with diagrams, token examples, and query scenarios.
AnalyzersElasticsearchLucene
0 likes · 10 min read

This article explains how Elasticsearch transforms simple keyword matching into a high‑performance, scalable search engine by using inverted indexes, term dictionaries, posting lists, term indexes, stored fields, doc values, segments, and distributed node architectures to achieve fast, reliable full‑text search on massive data sets.
ElasticsearchLuceneinverted index
0 likes · 16 min read

This article provides a comprehensive, top‑down and bottom‑up explanation of ElasticSearch’s core architecture, detailing nodes, shards, Lucene segments, inverted indexes, stored fields, document values, caching, query processing, routing, and scaling considerations for efficient search operations.
LuceneShardinginverted index
0 likes · 10 min read

This article explains ElasticSearch’s architecture and core mechanisms—including its reliance on Lucene segments, inverted indexes, stored fields, document values, caching, shard routing, and scaling strategies—while answering common questions about wildcard matching, index compression, and memory usage.
Lucenebig datasearch engine
0 likes · 11 min read

This article explains how JD ElasticSearch has integrated the high‑performance ZSTD compression algorithm, details the motivations behind its adoption, presents benchmark results comparing it with LZ4 and best_compression, and provides step‑by‑step instructions and code snippets for configuring and using the new jd_zstd codec in Elasticsearch.
CompressionElasticsearchJava
0 likes · 14 min read

The team analyzed Lucene’s disk‑based inverted index and built a custom in‑memory inverted‑index service for Himalaya’s ad engine, encoding terms as 64‑bit keys, supporting real‑time updates and BooleanQuery‑style and custom expression filtering, which cut query latency from ~50 ms to under 5 ms and enabled massive scaling.
JavaLuceneMemory Search
0 likes · 27 min read

By integrating ZSTD compression into Didi’s Elasticsearch 7.6, the team cut CPU usage by about 15 %, reduced index storage roughly 30 %, boosted write throughput up to 25 %, and eliminated over 20 servers, demonstrating a faster, more storage‑efficient solution for petabyte‑scale log ingestion.
CompressionElasticsearchLucene
0 likes · 10 min read

This article explains how Apache Solr implements neural search using dense vector fields, K‑Nearest Neighbor algorithms, and Hierarchical Navigable Small World graphs, detailing the underlying Lucene support, configuration options, query syntax, and integration with AI‑driven vector representations.
AIApache SolrDense Vectors
0 likes · 15 min read

This article provides a detailed overview of Elasticsearch, covering its underlying Lucene technology, data types, indexing mechanisms, cluster architecture, shard and replica management, mapping definitions, installation steps, health monitoring, write and storage processes, and performance optimization techniques for production deployments.
ElasticsearchLucenePerformance Tuning
0 likes · 36 min read

This article explains how the community reduced Elasticsearch search response time from seconds to tens of milliseconds by applying Index Sorting, detailing the problem background, initial quick fixes, deep Lucene analysis, implementation steps, performance testing, and practical trade‑offs.
BackendOptimizationIndexSortingLucene
0 likes · 14 min read

By defining index sorting on the publish_time field when creating the Elasticsearch index, the team transformed a multi‑second full‑scan query into a sub‑50 ms operation, demonstrating that pre‑ordered storage dramatically speeds up large‑result‑set sorts while modestly affecting write throughput.
DocValuesElasticsearchIndex Sorting
0 likes · 12 min read


政采云技术Mar 2, 2023 · Fundamentals
This article explains the two‑phase commit protocol, describes how Lucene implements it through a dedicated interface, details the preparation, commit, segment handling, deletion policies, and rollback procedures, and provides code snippets illustrating the core logic.
LuceneTwo-Phase Commitdistributed transactions
0 likes · 13 min read


政采云技术Mar 2, 2023 · Databases
This article explains the two-phase commit protocol for distributed transactions, details its generic workflow, and describes how Apache Lucene implements the protocol through its TwoPhaseCommit interface, including preparation, flushing, commit, segment handling, deletion policies, and rollback mechanisms with illustrative code examples.
JavaLuceneTwo-Phase Commit
0 likes · 12 min read

This article documents the evolution of a data platform, explains Elasticsearch and Lucene fundamentals, and presents practical index and search performance optimizations—including bulk writes, refresh control, memory allocation, doc‑values tuning, and pagination strategies—that enable cross‑month queries and sub‑second responses on billions of records.
ElasticsearchLucenePerformance Tuning
0 likes · 11 min read

This article introduces Elasticsearch as a scalable distributed search engine, compares it with Solr, walks through local, Docker, and Kubernetes installation, explains its document‑oriented data model, demonstrates CRUD operations via REST API, and shows how to integrate it with Spring Boot using the official Java client.
CRUDDockerElasticsearch
0 likes · 23 min read

This article explains the fundamentals of structured and unstructured data, introduces Lucene’s inverted index, and details how Elasticsearch builds on Lucene to provide distributed, near‑real‑time search with concepts such as clusters, shards, replicas, routing, and performance optimizations.
Distributed SearchElasticsearchLucene
0 likes · 36 min read


21CTONov 18, 2022 · Big Data
This article details a real‑world case study of optimizing Elasticsearch for massive daily data volumes, covering the underlying Lucene architecture, shard routing, index and search performance tweaks, practical configuration settings, and benchmark results that achieve sub‑second query responses on billions of records.
Lucenebig-dataindexing
0 likes · 13 min read

This article provides a comprehensive introduction to Elasticsearch, covering the distinction between structured and unstructured data, Lucene’s inverted index, ES core concepts such as clusters, nodes, shards and replicas, mapping, basic usage, storage mechanisms, and practical performance‑tuning tips for large‑scale search deployments.
ElasticsearchLucenedistributed systems
0 likes · 39 min read

This article presents a comprehensive case study of optimizing Elasticsearch and its underlying Lucene structures to achieve sub‑second query responses on billions of records, covering architecture basics, index design, doc‑values tuning, bulk‑write strategies, and extensive performance testing.
Lucenebig-dataindexing
0 likes · 12 min read

This article presents a comprehensive guide to optimizing Elasticsearch for massive datasets, covering Lucene fundamentals, index and shard architecture, practical performance‑tuning techniques, and real‑world case studies that achieve sub‑second query responses on billions of records.
ElasticsearchIndex OptimizationLucene
0 likes · 11 min read


政采云技术Aug 30, 2022 · Fundamentals
This article explains the key Lucene classes involved in document indexing, outlines the end‑to‑end write workflow—including preUpdate, obtainAndLock, updateDocument, exception handling, and post‑update flush logic—and discusses the strategies and thresholds that control when in‑memory buffers are flushed to disk.
ConcurrencyDocument WritingJava
0 likes · 16 min read

Tencent’s Cloud Log Service augments Lucene with a dedicated time‑series index—using timestamp ordering, a secondary index, reverse binary search, and histogram optimization—to cut log query complexity, delivering up to 40‑50× faster responses, higher concurrency, and markedly better performance than traditional ELK‑style and competing cloud log solutions.
LuceneVLDBcloud log service
0 likes · 14 min read

This article explains the fundamentals of structured and unstructured data, introduces Lucene's inverted index, describes Elasticsearch's distributed cluster architecture, node roles, sharding and replication mechanisms, indexing workflow with refresh and translog, storage segment model, and provides practical performance‑tuning recommendations.
ClusterElasticsearchLucene
0 likes · 36 min read


政采云技术May 12, 2022 · Fundamentals
This article explains Lucene's query types, the step‑by‑step query execution flow—including entry, rewrite, weight creation, scoring, and result collection—while providing code examples and performance considerations to help developers troubleshoot and optimize search performance.
BM25ElasticsearchJava
0 likes · 15 min read

This comprehensive guide explains what Elasticsearch does, its underlying Lucene engine, core concepts like clusters, shards, replicas, mappings, and provides practical steps for installation, configuration, indexing, storage mechanics, and performance optimization.
LuceneShardingcluster management
0 likes · 36 min read

This article provides a comprehensive overview of Elasticsearch, explaining its underlying Lucene technology, data modeling, cluster architecture, shard and replica mechanisms, indexing workflow, storage strategies, refresh and translog processes, as well as practical performance and JVM tuning tips for building scalable, near‑real‑time search solutions.
ElasticsearchLuceneSharding
0 likes · 37 min read

This article provides a comprehensive overview of Elasticsearch, covering data types, Lucene fundamentals, cluster discovery, node roles, shard and replica management, mapping, installation, health monitoring, indexing mechanics, storage strategies, refresh and translog processes, segment merging, and practical performance optimizations for production deployments.
ElasticsearchLucenedistributed systems
0 likes · 39 min read

This comprehensive article explains what Elasticsearch is, how it builds on Lucene to provide distributed real‑time search and analytics, covering data types, cluster components, shard routing, indexing pipelines, storage formats, segment merging, and practical performance‑tuning tips for production deployments.
ElasticsearchLuceneindexing
0 likes · 36 min read

This article provides a comprehensive overview of Elasticsearch, covering its data types, Lucene-based inverted index, cluster architecture, sharding and replication mechanisms, mapping definitions, basic usage, health monitoring, storage internals, and practical performance tuning tips for large‑scale search deployments.
ClusterElasticsearchLucene
0 likes · 36 min read

This article explains how Elasticsearch uses inverted indexes, term dictionaries, and compression techniques like FOR and Roaring Bitmaps to enable rapid full‑text search, contrasting its approach with traditional relational databases and offering practical indexing tips for large‑scale applications.
CompressionElasticsearchLucene
0 likes · 15 min read

This article explains how Elasticsearch leverages Lucene’s inverted index, detailing the structure of term dictionaries, postings lists, compression techniques like Frame‑of‑Reference and Roaring Bitmaps, and query optimizations such as filter caches and skip‑list intersections to achieve fast, memory‑efficient search.
CompressionElasticsearchLucene
0 likes · 19 min read

This article explains how Elasticsearch uses inverted indexes, term dictionaries, and compression techniques such as Frame‑of‑Reference and Roaring Bitmaps to deliver rapid full‑text search, efficient storage, and fast union queries, while also offering practical indexing tips for production use.
CompressionLucenePostings List
0 likes · 15 min read

This article explains how Elasticsearch uses inverted indexes, term dictionaries, and advanced compression techniques like Frame of Reference and Roaring Bitmaps to enable rapid, scalable search over massive datasets, contrasting its approach with traditional relational database queries and detailing practical optimization tips.
CompressionElasticsearchLucene
0 likes · 16 min read

This article explains how Elasticsearch implements near real-time search by using immutable inverted indexes, segment merging, sharding, and a translog for durability, detailing the challenges and solutions behind its distributed full‑text search architecture.
ElasticsearchLuceneNear Real-Time Search
0 likes · 9 min read

This article explains how Elasticsearch uses inverted indexes, term dictionaries, and postings lists—along with compression techniques like Frame of Reference and Roaring Bitmaps—to achieve fast, memory‑efficient search queries, and provides practical tips for optimizing indexing and query performance.
CompressionElasticsearchLucene
0 likes · 14 min read

This article explains how Elasticsearch uses inverted index structures—including term dictionaries, term indexes, and postings lists—combined with compression methods like Frame‑of‑Reference and Roaring Bitmaps to achieve fast search, efficient storage, and effective union queries compared to traditional relational databases.
CompressionElasticsearchLucene
0 likes · 14 min read

This article explains Elasticsearch's core mechanisms for indexing, querying, and searching data, covering the roles of coordinating nodes, primary and replica shards, refresh cycles, translog, commit/flush processes, and the underlying Lucene inverted index.
Data IngestionElasticsearchLucene
0 likes · 13 min read


21CTOSep 12, 2021 · Big Data
This article explains how Elasticsearch uses inverted indexes, term dictionaries, FST structures, and compression techniques like FOR and Roaring Bitmaps to dramatically speed up search queries over massive datasets while minimizing memory and disk usage.
CompressionLuceneinverted index
0 likes · 14 min read

This article explains how Elasticsearch achieves rapid search by using inverted indexes, detailing the structure of posting lists, term dictionaries, compression methods like Frame‑of‑Reference and Roaring Bitmaps, and how these techniques enable efficient union queries and filter caching.
CompressionElasticsearchLucene
0 likes · 14 min read

This article explains how Elasticsearch leverages Lucene's inverted index, term dictionary, term index, and compression methods such as Frame‑of‑Reference and Roaring Bitmaps to enable rapid search, efficient storage, and fast set operations for large‑scale data retrieval.
CompressionElasticsearchLucene
0 likes · 16 min read

This article explains how Elasticsearch achieves fast data retrieval by comparing it with traditional relational databases, detailing search engine fundamentals, the structure of Lucene's inverted index—including term dictionaries, postings lists, and term indexes—and the compression techniques such as Frame of Reference and Roaring Bitmaps that optimize storage and query performance.
CompressionElasticsearchLucene
0 likes · 14 min read

This article shares practical lessons from three iterations of a data platform, focusing on Elasticsearch and Lucene optimizations that enable cross‑month queries, year‑long data export, and sub‑second query responses for tables handling billions of rows per day.
ElasticsearchLuceneindexing
0 likes · 13 min read

This article describes how a top architect optimized Elasticsearch for handling billions of records, covering Lucene fundamentals, index and shard design, DocValues, query performance tuning, bulk indexing strategies, hardware considerations, and testing methods to achieve sub‑second query responses across multi‑year data ranges.
ElasticsearchIndex OptimizationLucene
0 likes · 12 min read

This article presents a comprehensive guide on optimizing Elasticsearch for large‑scale data platforms, covering Lucene fundamentals, index and shard architecture, doc‑values usage, routing strategies, practical performance‑tuning techniques, and real‑world testing results to achieve sub‑second query responses on billions of records.
ElasticsearchIndex OptimizationLucene
0 likes · 12 min read

The article introduces Apache Lucene 7.3.1, explains its core architecture and index hierarchy, details the two‑phase indexing and search workflow with code examples for document addition, deletion, merging, and query execution, and highlights its suitability for small‑to‑medium projects versus distributed alternatives.
Code ExampleJavaLucene
0 likes · 20 min read

This article explains how Elasticsearch leverages inverted indexes, term dictionaries, and advanced compression techniques like Frame of Reference and Roaring Bitmaps to enable rapid full‑text search, covering the underlying concepts, data structures, and query optimizations essential for high‑performance search applications.
CompressionElasticsearchLucene
0 likes · 17 min read

This article explains Elasticsearch's write, read, and search mechanisms, the role of coordinating nodes, primary and replica shards, refresh and commit cycles, Lucene's inverted index, and how data becomes searchable in near‑real‑time.
ElasticsearchLuceneNear Real-Time
0 likes · 12 min read

This article explains the architecture and core concepts of Elasticsearch and Lucene, outlines the requirements for cross‑month and high‑speed queries on massive datasets, and provides detailed index and search performance tuning techniques—including bulk writes, shard routing, doc‑values management, and pagination strategies—to achieve sub‑second response times on billions of records.
ElasticsearchIndex OptimizationLucene
0 likes · 13 min read

This article explains how Elasticsearch handles data ingestion, retrieval, and full‑text search by describing the roles of coordinating, primary, and replica nodes, the refresh‑commit‑flush cycle, segment files, translog, and the Lucene‑based inverted index that powers its near‑real‑time capabilities.
ElasticsearchLuceneRead Process
0 likes · 11 min read

This article explains how to optimize Elasticsearch for cross‑month and multi‑year queries on billions of records, covering Lucene fundamentals, index and search performance tweaks, configuration settings, and practical testing results to achieve sub‑second response times.
ElasticsearchLucenebig data
0 likes · 14 min read

This article explains the concept of inverted indexes, their role in full‑text search, and provides a detailed overview of how Apache Lucene implements inverted indexing, including term dictionaries, posting lists, query processing, and numeric handling with BKDTree.
BKDTreeLuceneinverted index
0 likes · 15 min read

This article explains Elasticsearch's internal mechanisms for writing, reading, and searching data, covering the roles of coordinating nodes, primary and replica shards, buffers, translog, segment files, refresh cycles, commit and flush operations, as well as Lucene's inverted index and how deletions and updates are handled.
ElasticsearchLuceneNear Real-Time
0 likes · 10 min read

This article explains how to design, tune, and optimize an Elasticsearch‑based data platform handling hundreds of billions of records, covering Lucene fundamentals, shard routing, index and query performance tricks, and practical benchmark results for large‑scale deployments.
ElasticsearchLuceneindexing
0 likes · 13 min read

This article explains the different index mechanisms used by MySQL (MyISAM and InnoDB) and Lucene, compares them with Elasticsearch's inverted index, and discusses how these structures affect storage, memory usage, and query performance.
ElasticsearchInnoDBLucene
0 likes · 8 min read

This article explains Elasticsearch’s internal architecture, from cloud clusters and nodes to shards and Lucene’s inverted index, covering indexing, storage structures, query processing, caching, scaling, routing, and real‑world request handling, with detailed diagrams and examples.
LuceneShardsdistributed
0 likes · 13 min read

This article shares practical Elasticsearch and Lucene optimization techniques—including index structure tuning, shard routing, DocValues management, and query pagination—to achieve sub‑second search performance on datasets exceeding a billion records while supporting multi‑year historical queries.
ElasticsearchLuceneindexing
0 likes · 13 min read

The article explains cosine similarity from basic geometry and vector math, derives its formula, and shows how it powers precision marketing, image classification, and text retrieval, while also detailing its industrial implementation in Lucene’s vector space model.
LuceneTF-IDFVector Space Model
0 likes · 18 min read

This article provides a comprehensive, top‑down and bottom‑up explanation of Elasticsearch’s internal architecture, covering clusters, nodes, shards, Lucene segments, inverted indexes, stored fields, document values, caching, merging, routing, scaling, and query processing, while addressing common performance questions.
CachingElasticsearchLucene
0 likes · 11 min read

This article explains the internal workings of Elasticsearch, covering its cloud‑based cluster architecture, Lucene‑based indexing structures such as segments, shards, inverted indexes, stored fields and doc values, as well as search processing, caching, merging, routing, and scaling strategies.
ElasticsearchLuceneSharding
0 likes · 13 min read

Elasticsearch, built on Apache Lucene, provides a distributed, near‑real‑time search platform that scales to billions of documents across thousands of nodes, supporting use cases such as log analytics, time‑series monitoring, and product search, while Tencent’s CES adds advanced availability, performance, and cost‑optimizing features.
ElasticsearchLucenebig data
0 likes · 17 min read

This article explains Elasticsearch’s write, read, and search workflows, detailing the roles of coordinating nodes, primary and replica shards, refresh and commit cycles, translog handling, and the underlying Lucene inverted index mechanism.
ElasticsearchLuceneNear Real-Time
0 likes · 11 min read

This article details how Ctrip's hotel search system was optimized through storage compression, spatial indexing, KV storage, semantic query generation, context‑aware error correction, and a custom domain‑specific language, balancing performance, flexibility, and user experience for large‑scale online travel services.
DSLLucenebackend optimization
0 likes · 19 min read

This article explains Elasticsearch's internal mechanisms for indexing, querying, and retrieving data, covering the roles of coordinating nodes, primary and replica shards, the refresh and commit cycles, near‑real‑time search, and the underlying Lucene inverted index.
Data IngestionElasticsearchLucene
0 likes · 12 min read

This article explains the motivations, requirements, and technical design of the DaJia app's search system, compares relational databases with Lucene‑based solutions, describes the inverted index mechanism, outlines common search workflows, and details the system's three iterative development phases and future improvement plans.
BackendElasticsearchLucene
0 likes · 12 min read

This article describes how the Qunar hotel search team reduced Lucene stored‑fields deserialization overhead and GC pressure by implementing a custom Codec that caches stored fields in memory, redesigning the storage format, and evaluating the performance and space benefits of the approach.
CachingLucenecodec
0 likes · 12 min read

iQIYI Douya’s emoji search evolved from ElasticSearch to a pure Lucene implementation and added semantic vector retrieval, enabling fast, scalable, and more accurate text‑based search of AI‑generated images for small‑to‑medium businesses by combining custom tokenization, dense embeddings, and hybrid ranking.
ElasticsearchLuceneSearch Architecture
0 likes · 14 min read

This article examines Elasticsearch query performance by explaining Lucene's underlying data structures, describing how composite queries are merged, and presenting benchmark numbers for various query types such as term, range, and combined queries, highlighting optimization techniques and practical conclusions.
BKD-TreeElasticsearchLucene
0 likes · 13 min read

This article offers a detailed, experience‑driven comparison of Elasticsearch against its main rivals—Lucene, Solr, relational databases, OpenTSDB, HBase, MongoDB, ClickHouse, and Druid—highlighting where Elasticsearch excels, where it falls short, and practical guidance for choosing the right data solution.
ElasticsearchLuceneSolr
0 likes · 15 min read


ITPUBMay 5, 2020 · Backend Development
This guide explains the background, requirements, Elasticsearch architecture, Lucene fundamentals, and practical tuning steps—including indexing, shard routing, doc values, and hardware choices—to achieve cross‑month, sub‑second query performance on datasets exceeding a billion records.
Lucene
0 likes · 12 min read

This article examines how Elasticsearch implements write operations on top of Lucene, detailing the challenges of Lucene's write path and describing Elasticsearch's distributed design, near‑real‑time refresh, translog reliability, shard replication, partial updates, and the complete write workflow from coordinating node to primary and replica shards.
ElasticsearchLuceneShard
0 likes · 14 min read

This article explains the architecture of Elasticsearch and Lucene, outlines common performance bottlenecks, and provides concrete indexing and query optimization techniques—including shard routing, refresh intervals, doc values, and hardware considerations—to achieve sub‑second query responses on billions of records.
ElasticsearchLuceneindexing
0 likes · 12 min read

This article explains the architecture of Elasticsearch and Lucene, outlines common performance bottlenecks, and provides concrete indexing and search optimization techniques—including bulk writes, shard routing, doc values tuning, and pagination strategies—to achieve sub‑second query responses on billions of records.
ElasticsearchLucenePerformance Tuning
0 likes · 14 min read

An in‑depth guide walks through Elasticsearch’s underlying Lucene architecture, explains shard routing and DocValues, then presents concrete index‑ and search‑performance tweaks—bulk writes, refresh intervals, memory allocation, SSD usage, field mapping, pagination strategies—and shows benchmark results that reduce query latency to seconds for billions of records.
ElasticsearchIndex OptimizationLucene
0 likes · 13 min read


ITPUBDec 5, 2019 · Big Data
This article explains how a data platform handling billions of daily records can be optimized for cross‑month queries and sub‑second response times by tuning Elasticsearch indexing, shard routing, Lucene structures, and hardware configurations.
LucenePerformance TuningSearch Optimization
0 likes · 13 min read

The article explains how Elasticsearch’s use of Lucene’s flush mechanism, concurrent shard writes, and IndexWriter buffering lead to an excess of small segments, outlines the flush conditions, and offers guidance on managing write concurrency for better performance.
ElasticsearchIndexWriterLucene
0 likes · 10 min read

This article presents a comprehensive guide to optimizing Elasticsearch for massive data volumes, covering Lucene fundamentals, index and shard design, practical performance‑tuning techniques, and real‑world testing results that enable cross‑month queries and sub‑second response times.
ElasticsearchIndex OptimizationLucene
0 likes · 14 min read


ITPUBSep 1, 2019 · Databases
This article explains the fundamental architecture of Elasticsearch and its underlying Lucene engine, detailing the data model, index hierarchy, and the step‑by‑step write, refresh, flush, and merge processes that enable near‑real‑time search and data durability.
Lucenedata indexingsearch engine
0 likes · 8 min read

This article explains the principles of full‑text search, contrasts structured and unstructured data retrieval methods, introduces Lucene, Solr, and Elasticsearch, and provides a detailed comparison of their features, community support, maturity, and documentation to help developers choose the right search engine for their projects.
ElasticsearchLuceneSolr
0 likes · 15 min read

This article chronicles Doug Cutting's journey from his humble beginnings at Stanford through his pioneering work on Lucene, Nutch, and Hadoop, highlighting how his innovations in search and distributed computing reshaped the big data landscape and led to the rise of Cloudera.
ClouderaDoug CuttingHadoop
0 likes · 8 min read

Meltwater details how it processes millions of daily media posts using a custom‑tuned Elasticsearch 1.7.6 cluster of over 400 nodes on AWS, covering data volume, query complexity, node configuration, indexing strategy, performance optimizations, and lessons learned for large‑scale search deployments.
AWSCluster ScalingElasticsearch
0 likes · 12 min read

This article explains the fundamentals of full‑text search, compares Lucene‑based engines Solr and Elasticsearch, and provides practical guidance on selecting the appropriate solution based on use cases, performance, scalability, and community support.
ElasticsearchLuceneSolr
0 likes · 17 min read

This guide details how to optimize Elasticsearch for handling billions of daily records, covering core Lucene concepts, index and shard configuration, performance‑tuning parameters, and practical testing methods to achieve sub‑second query responses and long‑term data retention.
ElasticsearchLucenebig data
0 likes · 13 min read

This article explains the core Elasticsearch architecture—including shard routing, primary‑replica interaction, document CRUD workflows, multi‑document APIs, segment merging, translog durability, and storage file formats—providing a comprehensive view of how near‑real‑time search is achieved on large‑scale data.
ElasticsearchLuceneSegment Merging
0 likes · 20 min read

This article explains the fundamentals of full‑text search, describing how Lucene builds and uses inverted indexes, the steps of tokenization, linguistic processing, term weighting, and relevance scoring, and illustrates these concepts with examples, tables, and diagrams.
LuceneTerm WeightVector Space Model
0 likes · 21 min read

This article introduces SolrCloud, explains its relationship with Lucene and Solr, provides environment setup instructions for a CentOS 7.3 cluster, details Maven dependencies, configuration files, and a comprehensive Java implementation using Spring Boot, including repository interfaces, utility classes, and extensive unit tests for adding, querying, and deleting documents.
DistributedSearchJavaLucene
0 likes · 8 min read

The second part of the Elasticsearch search‑tuning series explains how to pre‑index data, choose appropriate keyword or text mappings, minimize script use by preferring Painless or Lucene expressions, and efficiently force‑merge read‑only indices into single segments for better performance.
Data MappingElasticsearchLucene
0 likes · 8 min read

This article explains how Elasticsearch computes relevance scores using Lucene's practical scoring formula, term frequency, inverse document frequency, field-length norms, and query normalization, and demonstrates query-time boosting, constant_score, function_score, decay functions, and script_score with practical DSL examples.
ElasticsearchLuceneQuery Boosting
0 likes · 14 min read

This article presents a design for a transparent, flexible, and low‑resource distributed logging solution that uses Lucene for indexing, Apache Ignite for service and compute grids, and a custom Log4j2 appender, enabling fast, unified log queries across clustered applications.
Apache IgniteLog4j2Logging
0 likes · 15 min read

This article reviews major big‑data storage and query architectures—including HBase, Dremel/Parquet, pre‑aggregation systems, Lucene, and the custom Tindex solution—evaluating their strengths, weaknesses, and suitability for real‑time, high‑volume analytical workloads.
HBaseLuceneParquet
0 likes · 20 min read

This article walks through the fundamentals of search engine architecture, explains Lucene's role as an indexing library, details ElasticSearch's distributed design, clustering, sharding, and plugins, and demonstrates practical RESTful API usage and query DSL techniques for effective log analysis.
BackendLuceneindexing
0 likes · 23 min read
