This article explains Elasticsearch's logical and physical design, how documents are structured and indexed, the role of shards and replicas, and provides practical examples of indexing, searching, aggregations, and retrieving documents using RESTful APIs.
This article provides a detailed collection of basic Elasticsearch interview questions and answers, covering core concepts such as cluster and node architecture, indexing, mapping, search APIs, analyzers, tokenizers, aggregation, security, and operational tools, along with practical code snippets for common tasks.
This article introduces Elasticsearch as a Lucene‑based distributed search engine and demonstrates how to use curl commands to create, read, update, and delete documents and indices, providing step‑by‑step examples with command‑line output and screenshots.
The article offers a thorough, step‑by‑step guide to Elasticsearch, explaining how to check cluster health, create and manage indices, define mappings and field types, use dynamic mapping, and perform maintenance tasks such as shrink, split, rollover, and cache management, all illustrated with concrete API examples.
This article explains Elasticsearch's foundation on Lucene, compares its concepts to MySQL, describes inverted indexing, and provides a step‑by‑step guide for installing, configuring, and using the basemkhirat/elasticsearch Laravel plugin with code examples and tips for Chinese analysis.
The Hulu AI Class event showcased a series of technical talks covering large‑scale graph neural network optimizations, multi‑factor video ad placement algorithms, recommendation and search engine techniques, machine‑learning‑driven video codec improvements, and advanced content‑embedding methods, highlighting practical engineering experiences from Hulu’s Beijing office.
This article provides a comprehensive overview of general search engines, covering their classification, core workflow, key modules such as web crawlers, content processing, storage, user query handling, ranking strategies like TF‑IDF and PageRank, as well as anti‑cheat measures and user intent understanding.
The article explains cosine similarity from basic geometry and vector math, derives its formula, and shows how it powers precision marketing, image classification, and text retrieval, while also detailing its industrial implementation in Lucene’s vector space model.
A Beijing Internet Court case ruled that Baidu must pay nominal damages after failing to remove a user’s ID photo, uploaded to a defunct alumni site, which the search engine indexed and displayed at the top of results, highlighting legal responsibilities for personal data handling.
This article provides a comprehensive introduction to Elasticsearch, covering data types, the role of Lucene, cluster architecture, node roles, discovery mechanisms, shard and replica management, mapping, installation, health monitoring, indexing workflow, storage internals, refresh and translog processes, segment merging, and practical performance and JVM tuning tips.
This article reviews how major Chinese companies such as JD.com, Ctrip, Quark, 58.com, and Didi have adopted Elasticsearch for large‑scale search, log analysis, and real‑time analytics, detailing their cluster architectures, scaling strategies, and operational practices.
This article provides a comprehensive, top‑down and bottom‑up explanation of Elasticsearch’s internal architecture, covering clusters, nodes, shards, Lucene segments, inverted indexes, stored fields, document values, caching, merging, routing, scaling, and query processing, while addressing common performance questions.
This article explains the internal workings of Elasticsearch, covering its cloud‑based cluster architecture, Lucene‑based indexing structures such as segments, shards, inverted indexes, stored fields and doc values, as well as search processing, caching, merging, routing, and scaling strategies.
This article provides a comprehensive guide to improving Elasticsearch performance and stability by covering configuration file tweaks, system‑level settings, and usage‑level optimizations such as hot‑thread analysis, pending tasks, field storage, translog handling, refresh intervals, shard management, and best practices for routing and alias usage.
This guide walks through essential Elasticsearch operations, covering cluster health checks, creating and querying indices, managing settings, templates, aliases, index shrink, split, rollover, and detailed mapping definitions, providing practical code examples for each step.
This article consolidates practical Elasticsearch tuning techniques—including configuration file adjustments, system‑level optimizations, and usage‑level settings—to improve cluster performance, stability, and resource efficiency for production environments.
Elasticsearch 7.9.0 introduces critical security patches for field‑level leakage, updates script cache limits, refines field capabilities, improves snapshot restore throttling, expands thread‑pool write queue, deprecates dangling indices, and addresses known issues like mapping errors in machine‑learning indices.
This article explains Elasticsearch’s write, read, and search workflows, detailing the roles of coordinating nodes, primary and replica shards, refresh and commit cycles, translog handling, and the underlying Lucene inverted index mechanism.
This article introduces Elasticsearch as a distributed open‑source search and analytics engine, explains its core concepts and architecture, compares it with relational databases, details installation steps, configuration, indexing, analyzers, query DSL, pagination, sorting, and provides practical examples for building search functionality.
ElasticSearch achieves distributed search by organizing data into indices, types, mappings, documents, and fields, splitting indices into primary and replica shards across multiple nodes, with automatic master election and shard allocation, enabling horizontal scaling, high availability, and improved performance for large‑scale data workloads.
This article explains Elasticsearch's internal mechanisms for indexing, querying, and retrieving data, covering the roles of coordinating nodes, primary and replica shards, the refresh and commit cycles, near‑real‑time search, and the underlying Lucene inverted index.
Elasticsearch 7.8.1 introduces a range of new capabilities such as literal SUM/MIN/MAX/AVG in SQL, enhanced authorization for apm_user, updated index creation logging, composable template renaming, additional machine‑learning aggregations, snapshot/restore optimizations, and an improved update API, all aimed at boosting search and analytics performance.
Meituan’s AI‑driven search combines the Augur online inference framework—offering stateless, distributed feature operators, transformers, and a DSL for rapid, high‑throughput model scoring—with the Poker platform for model training, versioning, and experimentation, together accelerating iteration, improving performance, and enabling advanced model‑as‑feature ensembles.
This article details 58.com’s extensive use of Elasticsearch for search, analytics, and log processing, covering cluster optimization challenges, typical issues like disk exhaustion and write slowdown, practical solutions, development standards, ELKB architecture, real‑time log and MySQL slow‑log applications, platform‑as‑a‑service construction, and future roadmap plans.
This article explains the core components of a search engine—document processing, query processing, and matching—detailing each step from indexing to ranking, and discusses the document features that influence relevance, providing a comprehensive overview of information retrieval fundamentals.
This article explains Elasticsearch’s cluster and data layer architecture, detailing nodes, indices, shards, replicas, deployment models, and the trade‑offs of different distributed system designs, helping readers understand how Elasticsearch achieves scalability, reliability, and query performance.
This article provides a comprehensive tutorial on installing a single‑node Elasticsearch 7.x cluster, configuring its key settings, setting up the Elasticsearch‑head web UI, and includes a reference 6.x configuration file for production environments.
This article reviews how leading Chinese internet companies such as JD.com, Ctrip, Qunar, 58.com, and Didi have adopted Elasticsearch for large‑scale order search, log analysis, real‑time monitoring, and security, describing the evolution of cluster architectures, shard strategies, multi‑cluster pipelines, and performance optimizations.
This article explains how Elasticsearch uses Analyzer components—character filters, tokenizers, and token filters—to perform text analysis, reviews the built‑in analyzers such as standard, simple, stop, whitespace, keyword, pattern, language, ICU and IK, and provides practical _analyze API examples with code snippets and result screenshots.
This article examines Elasticsearch query performance by explaining Lucene's underlying data structures, describing how composite queries are merged, and presenting benchmark numbers for various query types such as term, range, and combined queries, highlighting optimization techniques and practical conclusions.
This article explains the differences between Master‑Slave and P2P ring architectures, introduces Elasticsearch’s core concepts, internal components, master election, shard routing, indexing and search processes, and discusses how the system avoids split‑brain scenarios and ensures high availability.
The article surveys how leading Chinese tech firms such as JD Daojia, Ctrip, Qunar, 58.com, and Didi have adopted Elasticsearch for large‑scale search, real‑time analytics, and security, detailing their evolving cluster architectures, shard strategies, data volumes, and supporting services.
Query understanding (QU) involves lexical, syntactic, and semantic analysis of user queries to enable effective search recall and ranking, covering modules such as preprocessing, correction, expansion, segmentation, intent detection, term importance, and guidance, with detailed discussion of algorithms, models, and system architecture.
This article offers a detailed, experience‑driven comparison of Elasticsearch against its main rivals—Lucene, Solr, relational databases, OpenTSDB, HBase, MongoDB, ClickHouse, and Druid—highlighting where Elasticsearch excels, where it falls short, and practical guidance for choosing the right data solution.
This article provides a detailed comparison of Solr 7.3 and Elasticsearch 7.4, covering their shared Lucene foundation, distributed architecture, feature sets, configuration approaches, storage options, shard management, and ecosystem differences to help readers choose the right search platform for their needs.
This comprehensive guide introduces Elasticsearch fundamentals, its features and use cases, then walks through integrating it with SpringBoot, configuring Maven dependencies, performing index and document operations, and demonstrates a variety of query types and aggregations using both RESTful APIs and Java code examples.
This article introduces Elasticsearch by explaining its core concepts such as indices, types, documents, mappings, and Query DSL, demonstrates common use cases, and provides step‑by‑step instructions for creating, updating, viewing, and deleting indices and documents using RESTful APIs, curl commands, and Docker‑compose deployment.
This article outlines a step‑by‑step procedure for safely taking an Elasticsearch node offline—checking master‑eligible settings, adjusting minimum_master_nodes, excluding the node from routing, waiting for shard relocation, stopping the service, and restoring the cluster routing—ensuring no data loss or service interruption.
This article explains what an inverted index is, why it’s essential for full‑text search, how it is built and queried, and common token transformations such as stop‑word removal, lemmatization, and stemming.
This article compares Elasticsearch and Solr, examining their history, community, licensing, core technologies, APIs, scalability, vendor support, ecosystem, performance, management tools, and visualization options to help organizations decide which open‑source search engine best fits their big‑data and search requirements.
This article introduces Elasticsearch as a distributed search and analytics engine, outlines its common application scenarios, provides step‑by‑step installation commands, explains core concepts such as documents and indices, and demonstrates basic indexing, retrieval, bulk processing, and aggregation operations.
Elasticsearch is a distributed, open‑source search and analytics engine built on Apache Lucene, offering a simple REST API, scalability, and a rich ecosystem—including Kibana, Beats, and Logstash—that enables storage, querying, and aggregation of text, numeric, and geospatial data for diverse use cases.
This article explains the architecture of Elasticsearch and Lucene, outlines common performance bottlenecks, and provides concrete indexing and search optimization techniques—including bulk writes, shard routing, doc values tuning, and pagination strategies—to achieve sub‑second query responses on billions of records.
This article explains how search engines collect, analyze, and rank web pages by describing the spider system, HTML and layout tree construction, feature extraction, and machine‑learning based classification methods used to understand page content and improve result relevance.
TurboSearch is Tencent AI Lab's next-generation large-scale search system, delivering distributed massive indexing, high-performance parallel retrieval, multi-granularity and multi-modal vector indexing, private Docker deployment, integrated NLP query analysis, extensible plugins, and robust operations for massive data and diverse search scenarios.
This article explains how query understanding (QP) transforms user search queries into structured semantic blocks and intent categories using rule‑based NLP, entity recognition, and post‑processing, and describes its taxonomy, implementation details, and practical impact on search engine results.
This article traces the evolution of web crawlers—from early Perl scripts to modern ByteDance agents—explaining their role in search engines, business models, anti‑crawling measures, and the impact on content creation and competition.
This article explains how 360 Search handles billions of daily crawls and hundred‑billion‑scale indexing by describing its overall architecture, core modules such as offline indexing and online retrieval, query analysis, relevance scoring, and the engineering techniques that enable efficient large‑scale web search.
This article describes how an e‑commerce platform built a big‑data pipeline—including TTLog, Lindorm, BCP, MetaQ, Blink and Xflush—to detect and verify search advertising placement errors in real time, covering background, implementation details, core challenges, and future optimization directions.
The article details JD.com's internally developed distributed search engine and recommendation platform, their new architectures, deep‑learning‑driven ranking and recall models, component‑based deployment, extensive performance testing, and coordinated operations that powered the massive 11.11 shopping event.
This article explains the fundamental architecture of Elasticsearch and its underlying Lucene engine, detailing the data model, index hierarchy, and the step‑by‑step write, refresh, flush, and merge processes that enable near‑real‑time search and data durability.
To avoid building separate search services for each Xianyu business, the team created a unified, generic search architecture based on Alibaba’s HA3 engine and a control layer that automates data dumping, indexing, query translation, and result ranking across five subsystems, enabling new services to be onboarded in minutes instead of weeks.
ByteDance has entered the Chinese search market with its new mobile‑first “Toutiao Search”, assembling a team from top tech firms and leveraging AI, NLP and computer‑vision technologies, sparking a fresh rivalry with Baidu’s long‑standing dominance.
This article provides a comprehensive overview of Elasticsearch, covering its role in handling structured and unstructured data, core concepts such as Lucene, inverted indexes, clusters, shards, replicas, mapping, indexing processes, storage mechanisms, and practical performance tuning tips for deployment.
This article explains how 360 Search processes billions of web pages daily, detailing its backend architecture, offline indexing, online retrieval, index organization, and relevance models that enable efficient search over a hundred‑billion‑scale web corpus.
This article explains the architecture of Elasticsearch clusters, covering core concepts such as nodes, indices, shards, replicas, indexing flow, mixed and tiered deployment models, data‑layer storage, and compares these designs with other distributed data system architectures.
This article narrates the origin and evolution of Elasticsearch, from its Lucene roots through Compass to the modern ELK Stack, illustrating how it simplifies full‑text search, log analysis, and real‑time monitoring for developers and operations teams.
This article provides a comprehensive tutorial for installing Elasticsearch 7.x in single‑node mode, configuring its key settings, deploying the Elasticsearch‑head web UI via Tomcat, and includes reference configuration files for Elasticsearch 6.x, complete with command‑line examples and code snippets.
This article describes the design and implementation of a spelling‑correction module for 58.com’s search engine, covering common query errors, rule‑based and statistical language‑model approaches, offline dictionary generation, n‑gram and Viterbi decoding, online workflow, and practical examples.
This article compiles common Elasticsearch interview questions and detailed answers covering cluster architecture, inverted index fundamentals, index design, write/query optimizations, master election, document indexing flow, search process, Linux tuning, and Lucene internals, providing practical guidance for candidates.
Open Distro for Elasticsearch is an AWS‑backed, fully open‑source distribution of Elasticsearch that adds enterprise‑grade security, alerting, SQL querying, and performance analysis features, includes Kibana for visualization, and follows a clear version‑release roadmap.
This tutorial walks you through installing Elasticsearch, creating indices, mapping schemas, inserting, updating, deleting, and searching documents using both cURL commands and the official Python client, illustrated with practical code examples and screenshots.
This article examines the unique characteristics of log data, outlines the challenges of using Elasticsearch at scale, and presents practical optimization techniques—including ingestion, mapping, time‑range search, metadata loading, and a custom C++ engine—to dramatically improve performance, stability, and cost efficiency.
This article walks through the fundamentals of search engines, explaining inverted indexes, the explosion of index size, core Elasticsearch concepts, its distributed architecture, and how it powers the ELK stack for log analysis, all illustrated with clear diagrams and examples.
The article explains how Xianyu guarantees high‑availability of its core Ha3‑based search engine through independent gateway deployment, multi‑datacenter disaster recovery, traffic isolation, comprehensive monitoring, pressure testing, gray releases, and automated/manual failover, enabling rapid issue detection, recovery, and continuous service stability.
This article reviews major in‑memory inverted index compression techniques such as PForDelta, PEF, and MILC, explains their principles and trade‑offs, and details practical optimizations applied at 58.com to achieve query performance comparable to uncompressed indexes while reducing memory usage by about 35 percent.
This article compares the two leading open‑source search engines, Solr and Elasticsearch, examining their architectures, features, deployment ease, scalability, community support, and ideal use cases to help you decide which solution best matches your application requirements.
This article details how JD.com’s order center migrated its Elasticsearch cluster through multiple stages—from an initial unoptimized deployment to a real‑time dual‑cluster backup solution—addressing scalability, reliability, shard tuning, version upgrades, and data synchronization strategies to support billions of documents and hundreds of millions of daily queries.
This article presents a comprehensive overview of constructing a search engine using Elasticsearch, covering architecture components, data read/write mechanisms, shard management, caching strategies, and real‑world case studies that illustrate performance tuning, isolation, and deployment best practices.
The article chronicles Elastic’s journey from a small open‑source search tool to a NYSE‑listed company, explaining Elasticsearch’s technical foundations, its real‑world applications, and what the IPO means for developers and the broader search‑technology ecosystem.
Elastic's NYSE debut, its rapid stock surge, the origins and technical strengths of Elasticsearch, and what the company's public listing means for developers and tech entrepreneurs are explored in detail, highlighting the journey from a personal recipe‑search tool to a global data‑search platform.
This article examines the high‑real‑time requirements of incremental data ingestion for search engines, compares three update schemes, and details how adopting a Kafka subscription‑based message‑queue approach dramatically improves latency and flexibility for the Nuomi search framework.
This article explains Elasticsearch mapping, covering dynamic mapping, field datatypes, meta‑fields, mapping parameters, and dynamic templates, while emphasizing the importance of careful schema design and the need for reindexing when changes are required.
This reflective essay recounts the author’s two‑and‑a‑half‑year stint at Infoseek, the strategic missteps around free‑mail branding, the transition to GO.COM, and the evolution of search‑engine ranking principles—from early link‑analysis patents to the so‑called confidence‑based bidding model.
This tutorial walks you through installing Elasticsearch, creating indices, inserting and updating documents, performing searches via REST API, and integrating Elasticsearch into Python applications using both raw HTTP requests and the official Python client, illustrated with practical examples and screenshots.
This article explains how 58.com’s in‑house C++ search kernel ESearch was architected to provide second‑level real‑time indexing, high‑concurrency low‑latency querying, flexible ranking models, and efficient storage structures for billions of daily queries across massive classified data.
This article explains the core of search engine technology by describing what an inverted index is, how it is built using single‑pass memory and multi‑way merge methods, how indexes can be partitioned and incrementally updated, and how Hadoop can be used for large‑scale indexing.
This article introduces Apache Solr and Elasticsearch, explains their shared Lucene foundation, compares their scalability, deployment ease, and key features, and highlights why Elasticsearch is often considered more efficient for large‑scale, multi‑tenant search applications.
This article explains Elasticsearch’s core mechanisms, covering the cluster startup sequence, node discovery and election, handling of failed nodes, cluster management APIs, and the detailed read/write processes including coordinating nodes, shard allocation, memory buffering, transaction logs, and query‑then‑fetch execution.
This article explains how 360’s Hulk private cloud platform deploys Elasticsearch with a dedicated master architecture, load‑balancing, per‑business isolated clusters, SearchGuard security, dynamic tokenization, self‑service user features, and advanced monitoring to achieve high‑performance, scalable search services.
This article explains the core challenges of search engines, the origins and mechanics of the PageRank algorithm, its handling of dead ends and spider traps, extensions like Topic‑Sensitive PageRank, and the various link‑spam attacks and countermeasures such as Spam Farms and TrustRank.
This guide explains how to set up a Go environment, install the open‑source Riot search engine, understand its goroutine‑based architecture for indexing and ranking, and provides a concise code example demonstrating document indexing and query execution for content‑driven websites.
The talk demonstrates how Meipai, a leading Chinese short‑video community, leverages large‑scale text mining and machine‑learning techniques—ranging from anti‑spam filtering to AI‑enhanced search—to enrich captions, comments, and metadata, improve user experience, and inspire further research on text data in video platforms.
The fourth Meitu Internet Technology Salon showcased practical AI and machine learning uses, highlighting Meipai’s text‑anti‑spam, hot‑topic detection, sentiment analysis and personalized video search, while Baidu demonstrated ML‑driven business intelligence tools for multi‑source data mining, user profiling, and intelligent enterprise and HR management.
The article recounts Xu Haiyang’s hands‑on experience designing Baidu’s early Spider system, describing its pure C procedural architecture, bug‑fixing journey, PageRank processing, team‑management analogies, and his later moves into AI and education entrepreneurship.
This article explains the fundamentals of search engine architecture, covering text collection, indexing, ranking and evaluation, and then traces Twitter's internal search evolution from MySQL full‑text search to the Earlybird index server, Blender aggregation, and smart memory‑SSD strategies.
This article provides a comprehensive overview of Elasticsearch 2.x, covering its distributed architecture, core concepts such as clusters, nodes, indices, shards and replicas, the ZenDiscovery master‑election process, scaling mechanisms, recovery, query features, and the underlying system components like Guice, Netty, and thread‑pool designs.
This article details Youzan's practical architecture, indexing pipeline, advanced search module, and performance‑tuning techniques for building a commercial e‑commerce search engine using Elasticsearch, Hadoop, Kafka, and custom backend integrations.
This article describes the practical architecture of Youzan's commercial e‑commerce search engine, covering data source integration, distributed real‑time indexing with Elasticsearch, Hadoop and Kafka, advanced search modules, and several performance‑tuning techniques for large‑scale deployments.
This article explains how Taobao scales to billions of daily page views using DNS‑based load balancing, LVS, domain sharding, CDN nodes, a distributed file system, sophisticated search processing, and massive data storage and real‑time log pipelines.
This article explains how Alibaba's Jushita platform leverages Apache Solr with a wide‑table data model and a custom QParser plugin to achieve real‑time, multi‑dimensional buyer filtering that traditional relational databases cannot handle efficiently in big‑data scenarios.
This article walks through installing Elasticsearch on Ubuntu, adding the IK Chinese analyzer plugin, configuring synonym filters, and using the official Node.js client to create indices, mappings, and perform full‑text searches with custom analyzers and highlighting.
The article explores historical privacy tactics of the USSR and the United States, offers practical habits for protecting personal information online, explains how to detect leaked data using search engines and social‑media checks, and suggests strategies for mitigating exposure and crafting false identities.
The article explains how search engine techniques such as large‑scale candidate recall, fine‑grained ranking, user profiling, and multi‑objective optimization are applied to news personalization at Toutiao, highlighting data sampling, machine‑learning pipelines, challenges of news freshness, and architectural evolution.
This article explains how graph traversal algorithms like BFS and DFS underpin web crawlers, illustrating the concepts with examples from China's road network and tracing the history from Euler's bridges to modern internet indexing.
Although sites like Taobao appear simple to users, they rely on massive distributed search, caching, storage, load‑balancing, CDN, logging, and data‑analysis systems that demand sophisticated backend engineering, massive infrastructure, and specialized algorithms, explaining why countless top engineers are required to keep them running.
This article details Suning's globally distributed search platform, covering its AI‑powered segmentation, ranking, no‑result handling, and the underlying distributed system architecture that enables rapid processing of millions of e‑commerce items.
