How Elasticsearch Handles Write, Read, and Search: Inside the Engine

Interview Questions

What is the working principle of Elasticsearch (ES) when writing data? When querying data? Briefly introduce the underlying Lucene and inverted index.

Interviewer's Psychology

The interviewer wants to verify whether you understand ES's basic principles, such as how data is written and searched, and whether you know what happens behind the scenes when a request is made.

Interview Question Analysis

ES Write Process

The client selects a node, which becomes the coordinating node.

The coordinating node routes the document to the appropriate node that holds the primary shard.

The primary shard processes the request and synchronizes the data to the replica shard s.

After the primary and all replicas have handled the request, the coordinating node returns the response to the client.

ES Read Process

The client sends a request to any node, which becomes the coordinate node.

The coordinate node hashes the doc id to determine the target shard and uses a round‑robin algorithm to select a primary or replica shard for load balancing.

The selected node returns the document to the coordinate node, which then forwards it to the client.

ES Search Process

Elasticsearch excels at full‑text search. For example, given three documents containing the word "java", a search for the keyword java returns the matching documents.

The client sends the query to a coordinate node.

The coordinating node forwards the request to all relevant primary or replica shards.

Query phase: each shard returns matching doc id s to the coordinating node, which merges, sorts, and paginates the results.

Fetch phase: the coordinating node pulls the actual document data from the shards using the doc id and returns the final result to the client.

Write requests go to the primary shard and are then synchronized to all replica shards; read requests can be served by either primary or replica shards using a random round‑robin algorithm.

Underlying Write Mechanics

Data is first written to an in‑memory buffer and the translog. When the buffer is near full or after a timeout, a refresh moves the data to the OS cache, making it searchable. By default, ES refreshes every second, providing near real‑time (NRT) search.

A manual refresh can force the buffer data into the OS cache immediately. After a refresh, the buffer is cleared because the data is persisted in the translog.

Every second a new segment file is created in the OS cache, containing the latest buffered data. When the translog grows large or after 30 minutes, a commit (also called flush) writes the buffered data to disk, creates a commit point, and fsyncs the OS cache to permanent storage.

If the machine crashes, up to 5 seconds of data may be lost because it resides only in the buffer, translog, or OS cache before being flushed to disk.

Data is written to the memory buffer, refreshed to the OS cache every second (making it searchable), and periodically flushed to disk; this design yields near real‑time search with a possible 5‑second data loss window.

Delete/Update Mechanics

Delete operations generate a .del file marking the document as deleted; updates mark the old document as deleted and write a new version. Segment files are merged periodically, physically removing deleted documents and creating a new commit point.

Lucene Basics

Lucene is a Java library that provides the algorithms for building inverted indexes. By using Lucene's API, developers can create and manage indexes stored on local disk.

Inverted Index

An inverted index maps terms to the list of document IDs containing those terms. It may also store term frequencies and positions. Search engines use this structure to quickly retrieve documents matching a query term.

The example shown is a simplified illustration; in a real inverted index, terms are sorted lexicographically.