Master Elasticsearch: Core Concepts, Architecture, Queries & Performance for Interviews

🚀 Whether for job interviews or daily work such as search, log analysis, and monitoring systems, Elasticsearch is an indispensable technology. This article systematically reviews ES knowledge across seven aspects—principles, architecture, queries, tuning, operations, security, and practice—to help you master interview essentials in one article.

1. Core Concepts and Data Model

Basic Objects

Master Node: manages the cluster and shard allocation.

Data Node: stores and retrieves data.

Coordinating Node: coordinates queries.

Ingest Node: preprocesses data (pipeline).

Mapping

Shard and Replica

Elasticsearch text vs keyword

2. Working Principles

Inverted Index

Storage format : term → document ID list.

Advantages : efficient fuzzy and full‑text search.

Disadvantages : high update cost (requires new segment).

During query, the analyzer tokenizes the input and quickly finds documents containing those terms.

Suitable for full‑text search but not for frequent updates.

Write Process

Client request reaches a coordinating node.

The coordinating node routes the request to the primary shard (based on routing algorithm).

After the primary shard writes, replica shards are synchronized.

Success response is returned.

Query Process

Query Phase : coordinating node broadcasts the request to all shards. Each shard returns document IDs with scores and sort values, but not the full document.

Fetch Phase : coordinating node retrieves full documents for the top N IDs from the relevant shards and merges them into the final response.

Near Real‑Time Search (NRT)

Lucene stores data in segments.

By default, a refresh occurs every second; newly indexed data becomes searchable after the refresh.

Analyzer

Analyzer composition: character filter → tokenizer → token filter.

Common analyzers: StandardAnalyzer (default English), IK Analyzer (Chinese), KeywordAnalyzer (no tokenization).

3. Query DSL Deep Dive

Exact Queries term: exact match, not analyzed. terms: matches multiple values. range: range query for numbers or dates.

Full‑Text Queries match: analyzed query. multi_match: cross‑field search. query_string / simple_query_string: supports syntax like AND/OR/*.

Compound Queries bool: combines must, must_not, should, filter. boosting: increase or decrease weight. function_score: custom scoring.

Aggregations

Bucket : terms, date_histogram, range.

Metric : avg, sum, min, max, cardinality.

Pipeline : derivative, moving_avg, bucket_script.

Pagination from+size: poor performance, suitable for shallow pagination. search_after: deep pagination based on sort fields. scroll: cursor‑based, suitable for bulk export.

4. Performance Optimization

1. Index Optimization

Disable the _all field and turn off dynamic mapping.

Control shard size to 10–50 GB; avoid many small indices.

Use appropriate analyzer (e.g., IK for Chinese).

Design field types according to business: store both text and keyword when needed.

2. Write Optimization

Bulk API for batch writes.

Adjust refresh_interval (default 1 s) to a larger value during heavy ingestion.

Disable replicas during import, re‑enable after completion.

Merge small segments to reduce flush frequency.

3. Query Optimization

Prefer filter over query when scoring is not required (cacheable).

Use keyword with doc_values for aggregations and sorting.

Avoid leading wildcards ( *abc) or regex queries.

Set appropriate index sorting.

4. Cluster Optimization

Allocate heap memory to ~50 % of physical RAM, but not exceeding 32 GB.

Separate hot (SSD, high QPS) and cold (HDD, historical data) nodes.

Use ILM (Index Lifecycle Management) for automatic rollover and data migration.

Avoid excessive shard count.

5. Operations and Monitoring

Cluster Health

Green: primary and replica shards are healthy.

Yellow: primary shards healthy, replica shards missing.

Red: primary shards lost, data unavailable.

Common Issues

Unassigned shards: disk full or node failure.

Query timeout: too many shards or unreasonable DSL.

Out‑of‑memory: large aggregations or small heap.

Monitoring Metrics

Heap usage and GC count.

Indexing/query TPS.

Shard distribution balance.

Disk I/O and queue length.

Snapshot & Recovery

Snapshot repositories: HDFS, S3, NAS.

Supports incremental snapshots.

Can be used for cross‑cluster migration.

Common Troubleshooting Commands GET _cluster/health: view cluster health. GET _cat/indices: view index status. GET _cluster/allocation/explain: explain unassigned shard reasons.

6. Security and High Availability

Permission Management

X‑Pack Security: authentication and RBAC.

TLS encrypted transport.

API keys / tokens.

High‑Availability Design

Primary‑replica shard mechanism.

Master node election (Zen Discovery / Raft).

Cross‑cluster search (CCS).

Cross‑cluster replication (CCR) for disaster recovery.

7. Ecosystem and Use Cases

Logging & Monitoring

ELK (Elasticsearch + Logstash + Kibana).

EFK (Elasticsearch + Fluentd + Kibana).

Metricbeat / Filebeat.

Search Applications

E‑commerce site search.

Intelligent recommendation (with scoring functions).

Geolocation search (geo queries).

APM & Observability

Elastic APM for tracing and performance analysis.

Kibana dashboards.

Security Scenarios

SIEM (Security Information and Event Management).

Intrusion detection log analysis.

8. Typical Interview Questions

Why is ES faster than a traditional database? → Inverted index + memory cache avoids full table scans.

Is ES real‑time search? → No, it is near real‑time with ~1 s latency.

How to handle deep pagination? → Use search_after or scroll instead of from+size.

Can the number of primary shards be changed? → No, it must be set at index creation; replica count can be changed.

How to address a Yellow/Red cluster status? → Yellow: insufficient replicas; Red: primary shard loss – check disk, nodes, shard allocation.

How to implement hot‑cold separation in ES? → Tiered nodes with ILM policies.

Difference between ES and Solr? → Solr uses ZooKeeper, strong schema, suited for structured search; ES has built‑in distribution, flexible schema, richer ecosystem.

9. Practical Experience Highlights (Interview Boost)

Log Platform Case : Million‑level logs searchable in seconds using Filebeat + ES + Kibana.

Query Performance Optimization : Reduced latency by 70 % using filter cache and keyword aggregations.

Cluster Disaster‑Recovery : Configured cross‑cluster replication (CCR) for data‑center‑level resilience.

Data Tiering : Hot nodes store recent 7 days, cold nodes store archives, saving ~60 % storage cost.

Conclusion

Elasticsearch’s knowledge base is extensive, but interview focus centers on:

Fundamental principles (inverted index, write/query flow).

Architecture mechanisms (shards, replicas, NRT).

Queries and tuning (DSL, deep pagination, filter vs query).

Operations and troubleshooting (cluster health, shard allocation, JVM tuning).

Application scenarios (logging, search, monitoring).

Mastering these topics not only prepares you for most interviews but also translates into real‑world projects.