Searchable Snapshots in Elasticsearch 7.14: Principles and Practical Implementation

This article introduces the searchable snapshot feature of Elasticsearch, which became production‑ready in version 7.14. It explains the underlying principles and provides a step‑by‑step practical guide using Tencent Cloud COS as the snapshot repository.

1. Searchable Snapshot Overview – Searchable snapshots allow direct queries on data stored in snapshots, typically kept on cheap object storage such as COS. By mounting snapshots on Cold or Frozen tier nodes, storage costs are dramatically reduced.

2. DataTier Model – Starting with ES 7.10, nodes are classified into tiers: Content, Hot, Warm, Cold, and Frozen. Each tier has specific workloads and hardware recommendations (e.g., Hot tier uses SSD, Cold/Frozen use HDD).

3. Node Role Optimization – Node roles are now defined via the node.roles array. Example configurations:

node.roles: ["data_hot", "data_content", "ingest", "ml", "remote_cluster_client", "transform"]

node.roles: ["master"]

node.roles: ["data_frozen"]

Dedicated Frozen nodes should have only the data_frozen role.

4. Index Allocation with _tier_preference – Allocation settings now use index.routing.allocation.include._tier_preference to control which tier a shard should reside on. Example value: data_hot,data_warm,data_cold .

5. Mount Types

Fully mounted index – the entire snapshot data is copied to the cluster; provides normal query performance and automatic recovery from corrupted shards.

Partially mounted index – only metadata is stored on the node; actual data remains in the snapshot and is fetched on demand. This is the key cost‑saving mode for Frozen tier.

Example API for partial mount:

POST /_snapshot/default_searchable_snapshot_repository/my_cos_snapshot/_mount?wait_for_completion=true&storage=shared_cache

{
  "index": "wurong-test-2021.11.26-000001",
  "renamed_index": "wurong-test-2021.11.26-000001_from_cos",
  "index_settings": {"index.number_of_replicas": 0},
  "ignore_index_settings": ["index.refresh_interval"]
}

6. Practical Setup

• Activate the Enterprise license:

POST /_license/start_trial?acknowledge=true

• Create a COS snapshot repository:

PUT _snapshot/default_searchable_snapshot_repository
{
  "type": "cos",
  "settings": {
    "app_id": "1254139681",
    "access_key_id": "xxxx",
    "access_key_secret": "xxxx",
    "bucket": "wurong-es-snapshpt",
    "region": "ap-guangzhou",
    "compress": true,
    "chunk_size": "500mb",
    "base_path": "/searchable-snapshot"
  }
}

• Define an ILM policy that rolls over after 20 minutes or 10 MB, then moves indices through Hot → Cold → Frozen tiers, applying searchable snapshot actions at each phase.

PUT _ilm/policy/searchable_snapshot_test_policy
{
  "policy": {
    "phases": {
      "hot": {"actions": {"rollover": {"max_size": "100mb", "max_primary_shard_size": "10mb", "max_age": "20m", "max_docs": 1000000}, "set_priority": {"priority": 100}}},
      "cold": {"min_age": "1h", "actions": {"searchable_snapshot": {"snapshot_repository": "default_searchable_snapshot_repository", "force_merge_index": true}, "set_priority": {"priority": 0}, "allocate": {"number_of_replicas": 0}} ,
      "frozen": {"min_age": "2h", "actions": {"searchable_snapshot": {"snapshot_repository": "default_searchable_snapshot_repository", "force_merge_index": true}}}
    }
  }
}

• Create an index template linking the ILM policy, rollover alias, and tier preference:

PUT _template/searchable_snapshot_test_template
{
  "order": 100,
  "index_patterns": ["wurong-test*"],
  "settings": {
    "index": {
      "lifecycle": {"name": "searchable_snapshot_test_policy", "rollover_alias": "wurong-test"},
      "routing": {"allocation": {"include": {"_tier_preference": "data_hot,data_warm,data_cold"}},
      "refresh_interval": "30s",
      "number_of_shards": "5",
      "translog": {"sync_interval": "5s", "durability": "async"},
      "max_result_window": "65536",
      "unassigned": {"node_left": {"delayed_timeout": "5m"}},
      "number_of_replicas": "1"
    }
  }
}

• Create the initial index with a date‑based name and set the alias as the write index.

PUT %3Cwurong-test-%7Bnow%2Fd%7D-000001%3E
{
  "aliases": {"wurong-test": {"is_write_index": true}}
}

Data is then continuously written by a load‑testing program. After the rollover and ILM phases trigger, indices are automatically mounted as fully or partially searchable snapshots, moved to Cold or Frozen tiers, and their storage usage reflects the mounting mode (full copies consume space, partial copies show near‑zero storage).

The article concludes that the described workflow demonstrates the technical principle and end‑to‑end configuration of searchable snapshots on Tencent Cloud Elasticsearch, providing a cost‑effective solution for large‑scale data.