Unlock Efficient Search and Storage with Elasticsearch Index Mapping

1. Mapping Basics

In Elasticsearch, a mapping defines the schema of an index, similar to a table definition in a relational database. It describes each field’s type, how it is indexed, and how queries are processed. Every index has an associated mapping type.

Mapping definitions include field data types such as text, keyword, integer, date, etc., which determine indexing and search behavior. Additional settings can control whether the original value is stored, whether doc values are created for sorting and aggregations, and so on.

2. Key Field Types and Their Uses

2.1 text

Purpose: Full‑text search; field is analyzed into terms and an inverted index is built.

Characteristics: Processed by an analyzer before indexing, enabling relevance scoring.

2.2 keyword

Purpose: Exact‑value search, sorting, aggregations, and scripting.

Characteristics: Not analyzed; the whole value is indexed as a single term.

2.3 Numeric types (integer, long, float, double)

Purpose: Store numbers such as price, quantity, rating.

Characteristics: Support range queries, sorting, and aggregations; stored without analysis.

2.4 date

Purpose: Store date and time values.

Characteristics: Accept multiple formats, convert to internal UTC epoch milliseconds, support range queries and time‑based aggregations.

2.5 boolean

Purpose: Store true/false values, often used for filtering.

Characteristics: Indexed as a single term; can be used in term queries.

2.6 geo (geo_point, geo_shape)

Purpose: Store geographic coordinates or shapes.

Characteristics: Enable distance calculations and shape queries, typically combined with map visualizations.

2.7 nested

Purpose: Store arrays of JSON objects while preserving independence of each object.

Characteristics: Allows precise queries and aggregations on nested objects.

3. Index Options

3.1 index

Controls whether a field is searchable. Default is true.

3.2 store

If true, the field’s original value is stored separately from _source, improving retrieval speed at the cost of extra storage.

3.3 doc_values

Column‑oriented on‑disk copy of field values used for sorting, aggregations, and scripting. Enabled by default for most types except text.

3.4 fielddata

In‑memory structure for sorting/aggregating text fields. Disabled by default because it can consume large memory; use a keyword sub‑field instead.

3.5 norms

Stores length normalization and term weight for relevance scoring. Disabling saves space but removes TF/length‑based scoring.

3.6 analyzer / search_analyzer

Define the analyzer used at index time and optionally a different one at search time. If search_analyzer is omitted, the index analyzer is reused.

3.7 null_value

Provides a default value for missing or null fields, ensuring consistent queries and aggregations. The value must match the field’s type.

3.8 format

Specifies date formats for date fields; required when the default format does not match the data.

3.9 ignore_above

For keyword fields, prevents indexing values longer than the specified character count.

3.10 eager_global_ordinals

When true, global ordinals are built at refresh time, speeding up aggregations at the expense of memory and refresh latency.

3.11 meta

Allows arbitrary metadata to be attached to a field; not used for indexing or searching.

3.12 copy_to

Copies the field’s content into other fields, useful for creating combined searchable fields without changing query logic.

4. Multi‑Fields

Multi‑fields let a single logical field be indexed in multiple ways, e.g., as both text for full‑text search and keyword for exact matching. They enable different analyzers, data types, language support, or custom search logic without altering the original document.

Multi‑fields do not increase the number of fields in the source document; they only generate additional index entries.

5. Meta Fields

Elasticsearch provides special meta fields such as _source and _field_names. The _all field was deprecated in 7.x; use multi_match queries for cross‑field search.

6. Dynamic Mapping

When a document contains fields not defined in the mapping, Elasticsearch infers their types automatically. This flexibility is convenient for development but should be disabled or tightly controlled in production.

7. Analyzers and Normalizers

For text fields, an analyzer breaks the input into terms. Elasticsearch offers many built‑in analyzers and supports custom ones. Normalizers apply similar processing to keyword fields (e.g., lower‑casing).

8. Settings and Optimization Recommendations

Define field types explicitly to avoid the uncertainty of dynamic mapping and improve query performance.

Leverage multi‑fields to support both full‑text and exact‑value queries on the same data.

Choose appropriate analyzers and normalizers based on the data and query patterns.

Monitor index performance with Elasticsearch’s monitoring tools and adjust mappings or queries when bottlenecks appear.

PUT /my_index
{
  "settings": {
    "index": {
      "number_of_shards": 1,
      "number_of_replicas": 1
    },
    "analysis": {
      "analyzer": {
        "my_custom_analyzer": {
          "type": "standard",
          "stopwords": ["and", "the"]
        }
      }
    }
  },
  "mappings": {
    "properties": {
      "title": {
        "type": "text",
        "analyzer": "my_custom_analyzer",
        "search_analyzer": "standard",
        "fields": {
          "keyword": {
            "type": "keyword"
          }
        }
      },
      "content": {
        "type": "text",
        "fielddata": true,
        "fields": {
          "raw": {
            "type": "keyword"
          }
        }
      },
      "date_published": {
        "type": "date",
        "format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis"
      },
      "is_published": {
        "type": "boolean"
      },
      "price": {
        "type": "float"
      },
      "stock_count": {
        "type": "integer",
        "doc_values": false,
        "ignore_above": 1000
      },
      "product_tags": {
        "type": "keyword",
        "norms": false,
        "eager_global_ordinals": true
      },
      "description": {
        "type": "text",
        "index": false,
        "store": true
      },
      "meta_data": {
        "type": "object",
        "enabled": false
      },
      "all_fields": {
        "type": "text",
        "copy_to": ["title", "content"]
      }
    }
  }
}

PUT /my_index
{
  "mappings": {
    "properties": {
      "user_age": {
        "type": "integer",
        "null_value": -1
      }
    }
  }
}
POST /my_index/_doc/1
{
  "user_age": 30
}
POST /my_index/_doc/2
{
  "user_age": null
}
GET /my_index/_search
{
  "query": { "match_all": {} }
}