Visualizing Elasticsearch Request Traffic with Infini Gateway: A Hands‑On Guide

Background and Goal

In modern cloud architectures, monitoring traffic between services—especially Elasticsearch requests—is critical. Infini Gateway (a lightweight, high‑performance gateway) can record and analyze this traffic, providing data for operations and business optimization.

1. Installing and Configuring Infini Gateway

Environment Configuration

The gateway reads settings from gateway.yml. After extracting the package, edit the following entries:

env:
  LOGGING_ES_ENDPOINT: http://localhost:9200
  LOGGING_ES_USER: elastic
  LOGGING_ES_PASS: password
  PROD_ES_ENDPOINT: http://localhost:9200
  PROD_ES_USER: elastic
  PROD_ES_PASS: password
  GW_BINDING: "0.0.0.0:8000"
  API_BINDING: "0.0.0.0:2900"

LOGGING_ES_ENDPOINT : address of the Elasticsearch cluster where request logs are stored.

PROD_ES_ENDPOINT : address of the production cluster that the gateway proxies.

GW_BINDING : listening address and port for the gateway proxy service.

API_BINDING : listening address and port for the gateway API.

You can adjust these values for development, testing, or production environments.

Enabling Cache

To reduce repeated requests to Elasticsearch, add set_cache and get_cache to the default_flow configuration:

- name: default_flow
  filter:
    - get_cache:
    - elasticsearch:
        elasticsearch: prod
        max_connection_per_node: 1000
    - set_cache:

This enables caching within the gateway, improving query performance.

Optional: INFINI Easysearch

If you prefer using INFINI Easysearch for log storage, install the index-management plugin:

bin/easysearch-plugin install index-management

Restart the gateway after installation. Note: when Elasticsearch is used directly for log storage, this plugin is unnecessary.

Creating an Index Template

Define a template for log indices to set lifecycle policies and field mappings:

curl -X PUT "localhost:9200/_template/logging_template" -H 'Content-Type: application/json' -d'
{
  "template": {
    "settings": {
      "index.lifecycle.name": "logging_policy"
    },
    "mappings": {
      "_doc": {
        "properties": {
          "timestamp": {"type": "date"},
          "request_duration": {"type": "float"}
        }
      }
    }
  }
}'

The template ensures efficient storage and retrieval of request logs.

2. Index Lifecycle Management (ILM)

Configure an ILM policy to automatically roll over, merge, and delete indices based on traffic volume, preventing index bloat and preserving query performance.

3. Importing the Kibana Dashboard

Download the dashboard file INFINI-Gateway-7.9.2-2021-01-15.ndjson.zip and import it via Kibana’s Saved Objects → Import UI. This provides pre‑built visualizations for Elasticsearch request metrics.

4. Starting the Gateway

./bin/gateway

On startup the gateway logs information such as:

[GATEWAY] A light-weight, powerful and high-performance elasticsearch gateway.
[GATEWAY] 1.0.0_SNAPSHOT, Wed Feb 3 00:12:02 2021 +0800, medcl
[GATEWAY] api server listen at: http://0.0.0.0:2900
[GATEWAY] entry [es_gateway] listen at: http://0.0.0.0:8000

At this point the gateway begins listening for traffic.

5. Updating Application Configuration

Redirect applications (Kibana, Logstash, Beats, etc.) to the gateway address. Example Kibana configuration:

elasticsearch.hosts: ["https://192.168.3.98:8000"]
elasticsearch.customHeaders: {"app": "kibana"}

Restart Kibana after saving the changes.

6. Result and Visualization

After completing the steps, all Elasticsearch requests routed through the gateway are logged, stored, and visualized in Kibana. Users can create custom dashboards showing traffic volume, request latency, success rates, and other key performance indicators.

Conclusion

Using Infini Gateway for traffic analysis and log collection provides clear insight into Elasticsearch request patterns, enables real‑time system monitoring, helps locate performance bottlenecks, and supports business process optimization.