Infinilabs Gateway Config File: A Layer‑by‑Layer Deep Dive

Overview

The Infinilabs Gateway is a high‑performance, extensible gateway used in microservice, distributed, and big‑data scenarios to handle traffic routing, access control, logging, metrics, and backend data aggregation. This guide dissects a complete gateway configuration file from top to bottom, providing a clear learning path for complex setups.

1. Environment Variables (env)

env:
  LOGGING_ES_ENDPOINT: https://localhost:9200/
  LOGGING_ES_USER: elastic
  LOGGING_ES_PASS: changeme
  PROD_ES_ENDPOINT: https://localhost:9200/
  PROD_ES_USER: elastic
  PROD_ES_PASS: changeme
  GW_BINDING: "0.0.0.0:8000"
  API_BINDING: "0.0.0.0:2900"

These variables define runtime values such as Elasticsearch endpoints, credentials, and the IP/port bindings for the gateway and its management API. They can be referenced elsewhere in the file with $[[env.VAR_NAME]], enabling portable and overridable configurations.

2. Basic Paths and Global Settings

path.data: data
path.logs: log
path.configs: config
configs.auto_reload: false

path.data – directory for persistent data (Badger KV, queue files, etc.).

path.logs – directory for log files.

path.configs – directory for additional configuration files.

configs.auto_reload – whether the gateway reloads the config automatically; typically set to false in production to avoid runtime instability.

3. Gateway Core Features

3.1 Gateway Layer (gateway)

gateway:
  disable_reuse_port_by_default: true

disable_reuse_port_by_default controls the SO_REUSEPORT option. If the underlying Linux kernel does not support or the feature is not needed, setting it to true disables port reuse to prevent conflicts.

3.2 Statistics (stats) and StatsD (statsd)

stats:
  enabled: true
  persist: true
  no_buffer: true
  buffer_size: 1000
  flush_interval_ms: 1000

statsd:
  enabled: false
  ...

stats.enabled – turns on internal request/latency/error counting.

persist – persists statistics across restarts.

no_buffer – writes statistics directly without buffering.

statsd.enabled – when true, forwards metrics to an external StatsD service (host, port, namespace, etc.). In this example it is disabled.

3.3 System API (api)

api:
  enabled: true
  network:
    binding: $[[env.API_BINDING]]
  security:
    enabled: false
    username: admin
    password: $[[keystore.API_PASS]]

api.enabled – exposes the management API at the address defined by API_BINDING.

security.enabled – toggles authentication for the API; when enabled, username and password (or a keystore entry) protect access.

4. Elasticsearch Service Definitions (elasticsearch)

elasticsearch:
  - name: prod
    enabled: true
    endpoints:
      - $[[env.PROD_ES_ENDPOINT]]
    discovery:
      enabled: false
    basic_auth:
      username: $[[env.PROD_ES_USER]]
      password: $[[env.PROD_ES_PASS]]
    traffic_control.max_bytes_per_node: 1010485760
    metadata_cache_enabled: false

  - name: logging-server
    enabled: true
    endpoints:
      - $[[env.LOGGING_ES_ENDPOINT]]
    basic_auth:
      username: $[[env.LOGGING_ES_USER]]
      password: $[[env.LOGGING_ES_PASS]]
    discovery:
      enabled: false

name distinguishes different clusters (e.g., prod for the production cluster, logging-server for log collection).

endpoints – list of Elasticsearch node URLs.

discovery.enabled – whether automatic node discovery is used.

basic_auth – credentials for clusters that require authentication.

traffic_control.max_bytes_per_node – per‑node request‑byte limit.

metadata_cache_enabled – enables caching of index mappings to improve performance.

5. Entry, Router, and Flow (entry, router, flow)

5.1 Entry (entry)

entry:
  - name: my_es_entry
    enabled: true
    router: my_router
    max_concurrency: 10000
    network:
      binding: $[[env.GW_BINDING]]
      reuse_port: true
    tls:
      enabled: true

entry.name – identifier for the listening instance (e.g., my_es_entry).

router – selects which router configuration handles incoming requests.

max_concurrency – maximum concurrent connections.

network.binding – address/port taken from GW_BINDING.

tls.enabled – enables TLS with custom certificates or built‑in generation.

5.2 Router (router)

router:
  - name: my_router
    default_flow: default_flow
    tracing_flow: logging_flow
    rules:
      - method:
          - "*"
        pattern:
          - "/_bulk"
          - "/{any_index}/_bulk"
        flow:
          - async_bulk_flow

router.name – router identifier.

default_flow – fallback flow when no rule matches.

tracing_flow – optional flow for request tracing or special handling.

rules – match HTTP methods and URL patterns; in this example all methods matching /_bulk or /{any_index}/_bulk are directed to async_bulk_flow.

5.3 Flow (flow)

flow:
  - name: default_flow
    filter:
      - basic_auth:
          valid_users:
            $[[env.PROD_ES_USER]]: $[[env.PROD_ES_PASS]]
      - elasticsearch:
          elasticsearch: prod
          max_connection_per_node: 1000

  - name: logging_flow
    filter:
      - logging:
          queue_name: request_logging
          max_request_body_size: 4096
          max_response_body_size: 4096

  - name: async_bulk_flow
    filter:
      - bulk_reshuffle:
          when:
            contains:
              _ctx.request.path: /_bulk
          elasticsearch: prod
          level: node
          partition_size: 10
          continue_metadata_missing: true
          fix_null_id: true
      - elasticsearch:
          elasticsearch: prod
          max_connection_per_node: 1000

flow.name – identifier of the processing pipeline.

filter – ordered list of filters applied to requests. basic_auth – enforces authentication using the provided user/password. elasticsearch – forwards the request to the specified cluster with a connection limit. logging – writes request information to the request_logging queue for later indexing. bulk_reshuffle – splits bulk /_bulk operations into partitions (size 10) to increase parallelism; continues even if metadata is missing and generates IDs for documents lacking them.

Combining entry , router , and flow defines the complete request‑handling path from reception to specific processing logic.

6. Pipelines (pipeline)

pipeline:
  - name: pipeline_logging_merge
    auto_start: true
    keep_running: true
    processor:
      - indexing_merge:
          input_queue: "logging"
          idle_timeout_in_seconds: 1
          elasticsearch: "logging-server"
          index_name: ".infini_logs"
          output_queue:
            name: "gateway-pipeline-logs"
            label:
              tag: "pipeline_logging"
          worker_size: 1
          bulk_size_in_kb: 1

The pipeline defines background jobs that consume messages from queues, merge them, and write the result to Elasticsearch. Parameters such as idle_timeout_in_seconds control how long the processor waits before forcing a commit when the batch size is insufficient.

7. Monitoring (metrics)

metrics:
  enabled: true
  queue: metrics
  logging_queue: logging
  instance:
    enabled: true
  network:
    enabled: true
    summary: true
    sockets: true

metrics.enabled – activates internal monitoring.

queue / logging_queue – designate queues for metric and log data.

instance.enabled – collects instance‑level stats (CPU, memory).

network.enabled – gathers network‑level throughput and connection metrics.

8. Disk Queue (disk_queue)

disk_queue:
  prepare_files_to_read: true
  eof_retry_delay_in_ms: 500
  cleanup_files_on_init: false
  retention:
    max_num_of_local_files: 20
  compress:
    segment:
      enabled: true
      delete_after_compress: true
      idle_threshold: 20

prepare_files_to_read – pre‑creates queue files for reading.

cleanup_files_on_init – removes leftover files on startup.

retention.max_num_of_local_files – caps the number of persisted queue files.

compress.segment.enabled – compresses completed queue files; delete_after_compress removes the original files.

idle_threshold – triggers additional cleanup when the specified number of files have been consumed and compressed.

9. Badger KV Store (badger)

badger:
  enabled: true
  single_bucket_mode: true
  path: ''
  memory_mode: false
  sync_writes: false

Badger – an embedded key‑value store used for caching metadata, tokens, sessions, etc.

single_bucket_mode – uses a single bucket to simplify management.

path – storage directory; if empty, defaults to path.data/gateway/node/{nodeID}/badger/.

memory_mode – when true, stores data only in memory.

sync_writes – when true, each write is flushed to disk for durability at the cost of performance.

10. Floating IP (floating_ip)

floating_ip:
  enabled: false
  netmask: 255.255.255.0
  ...

floating_ip.enabled – enables high‑availability IP failover; disabled in this example.

netmask , ip , interface – configure the virtual IP, subnet, and network interface for different environments.

priority – higher numbers give a node higher takeover priority.

11. Global Elasticsearch Module Settings (elastic)

elastic:
  elasticsearch: prod
  enabled: true
  remote_configs: false
  health_check:
    enabled: true
    interval: 30s
  availability_check:
    enabled: true
    interval: 30s
  metadata_refresh:
    enabled: true
    interval: 60s
  cluster_settings_check:
    enabled: false
    interval: 60s

elasticsearch – sets the default ES service (here prod).

health_check , availability_check , metadata_refresh – periodic checks for service health, availability, and metadata updates.

cluster_settings_check – can verify cluster configuration; disabled in this example.

Conclusion

The configuration demonstrates a highly modular and extensible design. At the macro level, environment variables provide one‑click switching between deployments, while core sections ( gateway, stats, api) define the gateway’s fundamental behavior. The mid‑level layers ( elasticsearch, entry, router, flow) specify backend services, routing rules, and processing pipelines. Finally, the micro‑level components ( disk_queue, badger, floating_ip, elastic) handle persistence, KV storage, high‑availability, and global health checks.

By following this hierarchical approach—starting from high‑level goals, drilling down to module configurations, and finally adjusting low‑level storage and network details—users can modify the gateway safely while preserving controllability and traceability.