Logstash 9.x vs Earlier Versions: Key Differences, Common Errors, and Fixes

1. Working Logstash 9.x Configuration

The author provides a complete app_logs.conf file that reads log files, parses them with a grok filter, and outputs to Elasticsearch using SSL. Running this configuration on a 9.0 installation writes data successfully, as shown by the console output and a screenshot of Kibana confirming the indexed logs.

[logstash@VM-8-2-centos config]$ cat app_logs.conf
input {
  file {
    path => "/var/log/app/*.log"
    start_position => "beginning"
  }
}
filter {
  grok {
    match => { "message" => "%{TIMESTAMP_ISO8601:timestamp} %{LOGLEVEL:loglevel} %{GREEDYDATA:message}" }
  }
}
output {
  elasticsearch {
    hosts => ["https://10.0.8.2:9200"]
    index => "app-logs-%{+YYYY.MM.dd}"
    user => "elastic"
    password => "xxxxx"
    ssl_enabled => true
    ssl_certificate_authorities => ["/elkstack/elasticsearch-9.0.0/config/certs/http_ca.crt"]
  }
}

2. Common Issues in Logstash 9.x and Their Solutions

2.1 Running as Root Is Disallowed

When Logstash is started as the root user, it aborts with the error Logstash cannot be run as superuser.. Earlier versions (7.x) allowed this, but 9.x enforces the restriction. The recommended solution is to create a non‑privileged user (e.g., logstash) and run Logstash under that account:

sudo useradd -m -s /bin/bash logstash
sudo passwd logstash

2.2 Elasticsearch Output Parameter Change

In 7.x the cacert setting was used to specify the CA certificate. Starting with 8.x and continuing in 9.x, cacert is obsolete; the correct parameter is ssl_certificate_authorities. Attempting to use cacert in 9.x produces an error like:

[ERROR][logstash.agent] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"Java::JavaLang::IllegalStateException", :message=>"Unable to configure plugins: (ConfigurationError) The setting cacert in plugin elasticsearch is obsolete and is no longer available. Set 'ssl_certificate_authorities' instead..."}

The official documentation confirms the new parameter and its compatibility.

2.3 Data Not Written to Elasticsearch

Even when Logstash starts without fatal errors, data may not appear in Elasticsearch. The author identifies three typical causes:

Insufficient file or certificate permissions.

Incorrect Elasticsearch credentials or missing ssl_certificate_authorities configuration.

Stale .sincedb files preventing file input from re‑reading logs.

Solutions include verifying the app_logs.conf syntax, checking the Elastic user’s permissions via curl, and clearing the sincedb files:

curl -u elastic:your_secure_password -X GET "https://localhost:9200/_security/user/elastic" --cacert /elkstack/elasticsearch-9.0.0/config/certs/http_ca.crt
rm /elkstack/logstash-9.0.0/data/plugins/inputs/file/.sincedb_*

After fixing certificate file permissions, data is successfully indexed.

3. Quick Configuration Validation with --config.test_and_exit

The flag --config.test_and_exit (or short -t) checks the syntax and plugin parameters of a Logstash configuration without starting the pipeline. It reports [INFO] Configuration OK on success or detailed error messages with line numbers on failure. This is useful for automated deployment scripts, but it does not verify runtime connectivity (e.g., Elasticsearch reachability) or actual data processing.

4. Summary

Logstash 9.x introduces stricter security and configuration requirements compared with 7.x, notably the prohibition of root execution and the replacement of cacert with ssl_certificate_authorities. Common pitfalls include permission problems, outdated parameters, and stale sincedb files. The author recommends:

Run Logstash under a dedicated non‑root user.

Validate configurations with --config.test_and_exit before deployment.

Ensure SSL settings use the new ssl_certificate_authorities parameter and that certificate files have correct permissions.

Check Elasticsearch credentials and clear sincedb files when data does not appear.