How to Build a Logback‑ELK Log Collection Server for Spring MVC

What is ELK?

ELK is an acronym for three open‑source tools: Elasticsearch (a distributed search and analytics engine), Logstash (a flexible data‑collection and processing pipeline), and Kibana (a data‑visualisation platform). Together they provide a complete solution for log ingestion, storage, and analysis.

ELK Workflow

The typical data flow is:

Application generates logs, which are processed by Logback.

Logback forwards the log events to Logstash.

Logstash parses and ships the data to Elasticsearch.

Elasticsearch stores the logs and Kibana visualises them for the user.

Environment Preparation

Required base environment:

JDK 1.8

CentOS 7 (do not run ELK services as root; create a dedicated user, e.g., elsearch)

Create a working directory and set proper permissions:

# mkdir elsearch
# groupadd elsearch
# useradd -g elsearch elsearch
# chown -R elsearch:elsearch /elsearch
# su elsearch
# cd elsearch

Download ELK Packages

Use wget to fetch the desired versions (example uses 5.2.2):

wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.2.2.tar.gz
wget https://artifacts.elastic.co/downloads/logstash/logstash-5.2.2.tar.gz
wget https://artifacts.elastic.co/downloads/kibana/kibana-5.2.2-linux-x86_64.tar.gz

Configure Elasticsearch

After extracting the archive, edit elasticsearch.yml with the following essential settings:

cluster.name: es_cluster_1
node.name: node-1
path.data: /usr/local/services/elasticsearch-5.2.2/data
path.logs: /usr/local/services/elasticsearch-5.2.2/logs
network.host: 192.168.10.200
http.port: 9200

Start Elasticsearch and verify it is reachable at http://192.168.10.200:9200/.

Configure Logstash

Extract the Logstash archive and create a minimal configuration file ( logstash-test.conf) that receives TCP input, decodes JSON, and forwards to Elasticsearch:

input {
  tcp {
    host => "192.168.10.200"
    port => 8082
    mode => "server"
    ssl_enable => false
    codec => json { charset => "UTF-8" }
  }
}
output {
  elasticsearch { hosts => "192.168.10.200:9200" index => "logstash-test" }
  stdout { codec => rubydebug {} }
}

Run Logstash with debugging enabled:

./bin/logstash -f config/logstash-test.conf --debug

Configure Kibana

Extract Kibana, then edit kibana.yml to match the Elasticsearch endpoint and expose a convenient port:

server.port: 8888
server.host: "192.168.10.200"
elasticsearch.url: "http://192.168.10.200:9200"

Start Kibana and open http://192.168.10.200:8888 in a browser.

Integrating Logback with ELK

Maven Dependencies

Add the following dependencies to your pom.xml:

<dependency>
  <groupId>net.logstash.logback</groupId>
  <artifactId>logstash-logback-encoder</artifactId>
  <version>4.11</version>
</dependency>

<dependency>
  <groupId>ch.qos.logback</groupId>
  <artifactId>logback-classic</artifactId>
  <version>1.2.3</version>
</dependency>

<dependency>
  <groupId>net.logstash.log4j</groupId>
  <artifactId>jsonevent-layout</artifactId>
  <version>1.7</version>
</dependency>

Logback Configuration (logback.xml)

Define a console appender, a rolling‑file appender, and a Logstash TCP socket appender that points to the Logstash server:

<configuration debug="false">
  <property name="LOG_HOME" value="E:/logs"/>

  <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
    <encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
      <pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{50} - %msg%n</pattern>
    </encoder>
  </appender>

  <appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
    <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
      <FileNamePattern>${LOG_HOME}/TestWeb.log_%d{yyyy-MM-dd}.log</FileNamePattern>
      <MaxHistory>30</MaxHistory>
    </rollingPolicy>
    <encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
      <pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{50} - %msg%n</pattern>
    </encoder>
    <triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
      <MaxFileSize>10MB</MaxFileSize>
    </triggeringPolicy>
  </appender>

  <appender name="stash" class="net.logstash.logback.appender.LogstashTcpSocketAppender">
    <destination>192.168.10.200:8082</destination>
    <encoder charset="UTF-8" class="net.logstash.logback.encoder.LogstashEncoder"/>
  </appender>

  <root level="INFO">
    <appender-ref ref="stash"/>
    <appender-ref ref="STDOUT"/>
    <appender-ref ref="FILE"/>
  </root>
</configuration>

Sample Spring MVC Controller

The controller logs a JSON string and returns a simple response:

package com.example.demo;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class TestEndpoints {
    private static final Logger logger = LoggerFactory.getLogger(TestEndpoints.class);

    @GetMapping("/product/{id}")
    public String getProduct(@PathVariable String id) {
        String data = "{\"name\":\"李东\"}";
        logger.info(data);
        return "product id : " + id;
    }

    @GetMapping("/order/{id}")
    public String getOrder(@PathVariable String id) {
        return "order id : " + id;
    }
}

When the endpoints are invoked, Logback forwards the JSON payload to Logstash, which stores it in Elasticsearch. The log entry becomes visible in Kibana, showing fields such as @timestamp, level, message, and the originating host.

Running the Complete Stack

Start Elasticsearch, then Kibana.

Start Logstash with the configuration file above.

Run the Spring Boot application (which includes the Logback configuration).

Invoke the REST endpoints (e.g., GET /product/123).

Open Kibana and search the logstash-test index to see the collected logs.

The resulting document in Elasticsearch looks like:

{
  "_index": "logstash-test",
  "_type": "logs",
  "_id": "AV3zu4jiJKLF9tWSjmZj",
  "_source": {
    "@timestamp": "2017-08-18T05:04:51.698Z",
    "level": "INFO",
    "host": "192.168.10.165",
    "logger_name": "com.example.demo.TestEndpoints",
    "message": "{\"name\":\"李东\"}",
    "thread_name": "http-nio-8081-exec-10",
    "port": 56525,
    "@version": 1,
    "level_value": 20000
  }
}

With the stack operational, you now have a functional log‑collection server that aggregates Java application logs and presents them in a searchable, visual interface.