Add Observability to Legacy Java Apps with OpenTelemetry Agent (Zero Code)

Introduction: The Untouchable Legacy System

Every enterprise has a legacy system that has been running for decades, evolving from COBOL to Java, constantly patched but never daringly changed. It is stable yet closed, usable yet invisible, and when problems arise you cannot see whether the system is stuck, which code segment is slow, or if there is a memory leak.

OpenTelemetry (OTel) can illuminate this "black box" with just a few commands and no code changes.

What is opentelemetry-javaagent?

If you need observability for an old system without modifying source code, the OpenTelemetry Java Agent (otel-javaagent) is your secret weapon.

It is an auto‑instrumentation agent delivered as a .jar file, e.g.: opentelemetry-javaagent.jar When you start a Java program, add the following JVM argument: -javaagent:./opentelemetry-javaagent.jar The agent automatically injects monitoring logic at runtime, capturing performance metrics, trace data, and exception information, and sends them to an OpenTelemetry Collector.

Without changing source code you can immediately see:

Program health metrics (CPU, memory, GC)

Request latency and trace paths

Method‑level performance bottlenecks

Error and exception statistics

Why is it ideal for legacy systems?

No recompilation required

No impact on business logic

Can be toggled on or off at any time

Negligible performance overhead (typically <3%)

For systems no one dares to touch, this is often the only safe option.

Stage 1: Basic Health Monitoring – Is the System Still Alive?

Goal

Quickly understand the runtime health of the system without any source‑code changes.

Environment Configuration

Use the opentelemetry-javaagent to add monitoring to the JVM by setting environment variables:

# — Basic system health configuration —
export OTEL_SERVICE_NAME=legacy-part-processor
export OTEL_METRICS_EXPORTER=otlp
export OTEL_EXPORTER_OTLP_ENDPOINT=http://127.0.0.1:4317
export OTEL_EXPORTER_OTLP_PROTOCOL=grpc
export OTEL_INSTRUMENTATION_RUNTIME_TELEMETRY_JAVA8_ENABLED=true

# Attach the agent to JVM startup options
export _JAVA_OPTIONS="-javaagent:./opentelemetry-javaagent.jar"

After setting these, run your legacy application as usual: ./legacy_app The original log output remains visible, confirming the application still runs while the OTel agent collects data in the background.

Data Flow

OpenTelemetry Agent → Collector → Prometheus → Grafana

Create basic Grafana dashboards showing average CPU usage, memory consumption, and JVM GC pause times. This lets you answer critical questions such as whether there is a memory leak, abnormal CPU load, or GC frequency causing latency.

At this point the black‑box system has at least its vital signs visible.

Stage 2: Performance Monitoring – How Fast Does It Run?

Beyond health metrics, you need to know business processing performance and potential bottlenecks. Assume the core logic resides in the processData() method.

Add the following environment variables to enable tracing for that method:

export OTEL_TRACES_EXPORTER=otlp
export OTEL_INSTRUMENTATION_METHODS_INCLUDE="LegacyJavaProcessor[processData]"

This automatically creates spans for the processData() method in the LegacyJavaProcessor class. After restarting the application, trace data is sent to the Collector, converted to metrics via spanmetrics, and displayed in Grafana as throughput (calls per minute), latency (processing time), and error rate (errors per minute).

Conclusion: First Step to Making a Black Box Transparent

Even a small amount of visibility is far better than complete blindness.

By simply introducing the opentelemetry-javaagent, you achieve:

Zero code changes

Rapid visualization of system health

Automatic tracing of performance bottlenecks

Reduced operational risk and data collection for future refactoring

Your old system may be ancient, but it doesn’t have to remain a black box—take this step and start lighting it up.

Final Thoughts

OpenTelemetry is not limited to cloud‑native environments; it can also help us understand, fix, and improve seemingly untouchable legacy systems, turning observability from a “new‑system standard” into a lifeline for legacy applications.