Build a Low‑Cost Observability Platform with OpenObserve and Vector

1. Introduction

OpenObserve is a Rust‑based cloud‑native observability platform that can handle petabyte‑scale data for logs, metrics, traces, and real‑user monitoring. Compared with Elasticsearch it is simpler, cheaper (storage cost reduced ~140×), and can be deployed in minutes.

It supports comprehensive data types, is fully compatible with OpenTelemetry, offers advanced query capabilities (SQL and PromQL), and includes a built‑in UI, eliminating the need for separate installations.

GitHub: https://github.com/openobserve/openobserve

Official site: https://openobserve.ai

2. Vector

Vector is a high‑performance observability data pipeline written in Rust. It collects, transforms, and routes logs, metrics, and traces to any backend, reducing cost, enriching data, and ensuring security. It is open source and up to 10× faster than many alternatives.

GitHub: https://github.com/vectordotdev/vector

3. Architecture

4. Deployment

4.1 Deploy OpenObserve

Create a deployment directory: mkdir -p /opt/openobserve && cd /opt/openobserve Create docker-compose.yaml:

version: '3.8'
services:
  openobserve:
    image: public.ecr.aws/zinclabs/openobserve:v0.15.0
    container_name: openobserve
    hostname: openobserve
    restart: always
    environment:
      - [email protected]
      - ZO_ROOT_USER_PASSWORD=Compl...#123
      - ZO_DATA_DIR=/data
    ports:
      - "5080:5080"
      - "4317:4317"
    volumes:
      - $PWD/openobserve_data:/data
    networks:
      - openobserve_net
networks:
  openobserve_net:
    driver: bridge

Start the service: docker-compose up -d Verify the deployment: curl http://localhost:5080/health Expected output: {"status":"ok"} Access the web UI at http://<server_ip>:5080 using the admin email and password defined above.

4.2 Configure Vector

Adjust the sink configuration to send data to OpenObserve. Example TOML snippet:

[sinks.openobserve]
type = "http"
inputs = [ "source_or_transform_id" ]
uri = "http://192.168.31.103:5080/api/default/default/_json"
method = "post"
auth.strategy = "basic"
auth.user = "[email protected]"
auth.password = "GJomtkRezK1m3QcY"
compression = "gzip"
encoding.codec = "json"
encoding.timestamp_format = "rfc3339"
healthcheck.enabled = false

The equivalent YAML representation is also provided in the original article.

5. Usage

The UI shows the home page, data‑source page, log view, frontend monitoring, and alerts (screenshots omitted for brevity).

5.1 Trace Integration

OpenObserve supports OpenTelemetry tracing. Below is a Python example that sends traces to OpenObserve.

git clone https://github.com/openobserve/sample-tracing-python.git
cd sample-tracing-python
vim tracing.py

from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.resources import SERVICE_NAME, Resource
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter

resource = Resource(attributes={SERVICE_NAME: "python-service"})
tracer_provider = TracerProvider(resource=resource)
url = 'http://localhost:5080/api/default/v1/traces'
headers = {"Authorization": "Basic cm9vdEBleGFtcGxlLmNvbTpDb21wbGV4cGFzcyMxMjM="}
exporter = OTLPSpanExporter(endpoint=url, headers=headers)
span_processor = BatchSpanProcessor(exporter)
tracer_provider.add_span_processor(span_processor)
trace.set_tracer_provider(tracer_provider)

Install dependencies and run the sample:

pip3 install -r requirements.txt
python3 app.py

OpenTelemetry Collector can also forward traces to OpenObserve via otlphttp or otlp exporters. Example collector snippet:

exporters:
  otlphttp/openobserve:
    endpoint: http://192.168.31.103:5080/api/default
    headers:
      Authorization: Basic YWRtaW5AZWFzeW9wcy5vbmxpbmU6R0pvbXRrUmV6SzFtM1FjWQ==
    stream-name: default
    tls:
      insecure: true

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [otlphttp/openobserve]

6. FAQ

How to change the default stream name: modify the Vector sink URL to use

http://<em>tenant</em>:5080/api/<em>tenant</em>/<em>stream_name</em>/_json

. The first segment after the host is the tenant name; the second is the stream name. Adjust authentication accordingly.

Additional configuration examples for custom streams (e.g., k8s-pod-logs, k8s-node-logs) are provided in the original article.

For further details, refer to the official OpenObserve documentation.

Done. Hope this helps! If you find it useful, please like and follow the author.