How to Extend SAE with Sidecar Containers for Custom Logging and Monitoring

Background

SAE (Serverless Application Engine) is a fully managed PaaS that abstracts Kubernetes complexities, allowing developers to focus on business logic while SAE handles lifecycle, microservice management, logging, and monitoring.

User Requirements

Customers increasingly need observability and operational capabilities without changing their application code. Specific demands include:

Free log collection: real‑time export of container logs to external systems such as Elasticsearch or Loki, e.g., using Filebeat to send logs to Kafka.

Custom metric collection: capture application‑specific performance metrics and push them to monitoring platforms like Prometheus.

Resource isolation: prevent resource contention when multiple processes run in a single container.

Solution Overview

SAE introduces sidecar containers, allowing one or more auxiliary containers to run alongside the main application container. Sidecars handle custom log collection, metric gathering, and other extensions while keeping the main container unchanged.

Application Multi‑Container Resource Allocation

In Kubernetes, each container can define resource request and limit. Adding a sidecar creates a resource‑sharing challenge, which SAE addresses with two allocation modes:

Shared Resource Mode

Configuration: sidecar request=0 and limit>0.

Characteristics: sidecar shares pod resources but cannot exceed its limit, protecting the main container from excessive consumption.

Suitable for resource‑sensitive workloads that aim to maximize utilization.

Independent Resource Mode

Configuration: sidecar request=limit.

Characteristics: sidecar reserves a fixed amount of CPU/memory, guaranteeing isolation (e.g., main container 1c2g, sidecar 0.5c0.5g leaves 0.5c1.5g for the main app).

Ideal for scenarios demanding strict resource isolation.

Operational Features for Sidecar Management

SAE provides a full‑stack operational suite for sidecars, including:

Sidecar status view (Pending, Running, CrashLoopBackOff) in the console.

Log access directly from the console.

WebShell access to execute commands inside the sidecar.

CPU and memory monitoring (currently in gray‑release).

Event subscription for sidecar restarts via the Event Center.

Instance‑level sidecar restart without pod recreation. saectl tool for Kubernetes‑level sidecar resource management, similar to kubectl.

Stability Guarantees

Isolation: sidecar failures do not affect main container traffic.

Auto‑recovery: SAE automatically restarts failed sidecars.

Resource limits: sidecar limits prevent it from starving the main container.

Case Study: Implementing Custom Log Collection

Scenario: Users want to stream SAE application logs to a self‑hosted log system using Filebeat and Kafka.

Step 1 – Add a Sidecar Container

In the SAE console, enable the "Add Sidecar" tab and configure:

Container name (e.g., filebeat).

Image:

swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/elastic/filebeat:8.15.3

.

Resource limits to protect the main app.

Step 2 – Configure Sidecar

Set the startup command to ./filebeat -e -c filebeat.kafka.yml and mount a ConfigMap containing filebeat.kafka.yml at /usr/share/filebeat/filebeat.kafka.yml. Example configuration:

filebeat.inputs:</code>
<code>  - type: log</code>
<code>    enabled: true</code>
<code>    paths:</code>
<code>      - /var/log/*.log</code>
<code>output.kafka:</code>
<code>  hosts: ["kafka1:9092", "kafka2:9092", "kafka3:9092"]</code>
<code>  topic: "topic"</code>
<code>  partition.round_robin:</code>
<code>    reachable_only: true</code>
<code>  required_acks: 1</code>
<code>  compression: gzip</code>
<code>  max_message_bytes: 1000000

Adjust log paths, Kafka addresses, and topic names as needed.

Step 3 – Share a Volume

Create an emptyDir volume and mount it to both the main and sidecar containers at /var/log so that Filebeat can read the application logs.

Step 4 – Verify Collection

After deployment, check the Kafka console; the sidecar should have forwarded logs from /var/log to the configured Kafka topic.

Future Outlook

Sidecar integration gives SAE powerful extensibility for logging and monitoring. Upcoming plans include support for multi‑tenant Istio service mesh, further simplifying service‑mesh deployment and management.