Mastering Service Mesh with Istio: Deploy, Manage, and Monitor on Kubernetes

Service Mesh Overview

Service Mesh (service mesh) is an infrastructure layer that handles communication between services, providing essential micro‑service components such as service discovery, load balancing, monitoring, traffic management, and access control. In practice it is implemented as a set of lightweight sidecar proxies deployed alongside applications, remaining transparent to the applications.

Key characteristics of a service mesh include independent governance via sidecars, application transparency, a dedicated infrastructure layer for service communication, and decoupling of retries, timeouts, monitoring, tracing, and service discovery from the business logic.

The mesh separates control and data planes: each application is paired with a sidecar that intercepts outbound requests, while governance policies are pushed from the control plane to the sidecars, allowing independent upgrades of business modules and governance rules.

Core Benefits

Decoupling of micro‑service governance from business logic by moving most SDK capabilities into sidecar processes.

Unified governance of heterogeneous systems, simplifying multi‑language deployments.

Observability: captures source, destination, protocol, URL, status code, latency, and duration for each request.

Traffic control: intelligent routing, timeout/retry, circuit breaking, fault injection, and mirroring.

Security: mutual TLS, authentication, and policy enforcement.

Robustness: supports fault injection for resilience testing.

Istio Overview

Istio is the most popular, feature‑rich service mesh implementation. It separates the data plane (Envoy sidecars) from the control plane, offering communication, security, traffic control, and observability functions. Istio integrates tightly with Kubernetes, using the platform to manage service lifecycles.

Istio Features

Connect: traffic management, load balancing, canary releases.

Secure: authentication, authorization.

Control: rate limiting, ACL.

Observe: monitoring, tracing.

Performance Summary

In a benchmark with 1,000 services and 2,000 sidecars, Istio 1.6.2 achieved 70,000 QPS. Typical resource usage: each Envoy sidecar handling 1,000 QPS consumes ~0.5 vCPU and 50 MiB memory; the telemetry service uses ~0.6 vCPU at 1,000 QPS; the Pilot component uses ~1 vCPU and 1.5 GiB memory. 90 % of requests see only ~6.3 ms added latency.

Deploying Istio on Kubernetes

Download and extract Istio, install the demo profile, and verify the components:

# wget https://github.com/istio/istio/releases/download/1.6.2/istio-1.6.2-linux-amd64.tar.gz
# tar -zxvf istio-1.6.2-linux-amd64.tar.gz -C /data/
cd /data/istio-1.6.2/
mv bin/istioctl /usr/bin
istioctl profile list
# install demo profile
istioctl install --set profile=demo
kubectl get pods -n istio-system

To uninstall:

istioctl manifest generate --set profile=demo | kubectl delete -f -

Sidecar Injection

Manual injection:

# kubectl apply -f <(istioctl kube-inject -f xxx.yaml)

Automatic injection (enable namespace label):

# kubectl label namespace xxx istio-injection=enabled

Gateway Configuration

Istio’s ingressgateway acts as a load balancer and terminates mTLS. Example Gateway and VirtualService:

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: httpbin-gateway
spec:
  selector:
    istio: ingressgateway
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - "*"
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: httpbin
spec:
  hosts:
  - "*"
  gateways:
  - httpbin-gateway
  http:
  - route:
    - destination:
        host: httpbin
        port:
          number: 8000

Bookinfo Demo Application

The Bookinfo sample consists of four micro‑services (productpage, details, reviews, ratings). The reviews service has three versions (v1, v2, v3) demonstrating different rating displays.

Deploy Bookinfo to the default namespace and enable automatic sidecar injection:

# label namespace for injection
kubectl label namespace default istio-injection=enabled
# apply Bookinfo manifests
kubectl apply -f samples/bookinfo/platform/kube/bookinfo.yaml

Create the ingress gateway for Bookinfo:

# apply gateway
kubectl apply -f samples/bookinfo/networking/bookinfo-gateway.yaml
# retrieve gateway address
export INGRESS_HOST=$(kubectl get po -l istio=ingressgateway -n istio-system -o jsonpath='{.items[0].status.hostIP}')
export INGRESS_PORT=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="http2")].nodePort}')
export GATEWAY_URL=$INGRESS_HOST:$INGRESS_PORT
echo http://$GATEWAY_URL/productpage

Accessing the Bookinfo URL shows the application UI, while Kiali visualizes the mesh topology and real‑time traffic distribution.

Observability and Monitoring

Istio ships with three built‑in monitoring tools:

Grafana – dashboards for request error rates, latency, and other metrics.

Kiali – service‑mesh topology, RPS, error rates, request/response sizes, pod logs, and live configuration editing.

Jaeger – distributed tracing of request flows, response times, and detailed payload inspection.

Dashboards can be opened via istioctl dashboard (e.g., istioctl dashboard grafana).

# list available dashboards
istioctl dashboard -h

Custom Monitoring Gateways

To expose Grafana, Kiali, Jaeger, and Zipkin through custom Istio gateways, define Gateway and VirtualService resources (see the source for full YAML). Apply them with kubectl apply -f monitor-gateway.yaml and verify the resources.

# apply custom monitoring gateways
kubectl apply -f monitor-gateway.yaml
kubectl get gateway,vs -n istio-system

External Load Balancer

An Nginx instance can be configured as a load balancer that forwards external traffic to the Istio ingressgateway’s NodePort.

upstream ingressgateway {
  server 10.100.132.8:22516;
  server 10.100.132.5:22516;
  server 10.100.132.6:22516;
}
server {
  listen 80 default_server;
  location / {
    proxy_pass http://ingressgateway;
    proxy_set_header Host $host;
    proxy_http_version 1.1;
  }
}

After updating DNS or /etc/hosts to point to the Nginx server, the Bookinfo application and all monitoring UIs become reachable through the external load balancer.