Mastering Telepresence: Seamless Local Development with Remote Kubernetes Clusters

Telepresence is an open‑source tool that allows you to run a single service locally while connecting it to a remote Kubernetes cluster.

Why Telepresence Is Needed

Kubernetes pod IPs are assigned by CNI and communicate over an overlay network; these IPs are only usable inside the cluster. Although you can manually map pod IPs to host entries using kubectl get po -o wide, this approach is cumbersome, especially when services reference internal DNS names such as xxxx.svc.cluster.local:1234. Telepresence automates DNS resolution, traffic proxying, and traffic interception, eliminating the need for manual host file edits.

Key Features

Cluster DNS resolution – resolves names like xxxx.svc.cluster.local on the local machine.

Cluster traffic proxy – forwards traffic from the local machine to the cluster.

Cluster traffic intercept – captures traffic destined for a specific service and redirects it to the local development environment.

DNS Resolution

On Linux, Telepresence creates a tun device named tel0 and uses systemd-resolved to add the cluster domain to the device. You can view the configuration with resolvectl status tel0:

resolvectl status tel0
Link 66 (tel0)
    Current Scopes: DNS LLMNR/IPv4 LLMNR/IPv6
    Protocols: -DefaultRoute +LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported
Current DNS Server: 10.0.165.145
    DNS Servers: 10.0.165.145
    DNS Domain: ~ambassador ~argocd ~cluster.local ~db ~default ~devops ~istio-system ~jaeger ~kube-public ~kube-system ~nacos ~observability

On macOS, Telepresence uses resolver hooks by creating files under /etc/resolver for each mapped namespace.

Daily Usage

Typical commands: telepresence connect – establish a connection. telepresence status – view connection status. curl -ik https://kubernetes.default – test connectivity. telepresence quit -u -r – disconnect and stop both user and root daemons.

Troubleshooting

1. Traffic intercept does not work

When an istio‑proxy sidecar is injected, it captures all traffic, preventing Telepresence’s traffic‑agent from intercepting. A temporary workaround is to disable sidecar injection by adding the annotation sidecar.istio.io/inject: "false" to the deployment.

diff --git a/develop/overlays/my_app/deployment.yaml b/develop/overlays/my_app/deployment.yaml
index 1049d335..26ee38d4 100644
--- a/develop/overlays/my_app/deployment.yaml
+++ b/develop/overlays/my_app/deployment.yaml
@@ -4,6 +4,9 @@ metadata:
   name: ttys3
 spec:
   template:
+    metadata:
+      annotations:
+        sidecar.istio.io/inject: "false"
     spec:
       containers:
       - name : my-app

2. Connection failures

Version mismatches between the Telepresence client and the traffic‑manager cause errors such as "port‑forward connection timed out" or "traffic manager gRPC API timed out". Ensure all users in a cluster use the exact same Telepresence version and that the appropriate RBAC permissions are granted (list namespaces, create pods/portforward in the ambassador namespace, etc.).

❯ telepresence version
Client: v2.5.4 (api v3)
Root Daemon: v2.5.4 (api v3)
User Daemon: v2.5.4 (api v3)

3. Complete uninstallation

Run telepresence uninstall --everything. If manual cleanup is required, delete the traffic‑manager deployment and its Helm secret:

k delete deploy -n ambassador traffic-manager
k delete secret sh.helm.release.v1.traffic-manager.v1 -n ambassador

4. Building custom images

When building, set TELEPRESENCE_VERSION and TELEPRESENCE_REGISTRY to keep client and image versions in sync:

# Build and push a custom image
make build image push-image TELEPRESENCE_VERSION=v2.5.4 TELEPRESENCE_REGISTRY=docker.io/ttys3

# Only build the binary
make build TELEPRESENCE_VERSION=v2.5.4 TELEPRESENCE_REGISTRY=docker.io/ttys3

5. DNS resolution issues with TLD conflicts

If a Kubernetes namespace collides with a public TLD (e.g., a namespace named rs), Telepresence may block resolution of all .rs domains. The source can be patched to skip eTLDs using publicsuffix.PublicSuffix before calling SetLinkDomains.

// Example patch snippet
if eTLD, icann := publicsuffix.PublicSuffix(path); icann && path == eTLD {
    dlog.Infof(c, "Skip set Link domains on device %q for [%s] due to conflict with ICANN eTLD [%s]", dev.Name(), path, eTLD)
    continue
}

6. Slow or timed‑out DNS queries

In some environments DNS lookups time out (e.g.,

LookupHost: lookup kubernetes.default on 10.0.165.145:53: i/o timeout

). The issue often traces back to a failing CoreDNS pod; restarting CoreDNS and the traffic‑manager usually resolves the problem.

7. Excluding specific IP ranges from the tel0 tunnel

Add a never-proxy entry to ~/.kube/config to keep traffic to certain IPs or subnets on the host network instead of the tunnel:

- cluster:
    certificate-authority-data: xxxxxx
    server: https://example.com
    extensions:
    - name: telepresence.io
      extension:
        never-proxy:
        - 10.10.8.9/32
  name: cluster-001

References

Local development and debugging services – kubernetes.io

systemd‑resolved DBus interface – freedesktop.org

Running Telepresence inside a Docker container – telepresence.io

Telepresence DNS configuration – telepresence.io