How to Build a Self‑Hosted P2P VPN with Headscale and Tailscale

Internal Network Penetration Overview

Most home broadband connections lack a public IP, so accessing a home network from outside typically requires VPNs or remote‑desktop tools, which often suffer from slow speeds and high latency due to central server bottlenecks.

Traditional star topology forces two devices behind multiple NAT layers to communicate through a central server, limiting the maximum transfer speed to the slowest link (e.g., 512 K/s) and requiring additional payment for higher bandwidth.

NAT Traversal and Mesh Topology

NAT traversal creates temporary firewall mappings that allow devices to send traffic directly to each other after an initial outbound packet. This “punch‑through” technique enables a mesh topology where each node can communicate at its own bandwidth without a central bottleneck.

Mesh topology delivers full bandwidth between peers, but NAT traversal may fail in complex networks; a fallback central server can be used when direct P2P fails.

Headscale Overview

Headscale is an open‑source implementation of Tailscale’s control plane. It enables private, self‑controlled VPN networks using the same NAT‑traversal techniques as Tailscale. The official Tailscale control server is not open source and performs poorly in some regions, so Headscale provides a locally hosted alternative.

Install Headscale Server (Ubuntu 22.04 example)

# Download
wget https://github.com/juanfont/headscale/releases/download/v0.16.4/headscale_0.16.4_linux_amd64 -O /usr/local/bin/headscale

# Make executable
chmod +x /usr/local/bin/headscale

# Create configuration directory
mkdir -p /etc/headscale

# Create a dedicated system user
useradd \
  --create-home \
  --home-dir /var/lib/headscale/ \
  --system \
  --user-group \
  --shell /usr/sbin/nologin \
  headscale

# Systemd service file (/lib/systemd/system/headscale.service)
[Unit]
Description=headscale controller
After=syslog.target network.target

[Service]
Type=simple
User=headscale
Group=headscale
ExecStart=/usr/local/bin/headscale serve
Restart=always
RestartSec=5

# Optional security hardening
NoNewPrivileges=yes
PrivateTmp=yes
ProtectSystem=strict
ProtectHome=yes
ReadWritePaths=/var/lib/headscale /var/run/headscale
AmbientCapabilities=CAP_NET_BIND_SERVICE
RuntimeDirectory=headscale

[Install]
WantedBy=multi-user.target

Configure Headscale

Edit /etc/headscale/config.yaml with the essential options:

---
# Public address that clients will use to reach the server
server_url: https://your.domain.com

# Address Headscale actually listens on
listen_addr: 0.0.0.0:8080

metrics_listen_addr: 127.0.0.1:9090

grpc_listen_addr: 0.0.0.0:50443
grpc_allow_insecure: false

# IP ranges allocated to clients
ip_prefixes:
  - fd7a:115c:a1e0::/48
  - 100.64.0.0/10

# DERP (relay) configuration – disabled by default
derp:
  server:
    enabled: false
  urls: []
  paths: []
  auto_update_enabled: true
  update_frequency: 24h

# SQLite storage
db_type: sqlite3
db_path: /var/lib/headscale/db.sqlite

tls_letsencrypt_hostname: ""
tls_cert_path: ""
tls_key_path: ""

randomize_client_port: false

For TLS you can let Headscale obtain a certificate via Let’s Encrypt (set tls_letsencrypt_hostname) or provide your own certificate files ( tls_cert_path and tls_key_path). When using a reverse proxy, leave the TLS fields empty and proxy the HTTP listen_addr port.

Start Headscale

# Enable and start the service
systemctl enable headscale --now

Deploy a DERP Relay (optional)

Compile the DERP server from source:

# Build DERP
go install tailscale.com/cmd/derper@main

# Move binary to /usr/local/bin
mv ${GOPATH}/bin/derper /usr/local/bin/derper

Create a dedicated user and systemd unit:

# Create user
useradd \
  --create-home \
  --home-dir /var/lib/derper/ \
  --system \
  --user-group \
  --shell /usr/sbin/nologin \
  derper

# Systemd service (/lib/systemd/system/derper.service)
[Unit]
Description=tailscale derper server
After=syslog.target network.target

[Service]
Type=simple
User=derper
Group=derper
ExecStart=/usr/local/bin/derper -c=/var/lib/derper/private.key -a=:8989 -stun-port=3456 -verify-clients
Restart=always
RestartSec=5

# Security hardening
NoNewPrivileges=yes
PrivateTmp=yes
ProtectSystem=strict
ProtectHome=yes
ReadWritePaths=/var/lib/derper /var/run/derper
AmbientCapabilities=CAP_NET_BIND_SERVICE
RuntimeDirectory=derper

[Install]
WantedBy=multi-user.target

# Enable and start DERP
systemctl enable derper --now

Configure Headscale to use the custom DERP server by adding a derper.yaml file and referencing it in config.yaml under derp.paths:

# /etc/headscale/derper.yaml
regions:
  901:
    regionid: 901
    regioncode: private-derper
    regionname: "My Private Derper Server"
    nodes:
      - name: private-derper
        regionid: 901
        hostname: derper.your.domain.com
        ipv4: 123.123.123.123
        stunport: 3456

# Update main config.yaml
derp:
  server:
    enabled: false
  urls: []
  paths:
    - /etc/headscale/derper.yaml
  auto_update_enabled: true
  update_frequency: 24h

Client Installation

Linux

Run the official installer script, which detects the distribution and installs the appropriate package:

curl -fsSL https://tailscale.com/install.sh | sh

Start the client and point it at your Headscale server:

tailscale up \
  --login-server https://your.domain.com \
  --advertise-routes=192.168.11.0/24 \
  --accept-routes=true \
  --accept-dns=false

Key options: --login-server: URL of the Headscale control server (required). --advertise-routes: Subnet that this node can route. --accept-routes: Accept routes advertised by other nodes. --accept-dns: Disable DNS configuration from the server (recommended).

macOS

Two options are provided:

Install the App Store version, set the server URL (e.g., https://your.domain.com/apple), then run the GUI client.

Compile the client from source for a command‑line experience:

# Install Go
brew install go

# Build the client binaries
go install tailscale.com/cmd/tailscale{,d}@main

# Install as a system daemon
sudo tailscaled install-system-daemon

After installation, use the same tailscale up command as on Linux.

Windows

Download the Windows client from https://your.domain.com/windows, install the GUI, and run the same tailscale up command in an elevated PowerShell window.

Network Debugging

Ping

Use tailscale ping 10.24.0.5 to test connectivity. The command first tries a DERP relay and falls back to direct P2P once a direct path is found.

Status

tailscale status

NetCheck

tailscale netcheck

Routing and Forwarding

To enable a node to act as a router, start it with --advertise-routes=192.168.1.0/24. Other nodes must enable --accept-routes=true. Then, on the Headscale server, enable routing for a specific node: headscale node route enable -a -i 12345 Removing the -a flag disables the advertised routes.

Additional Considerations

When using DERP on port 443, ACME will automatically request a certificate unless --certmode=manual is specified.

If you need to run DERP behind a load balancer, bind it to a non‑443 port and configure the LB accordingly.

Avoid using the default 100.64.0.0/10 CGNAT range if it collides with provider networks.