How Hybridnet Enables Unified Underlay/Overlay Networking in ACK Distro

Hybridnet Overview and Design Principles

Hybridnet is Alibaba’s open‑source Kubernetes CNI that provides a unified underlay + overlay networking plane for heterogeneous physical and virtual machines in hybrid‑cloud environments. The solution follows five design principles:

Define a single network model to reduce cognitive load and simplify long‑term evolution.

Abstract heterogeneous infrastructure to improve delivery robustness and lower PoC cost.

Within the unified model, offer high‑performance underlay networking for latency‑sensitive workloads and overlay networking for workloads that do not require direct‑layer performance.

Minimize external dependencies to keep the data‑plane lightweight.

Deep integration with Kubernetes, supporting dual‑stack, IP persistence, and fixed IP to preserve existing cloud‑migration habits.

Core Custom Resource Definitions (CRDs)

Hybridnet introduces three CRDs that model network resources:

Network : Represents a scheduling domain (a group of nodes with the same network characteristics). A nodeSelector can bind the Network to specific nodes; an empty selector makes the Network applicable to all nodes (e.g., the default overlay Network).

Subnet : Defines the IP address pool belonging to a Network. Fields include cidr, range.start, range.end, range.excludeIPs, and range.reservedIPs for fine‑grained allocation control.

IPInstance : A monitoring object that records each allocated container IP. kubectl get IPInstance shows the associated Pod, Subnet, and node.

Hybridnet in ACK Distro

When ACK Distro is installed, Hybridnet is the built‑in network plugin. By default the cluster creates:

# kubectl get network
NAME        NETID   SWITCHID
network-0   4       virtual-switch

# kubectl get subnet
NAME                 VERSION   CIDR            START       END   GATEWAY      TOTAL   USED   AVAILABLE   NETID   NETWORK
subnet-0-network-0   4         100.64.0.0/16                100.64.0.1   65533   2      65531               4       network-0

All Pods are launched using the overlay mode (Network network-0, Subnet subnet-0-network-0, CIDR 100.64.0.0/16).

Adding an Underlay Network

To provision an underlay network, apply a Network and a matching Subnet. The example below creates an underlay network on node izf8zdygpbo4hx57g2wah8z and reserves the IP range 192.168.56.100‑150 for containers:

---
apiVersion: networking.alibaba.com/v1
kind: Network
metadata:
  name: underlay-network1
spec:
  netID: 0
  nodeSelector:
    network: network1
  type: Underlay
---
apiVersion: networking.alibaba.com/v1
kind: Subnet
metadata:
  name: underlay-subnet1
spec:
  network: underlay-network1
  netID: 0
  range:
    version: "4"
    cidr: "192.168.56.0/24"
    gateway: "192.168.56.254"
    start: "192.168.56.100"
    end: "192.168.56.150"

Label the target node so that the underlay Network can schedule Pods on it:

kubectl label node izf8zdygpbo4hx57g2wah8z network=network1

Pods that should use the underlay network must be annotated with networking.alibaba.com/network-type: Underlay. The default network type can be switched to underlay by editing the hybridnet-webhook and hybridnet-manager Deployments and setting the environment variable DEFAULT_NETWORK_TYPE=Underlay:

spec:
  containers:
  - name: hybridnet-manager
    command:
    - /hybridnet/hybridnet-manager
    env:
    - name: DEFAULT_NETWORK_TYPE
      value: Underlay

Lifecycle of Network and Subnet Resources

Creating or deleting a Network/Subnet is performed by applying or deleting the corresponding CR YAML files. Deletion is constrained:

A Subnet can be removed only when no IPInstance objects reference it (i.e., no IPs are in use).

A Network can be deleted only after all its Subnets have been removed.

Key Operational Points

The default overlay Network provides a “network‑agnostic” layer that hides underlying infrastructure, enabling rapid cluster bootstrapping in any environment.

Underlay networks can be added later for performance‑critical workloads or when direct IP exposure is required.

Switching the default network type to Underlay makes newly created Pods use the underlay by default, while existing overlay Pods remain unaffected. Overlay Pods can still be created explicitly via the network-type annotation.

References

Hybridnet source code: https://github.com/alibaba/hybridnet

Hybridnet documentation: https://github.com/alibaba/hybridnet/wiki

ACK Distro repository: https://github.com/AliyunContainerService/ackdistro