Declarative Host Networking on Bare-Metal OpenShift

Bare-metal OpenShift inherits every networking decision the underlying host makes, and most of those decisions are usually made once, by hand, during commissioning. A bond is built with a particular naming convention, a VLAN tag is applied to satisfy a firewall rule that existed at the time, and the result is recorded in a runbook rather than in the cluster itself. Six months later a node is reimaged and nobody quite remembers why the bond used balance-alb instead of 802.3ad. We have inherited enough of these fleets to treat this pattern as a structural risk rather than a one-off oversight, and NMState is the mechanism we use to close it.

Host networking as declared state, not a build step

The Kubernetes NMState operator lets host network configuration be expressed as a NodeNetworkConfigurationPolicy and applied against node selectors, the same way a Deployment is applied against a namespace. A bonded, VLAN-tagged interface for a segmented workload zone looks like this:

apiVersion: nmstate.io/v1
kind: NodeNetworkConfigurationPolicy
metadata:
  name: bond0-vlan100-worker
spec:
  nodeSelector:
    node-role.kubernetes.io/worker: ""
  desiredState:
    interfaces:
      - name: bond0
        type: bond
        state: up
        link-aggregation:
          mode: 802.3ad
          options:
            miimon: '140'
          port: [ens3f0, ens3f1]
        mtu: 9000
      - name: bond0.100
        type: vlan
        state: up
        vlan:
          base-iface: bond0
          id: 100
        ipv4:
          enabled: true
          address:
            - ip: 198.51.100.15
              prefix-length: 24

The point is not the YAML, it is what the YAML replaces: a person on console, a checklist, and a hope that the next person follows the same checklist. The policy is version-controlled, reviewed like any other change, and reapplied automatically if a node’s live state drifts from it — including after a reimage, a NIC replacement, or a firmware-driven interface rename.

Controlled egress for segmented namespaces

Bonded and tagged interfaces solve connectivity at the host. The harder problem on a bare-metal cluster carrying workloads of different sensitivity is egress: which namespace’s traffic is permitted to leave through which path, with which source address, so that a downstream firewall or a partner network can make a decision on it. EgressIP gives a namespace a stable, routable identity distinct from the node it happens to land on — the address is assigned by the cluster to a node explicitly marked as egress-capable, not conjured on the namespace itself:

apiVersion: k8s.ovn.org/v1
kind: EgressIP
metadata:
  name: egress-restricted-zone
spec:
  egressIPs:
    - 203.0.113.20
  namespaceSelector:
    matchLabels:
      network-zone: restricted

Paired with a NetworkPolicy that denies namespace-to-namespace traffic by default and an EgressFirewall that limits external destinations to an explicit allow-list, this turns “which workloads can talk to what” from an assertion in an architecture document into an enforced, auditable property of the cluster. We build these three primitives together as a set, never in isolation — an egress IP without a default-deny policy behind it only adds a stable address to traffic that was never supposed to be reachable in the first place.

Why reconciliation beats hand-configured hosts

The case for this approach is not aesthetic. Hand-configured hosts fail in a specific, recurring way: the configuration is correct at the moment it is applied and silently wrong at some later moment, after a driver update, a replaced NIC, or a well-intentioned manual fix during an incident that never made it back into documentation. Nothing detects the drift until a workload cannot reach a dependency and someone starts comparing ip a output across nodes by hand.

A reconciled policy behaves differently. The operator continuously compares desired state to actual state and corrects deviation without a person deciding to intervene. Drift becomes a NodeNetworkState diff you can query, not an incident you discover downstream. This also changes how a change gets made in the first place: a network engineer proposes a policy change as a pull request, it is reviewed against the standard the organisation has agreed to, and it rolls out to every node the selector matches — rather than to whichever nodes someone remembered to touch.

The operating-model change this forces

None of this is free. It requires a network team to give up console-driven, per-host authority over interface configuration and accept that the source of truth is a Git repository, not a host’s running state. That is a genuine cultural shift for teams whose expertise was built on switch and NIC configuration done directly and well. The skill does not disappear — it moves upstream, into writing the policy correctly, defining node selectors that match the fleet’s actual topology, and reviewing changes before they are applied to production. Troubleshooting shifts from “log into the box” to “read the policy and the reconciliation status,” which is a faster diagnostic path once teams trust it, and a slower one until they do.

We treat that adoption curve as part of the engagement, not an afterthought to it: pairing the platform team with the network team through the first several policy changes, documenting the review standard, and leaving both groups able to operate the model independently. Declarative host networking only pays off if the team that inherits it actually runs it that way once we are gone.