Automatic Interconnection of Underlay and Overlay Subnets
If a cluster has both Underlay and Overlay subnets, by default, Pods under the Overlay subnet can access Pods' IPs in the Underlay subnet through a gateway using NAT. However, Pods in the Underlay subnet need to configure node routing to access Pods in the Overlay subnet.
To achieve automatic interconnection between Underlay and Overlay subnets, you can manually modify the YAML file of the Underlay subnet. Once configured, Kube-OVN will also use an additional Underlay IP to connect the Underlay subnet and the ovn-cluster logical router, setting the corresponding routing rules to enable interconnection.
TOC
ProcedureEnable Overlay Route Only for Underlay + MetalLBIsolation Between Underlay Subnets with u2oInterconnection EnabledStep 1: Configure kube-ovn-controllerStep 2: Configure Subnet IsolationProcedure
-
Go to Administrator.
-
In the left navigation bar, click on Cluster Management > Resource Management.
-
Enter Subnet to filter resource objects.
-
Click on ⋮ > Update next to the Underlay subnet to be modified.
-
Modify the YAML file, adding
u2oInterconnection: trueinspec. When using Underlay + MetalLB, also enable Overlay Route Only as described below. -
Click Update.
Note: Existing compute components in the Underlay subnet need to be recreated for the changes to take effect.
Enable Overlay Route Only for Underlay + MetalLB
When using U2O with MetalLB LoadBalancer Services on Underlay subnets, enable Overlay Route Only (u2oFeatures.overlayOnlyRouting: true) on every participating Underlay subnet where u2oInterconnection: true. Traffic between different Underlay subnets then uses the physical gateway, including return traffic to LoadBalancer clients in those subnets. Communication with Overlay subnets in the same VPC continues to use OVN.
This option requires u2oInterconnection: true and a networking plugin that supports u2oFeatures.overlayOnlyRouting. It changes OVN routing policies; the Pod's default gateway remains the U2O interconnection gateway. For the complete MetalLB configuration, see Kube-OVN Underlay + MetalLB LoadBalancer Service Configuration.
Isolation Between Underlay Subnets with u2oInterconnection Enabled
When multiple Underlay subnets have u2oInterconnection: true enabled without Overlay Route Only, traffic between them can be routed directly via the internal OVN network, bypassing the physical gateway.
For Underlay + MetalLB, enable Overlay Route Only on each participating Underlay subnet so that inter-subnet traffic follows the physical network's routing and isolation policies. This option does not block traffic by itself; enforce isolation with physical-network policies or subnet ACLs.
To additionally enforce isolation with subnet ACLs, first configure the kube-ovn-controller parameter, then configure the subnet isolation.
Step 1: Configure kube-ovn-controller
Modify the kube-ovn-controller Deployment to disable connection tracking skip for destination logical port IPs:
Add or modify the following argument:
--ls-ct-skip-dst-lport-ips controls whether to skip connection tracking (conntrack) for traffic destined to logical port IPs. The default value is true, which skips conntrack to improve performance. Setting it to false does not affect functionality but may slightly impact performance.
However, for Underlay subnets with ACL-based isolation, you must set it to false. Otherwise, gateway-to-Pod traffic will fail (e.g., ping requests reach the Pod but replies are dropped), because ACL isolation uses allow-related which requires conntrack state; without it, replies cannot be identified as "related" and get dropped.
Step 2: Configure Subnet Isolation
Configure the subnet with the following parameters:
ACL Parameters:
- The
aclsfield provides priority-based rule evaluation, offering more flexibility than standard Kubernetes NetworkPolicy. - When using
to-lportdirection,ip4.srcrefers to the source IP of incoming traffic. - Recommended priority range:
1002to1899to avoid conflicts with system default ACL rules.