If you're using the Oracle Linux yum server for system updates, enable the required yum repositories.

To enable the yum repositories:

1. Install the oracle-olcne-release-el9 release package to install the Oracle Cloud Native Environment yum repository configuration.

   sudo dnf install oracle-olcne-release-el9

2. Set up the repositories for the release you want to install.

   Enable the following yum repositories:

   • ol9_olcne18

   • ol9_addons

   • ol9_baseos_latest

   • ol9_appstream

   • ol9_UEKR7 (if hosts are running UEK R7)

   Use the dnf config-manager tool to enable the yum repositories. For hosts running UEK R7:

   sudo dnf config-manager --enable ol9_olcne18 ol9_addons ol9_baseos_latest ol9_appstream ol9_UEKR7

   For hosts running RHCK:

   sudo dnf config-manager --enable ol9_olcne18 ol9_addons ol9_baseos_latest ol9_appstream

3. Disable the yum repositories for previous Oracle Cloud Native Environment releases.

   sudo dnf config-manager --disable ol9_olcne17

4. Disable any developer yum repositories. To list the developer repositories that need to be disabled, use the dnf repolist command:

   sudo dnf repolist --enabled | grep developer

   Disable the repositories returned using the dnf config-manager tool. For example:

   sudo dnf config-manager --disable ol9_developer

If you're registered to use ULN, use the ULN web interface to subscribe the system to the appropriate channels.

To subscribe to the ULN channels:

1. Sign in to https://linux.oracle.com.

2. On the Systems tab, click the link named for the system in the list of registered machines.

3. On the System Details page, click Manage Subscriptions.

4. On the System Summary page, select each required channel from the list of available channels and click the arrow to move the channel to the list of subscribed channels.

   Subscribe the system to the following channels:

   • ol9_x86_64_olcne18

   • ol9_x86_64_addons

   • ol9_x86_64_baseos_latest

   • ol9_x86_64_appstream

   • ol9_x86_64_UEKR7 (if hosts are running UEK R7)

   Ensure the systems aren't subscribed to the ol9_x86_64_olcne17 or ol9_x86_64_developer channels.

If you're using the Oracle Linux yum server for system updates, enable the required yum repositories.

To enable the yum repositories:

1. Install the oracle-olcne-release-el8 release package to install the Oracle Cloud Native Environment yum repository configuration.

   sudo dnf install oracle-olcne-release-el8

2. Set up the repositories for the release you want to install.

   Enable the following yum repositories:

   • ol8_olcne18

   • ol8_addons

   • ol8_baseos_latest

   • ol8_appstream

   • ol8_kvm_appstream

   • ol8_UEKR7 (if hosts are running UEK R7)

   • ol8_UEKR6 (if hosts are running UEK R6)

   Use the dnf config-manager tool to enable the yum repositories. For hosts running UEK R7:

   sudo dnf config-manager --enable ol8_olcne18 ol8_addons ol8_baseos_latest ol8_appstream ol8_kvm_appstream ol8_UEKR7

   For hosts running UEK R6:

   sudo dnf config-manager --enable ol8_olcne18 ol8_addons ol8_baseos_latest ol8_appstream ol8_kvm_appstream ol8_UEKR6

   For hosts running RHCK:

   sudo dnf config-manager --enable ol8_olcne18 ol8_addons ol8_baseos_latest ol8_appstream ol8_kvm_appstream

3. Disable the yum repositories for previous Oracle Cloud Native Environment releases.

   sudo dnf config-manager --disable ol8_olcne17 ol8_olcne16 ol8_olcne15 ol8_olcne14 ol8_olcne13 ol8_olcne12

4. Disable any developer yum repositories. To list the developer repositories that need to be disabled, use the dnf repolist command:

   sudo dnf repolist --enabled | grep developer

   Disable the repositories returned using the dnf config-manager tool. For example:

   sudo dnf config-manager --disable ol8_developer

If you're registered to use ULN, use the ULN web interface to subscribe the system to the appropriate channels.

To subscribe to the ULN channels:

1. Sign in to https://linux.oracle.com.

2. On the Systems tab, click the link named for the system in the list of registered machines.

3. On the System Details page, click Manage Subscriptions.

4. On the System Summary page, select each required channel from the list of available channels and click the arrow to move the channel to the list of subscribed channels.

   Subscribe the system to the following channels:

   • ol8_x86_64_olcne18

   • ol8_x86_64_addons

   • ol8_x86_64_baseos_latest

   • ol8_x86_64_appstream

   • ol8_x86_64_kvm_appstream

   • ol8_x86_64_UEKR7 (if hosts are running UEK R7)

   • ol8_x86_64_UEKR6 (if hosts are running UEK R6)

   Ensure the systems aren't subscribed to the following channels:

   • ol8_x86_64_developer

   • ol8_x86_64_olcne17

   • ol8_x86_64_olcne16

   • ol8_x86_64_olcne15

   • ol8_x86_64_olcne14

   • ol8_x86_64_olcne13

   • ol8_x86_64_olcne12

For a cluster with a single control plane node, the following ports are required to be open in the firewall.

sudo firewall-cmd --add-port=8091/tcp --permanent

sudo systemctl restart firewalld.service

sudo firewall-cmd --zone=trusted --add-interface=cni0 --permanent
sudo firewall-cmd --add-port=8090/tcp --permanent
sudo firewall-cmd --add-port=10250/tcp --permanent
sudo firewall-cmd --add-port=10255/tcp --permanent
sudo firewall-cmd --add-port=8472/udp --permanent

sudo systemctl restart firewalld.service

sudo firewall-cmd --zone=trusted --add-interface=cni0 --permanent
sudo firewall-cmd --add-port=8090/tcp --permanent
sudo firewall-cmd --add-port=10250/tcp --permanent
sudo firewall-cmd --add-port=10255/tcp --permanent
sudo firewall-cmd --add-port=8472/udp --permanent
sudo firewall-cmd --add-port=6443/tcp --permanent

sudo systemctl restart firewalld.service

For a highly available cluster, open all the firewall ports as described in Non-HA Cluster Firewall Rules, along with the following extra ports on the control plane nodes.

On the Kubernetes control plane nodes run:

sudo firewall-cmd --add-port=10251/tcp --permanent
sudo firewall-cmd --add-port=10252/tcp --permanent
sudo firewall-cmd --add-port=2379/tcp --permanent
sudo firewall-cmd --add-port=2380/tcp --permanent

Restart the firewall for these rules to take effect:

sudo systemctl restart firewalld.service

The Platform CLI checks it can access the container registry, and possibly other Internet resources, to pull any required container images. Unless you intend to set up a local registry mirror for container images, the systems where you intend to install Oracle Cloud Native Environment must either have direct internet access, or must be configured to use a proxy.

If you're using Oracle Linux 9 hosts and you have firewalld enabled, you must add an nftables rule on the Kubernetes nodes.

If you don't need to persist the rule over reboots, set up the rule on each Kubernetes node using:

sudo nft add rule inet firewalld filter_FORWARD_POLICIES_post accept

If you do need to persist the rule over host reboots (recommended), set up a rule file and create a systemd service that's enabled. On each Kubernetes node, run the following.

1. Create a file named /etc/nftables/forward-policies.nft which contains the rule by entering:

   sudo sh -c "cat > /etc/nftables/forward-policies.nft << EOF
   flush chain inet firewalld filter_FORWARD_POLICIES_post
   
   table inet firewalld {
           chain filter_FORWARD_POLICIES_post {
                   accept
           }
   }
   EOF"

2. Create a file named /etc/systemd/system/forward-policies.service which contains the systemd service by entering:

   sudo sh -c "cat > /etc/systemd/system/forward-policies.service << EOF
   [Unit]
   Description=Idempotent nftables rules for forward-policies
   PartOf=firewalld.service
   
   [Service]
   ExecStart=/sbin/nft -f /etc/nftables/forward-policies.nft
   ExecReload=/sbin/nft -f /etc/nftables/forward-policies.nft
   Restart=always
   StartLimitInterval=0
   RestartSec=10
   
   [Install]
   WantedBy=multi-user.target
   EOF"

3. Enable and restart the rule service:

   sudo systemctl enable forward-policies.service
   sudo systemctl restart forward-policies.service

The Platform CLI configures a Flannel network as the network fabric used for communications between Kubernetes pods. This overlay network uses VxLANs for network connectivity. For more information on Flannel, see the upstream documentation.

By default, the Platform CLI creates a network in the 10.244.0.0/16 range to host this network. The Platform CLI provides an option to set the network range to another range, if required, during installation. Systems in an Oracle Cloud Native Environment deployment must not have any network devices configured for this reserved IP range.

The Platform CLI checks whether the br_netfilter module is loaded and exits if it's not available. This module is required to enable transparent masquerading and for Virtual Extensible LAN (VxLAN) traffic for communication between Kubernetes pods across the cluster. If you need to check whether it's loaded, run:

sudo lsmod|grep br_netfilter

If you see the output similar to the following, the br_netfilter module is loaded.

br_netfilter           24576  0 
bridge                155648  2 br_netfilter,ebtable_broute

Kernel modules are loaded as they're needed, and it's unlikely that you need to load this module manually. You can load the module manually and add it as a permanent module by running:

sudo modprobe br_netfilter
sudo sh -c 'echo "br_netfilter" > /etc/modules-load.d/br_netfilter.conf'

Kubernetes requires that packets traversing a network bridge are processed for filtering and for port forwarding. To achieve this, tunable parameters in the kernel bridge module are automatically set when the kubeadm package is installed and a sysctl file is created at /etc/sysctl.d/k8s.conf that contains the following lines:

net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1

If you change this file, or create anything similar yourself, run the following command to load the bridge tunable parameters:

sudo /sbin/sysctl -p /etc/sysctl.d/k8s.conf

Network Address Translation (NAT) is sometimes required when one or more Kubernetes worker nodes in a cluster are behind a NAT gateway. For example, you might want to have a control plane node in a secure company network while having other worker nodes in a publicly accessible demilitarized zone which is less secure. The control plane node would access the worker nodes through the worker node's NAT gateway. Or you might have a worker node in a legacy network that you want to use in the cluster that's primarily on a newer network. The NAT gateway, in these cases, translates requests for an IP address accessible to the Kubernetes cluster into the IP address on the subnet behind the NAT gateway.

Regardless of what switches or network equipment you use to set up the NAT gateway, you must configure the following for a node behind a NAT gateway:

• The node's interface behind the NAT gateway must have an public IP address using the /32 subnet mask that's reachable by the Kubernetes cluster. The /32 subnet restricts the subnet to one IP address, so that all traffic from the Kubernetes cluster flows through this public IP address.

• The node's interface must also include a private IP address behind the NAT gateway that the switch uses NAT tables to match the public IP address to.

For example, you can use the following command to add the reachable IP address on the ens5 interface:

sudo ip addr add 192.168.64.6/32 dev ens5 

You can then use the following command to add the private IP address on the same interface:

sudo ip addr add 192.168.192.2/18 dev ens5