1. Oracle SOA Suite on Kubernetes
  2. Appendix
  3. Quick Start Deployment On-Premise
  4. Prepare a Virtual Machine for the Kubernetes Cluster

Prepare a Virtual Machine for the Kubernetes Cluster

For illustration purposes, these instructions are for Oracle Linux 8. If you are using a different flavor of Linux, you will need to adjust the steps accordingly.

Note:

These steps must be run with the root user, unless specified otherwise. Any time you see YOUR_USERID in a command, you should replace it with your actual userid.

  1. Prerequisites
    1. Choose the directories where your Kubernetes files will be stored. The Kubernetes directory is used for the /var/lib/kubelet file system and persistent volume storage.
      export kubelet_dir=/u01/kubelet
mkdir -p $kubelet_dir
ln -s $kubelet_dir /var/lib/kubelet
    2. Verify that IPv4 forwarding is enabled on your host.

      Note:

      Replace eth0 with the ethernet interface name of your compute resource if it is different. 
      /sbin/sysctl -a 2>&1|grep -s 'net.ipv4.conf.eth0.forwarding'
/sbin/sysctl -a 2>&1|grep -s 'net.ipv4.conf.lo.forwarding'
/sbin/sysctl -a 2>&1|grep -s 'net.ipv4.ip_nonlocal_bind'

      For example: Verify that all are set to 1:

      net.ipv4.conf.eth0.forwarding = 1
net.ipv4.conf.lo.forwarding = 1
net.ipv4.ip_nonlocal_bind = 1

      Solution: Set all values to 1 immediately:

      /sbin/sysctl net.ipv4.conf.eth0.forwarding=1
/sbin/sysctl net.ipv4.conf.lo.forwarding=1
/sbin/sysctl net.ipv4.ip_nonlocal_bind=1
    3. To preserve the settings permanently: Update the above values to 1 in files in /usr/lib/sysctl.d/, /run/sysctl.d/, and /etc/sysctl.d/.
    4. Verify the iptables rule for forwarding. Kubernetes uses iptables to handle many networking and port forwarding rules. A standard container installation may create a firewall rule that prevents forwarding. Verify if the iptables rule to accept forwarding traffic is set:
      /sbin/iptables -L -n | awk '/Chain FORWARD / {print $4}' | tr -d ")"

      If the output is “DROP”, then run the following command:

      /sbin/iptables -P FORWARD ACCEPT

      Verify if the iptables rule is properly set to “ACCEPT”:

      /sbin/iptables -L -n | awk '/Chain FORWARD / {print $4}' | tr -d ")"
    5. Disable and stop firewalld:
      systemctl disable firewalld
systemctl stop firewalld
  2. Install CRI-O and Podman

    Note:

    If you have already configured CRI-O and Podman, continue to Install and configure Kubernetes.
    1. Make sure that you have the right operating system version:
      uname -a
more /etc/oracle-release

      Example output:

      Linux xxxxxx 5.15.0-100.96.32.el8uek.x86_64 #2 SMP Tue Feb 27 18:08:15 PDT 2024 x86_64 x86_64 x86_64 GNU/Linux
Oracle Linux Server release 8.6
    2. Installing CRI-O:

      Add OLCNE( Oracle Cloud Native Environment ) Repository to dnf config-manager. This allows dnf to install the additional packages required for CRI-O installation.

      On Oracle Linux 8:

      dnf config-manager --add-repo https://yum.oracle.com/repo/OracleLinux/OL8/olcne18/x86_64

      On Oracle Linux 9:

      dnf config-manager --add-repo https://yum.oracle.com/repo/OracleLinux/OL9/olcne18/x86_64

      Install the cri-o:

      dnf install -y cri-o

      Note:

      Note: To install a different version of CRI-O or on a different operating system, see CRI-O Installation Instructions.
    3. Start the CRI-O service:

      Set up Kernel Modules and Proxies

      ### Enable kernel modules overlay and br_netfilter which are required for Kubernetes Container Network Interface (CNI) plugins
modprobe overlay
modprobe br_netfilter

### To automatically load these modules at system start up create config as below
cat <<EOF > /etc/modules-load.d/crio.conf
overlay
br_netfilter
EOF
sysctl --system

### Set the environmental variable CONTAINER_RUNTIME_ENDPOINT to crio.sock to use crio as the container runtime
export CONTAINER_RUNTIME_ENDPOINT=unix:///var/run/crio/crio.sock

### Setup Proxy for CRIO service
cat <<EOF > /etc/sysconfig/crio
http_proxy=http://REPLACE-WITH-YOUR-COMPANY-PROXY-HOST:PORT
https_proxy=http://REPLACE-WITH-YOUR-COMPANY-PROXY-HOST:PORT
HTTPS_PROXY=http://REPLACE-WITH-YOUR-COMPANY-PROXY-HOST:PORT
HTTP_PROXY=http://REPLACE-WITH-YOUR-COMPANY-PROXY-HOST:PORT
no_proxy=localhost,127.0.0.0/8,ADD-YOUR-INTERNAL-NO-PROXY-LIST,/var/run/crio/crio.sock
NO_PROXY=localhost,127.0.0.0/8,ADD-YOUR-INTERNAL-NO-PROXY-LIST,/var/run/crio/crio.sock
EOF

      Set the runtime for CRI-O

      ### Setting the runtime for crio
## Update crio.conf
vi /etc/crio/crio.conf
## Append following under [crio.runtime]
conmon_cgroup = "kubepods.slice"
cgroup_manager = "systemd"
## Uncomment following under [crio.network]
network_dir="/etc/cni/net.d"
plugin_dirs=[
    "/opt/cni/bin",
    "/usr/libexec/cni",
]

      Start the CRI-O Service

      ## Restart crio service
systemctl restart crio.service
systemctl enable --now crio
    4. Installing Podman:

      On Oracle Linux 8, if podman is not available, then install Podman and related tools with following command syntax:

      shell$ sudo dnf module install container-tools:ol8

      On Oracle Linux 9, if podman is not available, then install Podman and related tools with following command syntax:

      $ sudo dnf install container-tools

      Since the setup uses docker CLI commands, on Oracle Linux 8/9, install the podman-docker package if not available, that effectively aliases the docker command to podman,with following command syntax:

      $ sudo dnf install podman-docker
    5. Configure Podman rootless:

      For using podman with your User ID (Rootless environment), Podman requires the user running it to have a range of UIDs listed in the files /etc/subuid and /etc/subgid. Rather than updating the files directly, the usermod program can be used to assign UIDs and GIDs to a user with the following commands:

      $ sudo /sbin/usermod --add-subuids 100000-165535 --add-subgids 100000-165535
              <REPLACE_USER_ID>
            $ podman system migrate

      Note:

      The above podman system migrate needs to be executed with your User ID and not root.

      Verify the user-id addition

      cat /etc/subuid
cat /etc/subgid

      Expected similar output

      opc:100000:65536
<user-id>:100000:65536
  3. Install and Configure Kubernetes
    1. Add the external Kubernetes repository: 
      cat <<EOF | sudo tee /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://pkgs.k8s.io/core:/stable:/v1.28/rpm/
enabled=1
gpgcheck=1
gpgkey=https://pkgs.k8s.io/core:/stable:/v1.28/rpm/repodata/repomd.xml.key
exclude=kubelet kubeadm kubectl cri-tools kubernetes-cni
EOF
    2. Set SELinux in permissive mode (effectively disabling it):
      export PATH=/sbin:$PATH setenforce 0 sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config
    3. Export proxy and enable kubelet:
      ### Get the nslookup IP address of the master node to use with apiserver-advertise-address during setting up Kubernetes master
### as the host may have different internal ip (hostname -i) and nslookup $HOSTNAME
ip_addr=`nslookup $(hostname -f) | grep -m2 Address | tail -n1| awk -F: '{print $2}'| tr -d " "`
echo $ip_addr

### Set the proxies
export NO_PROXY=localhost,127.0.0.0/8,ADD-YOUR-INTERNAL-NO-PROXY-LIST,/var/run/crio/crio.sock,$ip_addr,.svc
export no_proxy=localhost,127.0.0.0/8,ADD-YOUR-INTERNAL-NO-PROXY-LIST,/var/run/crio/crio.sock,$ip_addr,.svc
export http_proxy=http://REPLACE-WITH-YOUR-COMPANY-PROXY-HOST:PORT
export https_proxy=http://REPLACE-WITH-YOUR-COMPANY-PROXY-HOST:PORT
export HTTPS_PROXY=http://REPLACE-WITH-YOUR-COMPANY-PROXY-HOST:PORT
export HTTP_PROXY=http://REPLACE-WITH-YOUR-COMPANY-PROXY-HOST:PORT

### Install the kubernetes components and enable the kubelet service so that it automatically restarts on reboot
dnf install -y kubeadm kubelet kubectl
systemctl enable --now kubelet
    4. Ensure net.bridge.bridge-nf-call-iptables is set to 1 in your sysctl to avoid traffic routing issues:
      cat <<EOF >  /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
EOF
sysctl --system
    5. Disable swap check:
      sed -i 's/KUBELET_EXTRA_ARGS=/KUBELET_EXTRA_ARGS="--fail-swap-on=false"/' /etc/sysconfig/kubelet
cat /etc/sysconfig/kubelet
### Reload and restart kubelet
systemctl daemon-reload
systemctl restart kubelet
    6. Pull the images using crio:
      kubeadm config images pull --cri-socket unix:///var/run/crio/crio.sock
  4. Set up Helm
    1. Install Helm v3.10.2+.

      1. a. Download Helm from https://github.com/helm/helm/releases.

      For example, to download Helm v3.10.2:

      wget https://get.helm.sh/helm-v3.10.2-linux-amd64.tar.gz

      2. Unpack tar.gz:

      tar -zxvf helm-v3.10.2-linux-amd64.tar.gz

      Find the Helm binary in the unpacked directory, and move it to its desired destination:

      mv linux-amd64/helm /usr/bin/helm
    2. Run helm version to verify its installation:
      helm version
  version.BuildInfo{Version:"v3.10.2", GitCommit:"50f003e5ee8704ec937a756c646870227d7c8b58", GitTreeState:"clean", GoVersion:"go1.18.8"}