Cluster Updates

Learn how to update a Kubernetes cluster by updating the Oracle Container Host for Kubernetes (OCK) image on each Kubernetes node.

This section shows you how to update nodes to the latest Kubernetes patch release, or to update them to the next Kubernetes minor release.

Patch releases include errata updates and might include Common Vulnerabilities and Exposures (CVE) fixes, Kubernetes updates, OS updates, and so on. An update to the next Kubernetes minor version is performed in the same way as patch updates, with one extra step to set the Kubernetes version number.

Oracle CNE delivers all updates through updated Oracle Container Host for Kubernetes (OCK) images. Updates are delivered through an OCK image that's specific to the Kubernetes minor version, for example for Kubernetes Release 1.31.

Each node periodically polls the container registry to check for updates to the OCK image it's running, or for an image for the target Kubernetes version if you're upgrading Kubernetes. When you set the Kubernetes version for an upgrade, the image for that version is pulled and staged on the nodes in the cluster. Patch updates are downloaded to each node automatically and don't need to be staged before a node update.

When an update is available, use the ocne node update command to reboot a node to use the new image. Running the ocne node update command for a node completes the following actions:

The node is drained (using the kubectl drain command) from the cluster. This evicts the pods from the node.
The host OCK image is installed on the node, and the node is restarted.
The node is returned to the cluster (using the kubectl uncordon command) and is made available to run pods.

Update nodes sequentially, starting with the control plane nodes.

Tip:

To save time, you can start the update process as soon as one of the control plane nodes has been annotated as having an update available.

You can update a Highly Available cluster without bringing the cluster down. As one control plane node is taken offline, another control plane node takes control of the cluster. In a cluster with a single control plane node, the control plane node is offline for a short time while the update is performed.

If applications are running on more than one worker node, they remain up, and available, during an update.

Best Practices for Cluster Updates

Learn about best practices for updating Kubernetes clusters.

The following list describes best practices to be followed when updating a Kubernetes cluster in a production environment:

Back up etcd database: In the rare event of an OCK image update failure, the update is rolled back to the previous OCK image. The host reboots into the previous OCK image and rejoins the cluster. However, despite such safeguards being in place, we recommend you follow best practice and back up the etcd database before updating a cluster.

Update control plane nodes before worker nodes: Always update the nodes in the control plane first, one node at a time. Confirm the update on the control plane node you're working on has completed, and that the node has rejoined the cluster, before starting an update on another node.
Update nodes immediately after staging a new Kubernetes version: When you stage an image for a Kubernetes minor upgrade, the cluster stops polling for patch updates for the current version. Therefore, apply the upgrade as soon as possible.
Check Kubernetes rules: Certain Kubernetes configurations might prevent a node from being taken offline for upgrade. For example, the minAvailable field of the PodDisruptionBudget object sets the minimum number of pods that must always be available. For a node to be taken offline, you might need to increase the number of running pods to exceed the number set in the minAvailable field. For more information about PodDisruptionBudgets see the upstream Kubernetes documentation.

Tip:

You can also use the --disable-eviction option with the ocne node update command to bypass PodDisruptionBudget and force pods to be deleted during the draining process. Use with caution.

Kubernetes Patch Updates

Describes updating to Kubernetes patch releases.

When an image update is detected, the image is automatically pulled, verified, and staged on each node. After the image has been staged, the node is annotated to show an update is available.

You can check whether nodes have an available update using the ocne cluster info command. When an OCK image is ready to install, the output of this command shows the Update Available field is set to true for a node.

Note:

We recommend you run the ocne cluster info command often to check for updates. If you miss an update, and a new one becomes available, the latest one is pulled and staged and ready to use. The latest patch image is always made available on the node. If you miss a patch update, you can install the latest.

When an update is staged, use the ocne node update command to reboot the node to use the new image.

Installing a Kubernetes Patch Release

Update the Oracle Container Host for Kubernetes (OCK) image on Kubernetes nodes to install the latest Kubernetes patch release.

Each node in a Kubernetes cluster periodically polls the container registry to check for patch updates to the Oracle Container Host for Kubernetes (OCK) image it's running. When an update is detected, the image is automatically pulled, verified, and staged on each node, and the nodes are annotated to show an update is available.

Confirm an update is available for the cluster nodes.
Use the ocne cluster info command to confirm the nodes are staged with an updated OCK image. Use the syntax:
```
ocne cluster info
[{-N|--node}] nodename, ...
[{-s|--skip-nodes }]
```
For more information on the syntax options, see Oracle Cloud Native Environment: CLI.

For example:
```
ocne cluster info
```
When an OCK image is available, staged, and ready to install, the output of this command shows the Update Available field to be true for a node.
Update the control plane nodes.
Update the control plane nodes, one node at a time, with the staged OCK image.

Use the ocne node update command to update each node. Use the syntax:
```
ocne node update 
[{-d|--delete-emptydir-data}]
[{-c|--disable-eviction}]
{-N|--node} name
[{-p|--pre-update-mode} mode]
[{-t|--timeout} minutes] 
```
For more information on the syntax options, see Oracle Cloud Native Environment: CLI.

For example:
```
ocne node update --node mynode
```
Replace mynode with the name of the control plane node.

Tip:

After each node is updated, use the ocne cluster info command to check the update is complete. Node updates are asynchronous. The update is complete only when the output to this command reports an update is no longer available for a node.
Update the worker nodes.

Use the ocne node update command to update each worker node.

Kubernetes Minor Updates

Describes updating to Kubernetes minor releases.

You can upgrade a cluster to the next Kubernetes minor version when an OCK image becomes available for that version. To do this, you use the ocne cluster stage command to set the target Kubernetes version.

The target Kubernetes version must be the next available minor version. For example, to upgrade from Kubernetes Release 1.29 to 1.31, first set the target Kubernetes release to 1.30 and update all the nodes, then set the target version to 1.31 and update the nodes again.

The nodes then poll the container registry for an OCK image for the target Kubernetes version. When an image is available the nodes pull and stage the image, and the nodes are annotated to show an update is available, in the same way as patch updates. Again, you then manually update each node using the ocne node update command.

Tip:

If the cluster is running on OCI, and provisioned with oci provider, you can also update to a Kubernetes minor version using the Kubernetes Cluster API controllers. Using the Cluster API to upgrade provisions new compute instances using the new OCK image, instead of upgrading existing nodes, one at a time. For information on upgrading clusters provisioned with the oci provider, see Upgrading to a Kubernetes Minor Release.

Upgrading to a Kubernetes Minor Release

Update the Oracle Container Host for Kubernetes (OCK) image on Kubernetes nodes to upgrade to the next Kubernetes minor release.

Upgrade a Kubernetes cluster to the next minor Kubernetes version when an Oracle Container Host for Kubernetes (OCK) image becomes available for that version. Use the ocne cluster stage command to set the target Kubernetes version.

Set the target Kubernetes version.
Use ocne cluster stage command to stage the target Kubernetes version. The syntax to use is:
```
ocne cluster stage 
[{-c|--config} path] 
[{-r|--os-registry} registry]
[{-t|--transport} transport]
{-v|--version} version
```
For more information on the syntax options, see Oracle Cloud Native Environment: CLI.

For example:
```
ocne cluster stage --version 1.31
```
Confirm an update is available for the cluster nodes.
Use the ocne cluster info command to confirm the nodes are staged with an updated OCK image. Use the syntax:
```
ocne cluster info
[{-N|--node}] nodename, ...
[{-s|--skip-nodes }]
```
For more information on the syntax options, see Oracle Cloud Native Environment: CLI.

For example:
```
ocne cluster info
```
When an OCK image is available, staged, and ready to install, the output of this command shows the Update Available field to be true for a node.
Update the control plane nodes.
Update the control plane nodes, one node at a time, with the staged OCK image.

Use the ocne node update command to update each node. Use the syntax:
```
ocne node update 
[{-d|--delete-emptydir-data}]
[{-c|--disable-eviction}]
{-N|--node} name
[{-p|--pre-update-mode} mode]
[{-t|--timeout} minutes] 
```
For more information on the syntax options, see Oracle Cloud Native Environment: CLI.

For example:
```
ocne node update --node mynode
```
Replace mynode with the name of the control plane node.

Tip:

After each node is updated, use the ocne cluster info command to check the update is complete. Node updates are asynchronous. The update is complete only when the output to this command reports an update is no longer available for a node.
Update the worker nodes.

Use the ocne node update command to update each worker node.
Confirm Kubernetes has been upgraded.
Use the kubectl get nodes command to confirm all nodes have been upgraded and are listed with the updated Kubernetes version.
```
kubectl get nodes
```