Oracle Cloud Infrastructure Documentation

Cluster Autoscaler

When you enable the Cluster Autoscaler add-on, you can pass the following key/value pairs as arguments.

Configuration Arguments Common to all Cluster Add-ons
Key (API and CLI) Key's Display Name (Console) Description Required/Optional Default Value Example Value
affinity affinity

A group of affinity scheduling rules.

JSON format in plain text or Base64 encoded.

 Optional null null
nodeSelectors node selectors

You can use node selectors and node labels to control the worker nodes on which add-on pods run.

For a pod to run on a node, the pod's node selector must have the same key/value as the node's label.

Set nodeSelectors to a key/value pair that matches both the pod's node selector, and the worker node's label.

JSON format in plain text or Base64 encoded.

 Optional null {"foo":"bar", "foo2": "bar2"}

The pod will only run on nodes that have the foo=bar or foo2=bar2 label.
numOfReplicas numOfReplicas The number of replicas of the add-on deployment.

(For CoreDNS, use nodesPerReplica instead.)

 Required 1

Creates one replica of the add-on deployment per cluster.

 2

Creates two replicas of the add-on deployment per cluster.
rollingUpdate rollingUpdate

Controls the desired behavior of rolling update by maxSurge and maxUnavailable.

JSON format in plain text or Base64 encoded.

 Optional null null
tolerations tolerations

You can use taints and tolerations to control the worker nodes on which add-on pods run.

For a pod to run on a node that has a taint, the pod must have a corresponding toleration.

Set tolerations to a key/value pair that matches both the pod's toleration, and the worker node's taint.

JSON format in plain text or Base64 encoded.

 Optional null [{"key":"tolerationKeyFoo", "value":"tolerationValBar", "effect":"noSchedule", "operator":"exists"}]

Only pods that have this toleration can run on worker nodes that have the tolerationKeyFoo=tolerationValBar:noSchedule taint.
topologySpreadConstraints topologySpreadConstraints

How to spread matching pods among the given topology.

JSON format in plain text or Base64 encoded.

 Optional null null
Configuration Arguments Specific to this Cluster Add-on
Key (API and CLI) Key's Display Name (Console) Description Required/Optional Default Value Example Value
annotations annotations

Annotations to pass to the Cluster Autoscaler deployment.

For example, "{\"prometheus.io/scrape\":\"true\",\"prometheus.io/port\":\"8086\"}"

JSON format in plain text or Base64 encoded.

 Optional ""
authType authType The authentication type the Cluster Autoscaler uses while making requests, as one of:
  • instance specifies instance principal
  • workload specifies workload identity
 Required instance
balanceSimilarNodeGroups balanceSimilarNodeGroups Detect similar node groups and balance the number of nodes between them. Optional false
balancingIgnoreLabel balancingIgnoreLabel Define a node label that should be ignored when considering node group similarity. One label per flag occurrence. The format is label1, label2. Optional ""
balancingLabel balancingLabel Define a node label to use when comparing node group similarity. If set, all other comparison logic is disabled, and only labels are considered when comparing groups. One label per flag occurrence. The format is label1, label2. Optional ""
cluster-autoscaler.ContainerResources cluster-autoscaler container resources

You can specify the resource quantities that the add-on containers request, and set resource usage limits that the add-on containers cannot exceed.

JSON format in plain text or Base64 encoded.

 Optional null {"limits": {"cpu": "500m", "memory": "200Mi" }, "requests": {"cpu": "100m", "memory": "100Mi"}}

Create add-on containers that request 100 milllicores of CPU, and 100 mebibytes of memory. Limit add-on containers to 500 milllicores of CPU, and 200 mebibytes of memory.
cordonNodeBeforeTerminating (Not shown in Console) Should CA cordon nodes before terminating during downscale process. Optional false
coresTotal (Not shown in Console) Minimum and maximum number of cores in cluster, in the format <min>:<max>. Cluster autoscaler will not scale the cluster beyond these numbers. Optional 0:320000
daemonsetEvictionForEmptyNodes (Not shown in Console) Whether DaemonSet pods will be gracefully terminated from empty nodes. Optional false
daemonsetEvictionForOccupiedNodes (Not shown in Console) Whether DaemonSet pods will be gracefully terminated from non-empty nodes. Optional true
debuggingSnapshotEnabled (Not shown in Console) Whether the debugging snapshot of cluster autoscaler feature is enabled. Optional false
emitPerNodegroupMetrics (Not shown in Console) If true, emit per node group metrics. Optional false
enforceNodeGroupMinSize (Not shown in Console) Should CA scale up the node group to the configured min size if needed. Optional false
estimator (Not shown in Console) Type of resource estimator to be used in scale up. Optional binpacking
expander expander Type of node group expander to be used in scale up.

Note that expander=price is not supported.

 Optional random
expendablePodsPriorityCutoff (Not shown in Console) Pods with priority below cutoff will be expendable. They can be killed without any consideration during scale down and they don't cause scale up. Pods with null priority (PodPriority disabled) are non-expendable. Optional -10
ignoreDaemonsetsUtilization (Not shown in Console) Whether DaemonSet pods will be ignored when calculating resource utilization for scaling down. Optional false
ignoreMirrorPodsUtilization (Not shown in Console) Whether Mirror pods will be ignored when calculating resource utilization for scaling down. Optional false
leaderElect (Not shown in Console) Start a leader election client and gain leadership before executing the main loop. Enable this when running replicated components for high availability. Optional true
leaderElectLeaseDuration (Not shown in Console) The duration that non-leader candidates will wait after observing a leadership renewal until attempting to acquire leadership of a led but un-renewed leader slot. This is effectively the maximum duration that a leader can be stopped before it is replaced by another candidate. This is only applicable if leader election is enabled. Optional 15s
leaderElectRenewDeadline (Not shown in Console) The interval between attempts by the active cluster autoscaler to renew a leadership slot before it stops leading. This must be less than or equal to the lease duration. This is only applicable if leader election is enabled. Optional 10s
leaderElectResourceLock (Not shown in Console) The type of resource object that is used for locking during leader election. Supported options are leases (default), endpoints, endpointsleases, configmaps, and configmapsleases. Optional leases
leaderElectRetryPeriod (Not shown in Console) The duration the clients should wait between attempting acquisition and renewal of a leadership. This is only applicable if leader election is enabled. Optional 2s
maxAutoprovisionedNodeGroupCount (Not shown in Console) The maximum number of auto-provisioned groups in the cluster. Optional 15
maxEmptyBulkDelete maxEmptyBulkDelete Maximum number of empty nodes that can be deleted at the same time. Optional 10
maxFailingTime (Not shown in Console) Maximum time from last recorded successful autoscaler run before automatic restart. Optional 15m
maxGracefulTerminationSec (Not shown in Console) Maximum number of seconds CA waits for pod termination when trying to scale down a node. Optional 600
maxInactivity (Not shown in Console) Maximum time from last recorded autoscaler activity before automatic restart. Optional 10m
maxNodeProvisionTime maxNodeProvisionTime Maximum time CA waits for node to be provisioned. Optional 15m
maxNodesTotal (Not shown in Console) Maximum number of nodes in all node pools. Cluster autoscaler will not grow the cluster beyond this number. Optional 0
maxTotalUnreadyPercentage (Not shown in Console) Maximum percentage of unready nodes in the cluster. After this is exceeded, CA halts operations. Optional 45
memoryTotal (Not shown in Console) Minimum and maximum number of gigabytes of memory in cluster, in the format <min>:<max>. Cluster autoscaler will not scale the cluster beyond these numbers. Optional 0:6400000
minReplicaCount (Not shown in Console) Minimum number or replicas that a replica set or replication controller should have to allow their pods deletion in scale down. Optional 0
nodes nodes

A list of Minimum number of nodes, Maximum number of nodes, and the OCID of the nodepool to be managed by cluster autoscaler.

The format is <min>:<max>:<node-pool1-ocid>, <min>:<max>:<node-pool2-ocid>.

Use either nodes or nodeGroupAutoDiscovery, but not both.

JSON format in plain text or Base64 encoded.

 Required, if nodeGroupAutoDiscovery not set ""
nodeGroupAutoDiscovery (Not shown in Console)

A list of tag key/value pairs, Minimum number of nodes, Maximum number of nodes, and the OCID of the compartment in which the node pool to be managed by cluster autoscaler is located.

The format is compartmentId:<compartment-ocid>,nodepoolTags:<tagKey1>=<tagValue1>&<tagKey2>=<tagValue2>,min:<min-nodes>,max:<max-nodes>

Use either nodeGroupAutoDiscovery or nodes, but not both.

Supported with Cluster Autoscaler version 1.30.3, version 1.31.1, and version 1.32.0, and later.

JSON format in plain text or Base64 encoded.

 Required, if nodes not set
okTotalUnreadyCount (Not shown in Console) Number of allowed unready nodes, irrespective of maxTotalUnreadyPercentage. Optional 3
recordDuplicatedEvents (Not shown in Console) Enable the autoscaler to print duplicated events within a 5 minute window. Optional false
scaleDownCandidatesPoolMinCount (Not shown in Console)

Minimum number of nodes that are considered as additional non empty candidates for scale down when some candidates from previous iteration are no longer valid. When calculating the pool size for additional candidates we take.

max(#nodes * scale-down-candidates-pool-ratio,
              scale-down-candidates-pool-min-count)
 Required 50
scaleDownCandidatesPoolRatio (Not shown in Console) A ratio of nodes that are considered as additional non-empty candidates for scale down when some candidates from previous iteration are no longer valid. Lower value means better CA responsiveness but possible slower scale down latency. Higher value can affect CA performance with big clusters (hundreds of nodes). Set to 1.0 to turn this heuristics off - CA will take all nodes as additional candidates. Required 0.1
scaleDownDelayAfterAdd scaleDownDelayAfterAdd How long after scale up that scale down evaluation resumes. Required 10m
scaleDownDelayAfterDelete (Not shown in Console) How long after node deletion that scale down evaluation resumes, defaults to scan-interval. Required 10s
scaleDownDelayAfterFailure (Not shown in Console) How long after scale down failure that scale down evaluation resumes. Required 3m
scaleDownEnabled scaleDownEnabled Should CA scale down the cluster. Optional true
scaleDownNonEmptyCandidatesCount (Not shown in Console) Maximum number of non empty nodes considered in one iteration as candidates for scale down with drain. Lower value means better CA responsiveness but possible slower scale down latency. Higher value can affect CA performance with big clusters (hundreds of nodes). Set to non positive value to turn this heuristic off - CA will not limit the number of nodes it considers. Required 30
scaleDownUnneededTime scaleDownUnneededTime How long a node should be unneeded before it is eligible for scale down. Required 10m
scaleDownUnreadyTime (Not shown in Console) How long an unready node should be unneeded before it is eligible for scale down. Required 20m
scaleDownUtilizationThreshold (Not shown in Console) Node utilization level, defined as sum of requested resources divided by capacity, below which a node can be considered for scale down. Required 0.5
scanInterval scanInterval How often cluster is re-evaluated for scale up or down. Optional 10s
skipNodesWithCustomControllerPods (Not shown in Console) If true, cluster autoscaler will never delete nodes with pods owned by custom controllers. Optional true
skipNodesWithLocalStorage (Not shown in Console) If true, cluster autoscaler will never delete nodes with pods with local storage, e.g. EmptyDir or HostPath. Optional true
skipNodesWithSystemPods (Not shown in Console) If true, cluster autoscaler will never delete nodes with pods from kube-system (except for DaemonSet or mirror pods). Optional true
statusConfigMapName (Not shown in Console) The name of the status ConfigMap that CA writes. Optional cluster-autoscaler-status
stderrthreshold (Not shown in Console) The log severity threshold, beyond which logs are sent to stderr. For example, if you set this to error, all logs with a severity higher than error are sent to stderr. Optional info
unremovableNodeRecheckTimeout unremovableNodeRecheckTimeout The timeout before we check again a node that couldn't be removed before. Required 5m
v (Not shown in Console) The number for the verbosity of logging. Optional 0
writeStatusConfigmap (Not shown in Console) Should CA write status information to a configmap. Optional true