Verifying the Horizontal Pod Autoscaler

12.5 Verifying the Horizontal Pod Autoscaler

To verify the Horizontal Pod Autoscaler (HPA) works, perform the following steps:

Check the current status of the Oracle Unified Directory (OUD) servers:

kubectl get pods -n oudns

The output will look similar to the following:

NAME                              READY   STATUS      RESTARTS   AGE
oud-ds-rs-0                       1/1     Running     0          5h15m
oud-ds-rs-1                       1/1     Running     0          5h9m
oud-ds-rs-2                       1/1     Running     0          5h2m
oud-pod-cron-job-28242120-bwtcz   0/1     Completed   0          61m
oud-pod-cron-job-28242150-qf8fg   0/1     Completed   0          31m
oud-pod-cron-job-28242180-q69lm   0/1     Completed   0          92s

In the above example. oud-ds-rs-0, oud-ds-rs-1, and oud-ds-rs-2 are running.

To test HPA can scale up the OUD servers, run the following commands:
```
kubectl exec --stdin --tty oud-ds-rs-0 -n oudns -- /bin/bash
```
This will take you inside a bash shell inside the oud-ds-rs-0 pod:
```
[oracle@oud-ds-rs-0 oracle]$
```
Inside the bash shell, run the following command to increase the load on the CPU:
```
[oracle@oud-ds-rs-0 oracle]$ dd if=/dev/zero of=/dev/null
```
This command will continue to run in the foreground.

Repeat the step above for the oud-ds-rs-1 pod:

kubectl exec --stdin --tty oud-ds-rs-1 -n oudns -- /bin/bash
[oracle@oud-ds-rs-1 oracle]$
[oracle@oud-ds-rs-1 oracle]$ dd if=/dev/zero of=/dev/null

In a command window outside the bash shells, run the following command to view the current CPU usage:
```
kubectl get hpa -n oudns
```
The output will look similar to the following:
```
NAME          REFERENCE               TARGETS    MINPODS   MAXPODS   REPLICAS   AGE
oud-sts-hpa   StatefulSet/oud-ds-rs   125%/70%   3         5         3          5m15s
```
In the above example the CPU has increased to 125%. As this is above the 70% limit, the autoscaler increases the replicas by starting additional OUD servers.

Run the following to see if any more OUD servers are started:

kubectl get pods -n oudns

The output will look similar to the following:

NAME                              READY   STATUS      RESTARTS   AGE
oud-ds-rs-0                       1/1     Running     0          5h50m
oud-ds-rs-1                       1/1     Running     0          5h44m
oud-ds-rs-2                       1/1     Running     0          5h37m
oud-ds-rs-3                       1/1     Running     0          9m29s
oud-ds-rs-4                       1/1     Running     0          5m17s
oud-pod-cron-job-28242150-qf8fg   0/1     Completed   0          66m
oud-pod-cron-job-28242180-q69lm   0/1     Completed   0          36m
oud-pod-cron-job-28242210-kn7sv   0/1     Completed   0          6m28s

In the example above one more OUD server has started (oud-ds-rs-4).

Note:

It may take some time for the server to appear and start. Once the server is at READY status of 1/1, the server is started.

To stop the load on the CPU, in both bash shells, issue a Control C, and then exit the bash shell:
```
[oracle@oud-ds-rs-0 oracle]$ dd if=/dev/zero of=/dev/null
^C
[oracle@oud-ds-rs-0 oracle]$ exit
```

Run the following command to view the current CPU usage:

kubectl get hpa -n oudns

The output will look similar to the following:

NAME          REFERENCE               TARGETS   MINPODS   MAXPODS   REPLICAS   AGE
oud-sts-hpa   StatefulSet/oud-ds-rs   4%/70%    3         5         5          40m

In the above example CPU has dropped to 4%. As this is below the 70% threshold, you should see the autoscaler scale down the servers:

kubectl get pods -n oudns

The output will look similar to the following:

NAME                              READY   STATUS        RESTARTS   AGE
oud-ds-rs-0                       1/1     Running       0          5h54m
oud-ds-rs-1                       1/1     Running       0          5h48m
oud-ds-rs-2                       1/1     Running       0          5h41m
oud-ds-rs-3                       1/1     Running       0          13m
oud-ds-rs-4                       1/1     Terminating   0          8m27s
oud-pod-cron-job-28242150-qf8fg   0/1     Completed     0          70m
oud-pod-cron-job-28242180-q69lm   0/1     Completed     0          40m
oud-pod-cron-job-28242210-kn7sv   0/1     Completed     0          10m

Eventually, the extra OUD pod will disappear:

NAME                              READY   STATUS        RESTARTS   AGE
oud-ds-rs-0                       1/1     Running       0          5h57m
oud-ds-rs-1                       1/1     Running       0          5h51m
oud-ds-rs-2                       1/1     Running       0          5h44m
oud-ds-rs-3                       1/1     Running       0          16m
oud-pod-cron-job-28242150-qf8fg   0/1     Completed     0          73m
oud-pod-cron-job-28242180-q69lm   0/1     Completed     0          43m
oud-pod-cron-job-28242210-kn7sv   0/1     Completed     0          13m