1. Deploying and Managing Oracle Identity Governance on Kubernetes
  2. Administering Oracle Identity Governance on Kubernetes
  3. Patching and Upgrading
  4. Upgrading from Oracle Identity Governance 12.2.1.4 to 14.1.2
  5. Restoring After a Failed Upgrade

16.2.13 Restoring After a Failed Upgrade

If the upgrade fails at any point, you can restore back to the Oracle Identity Governance 12c deployment using the following steps:
  1. Shut down the OIG 14c deployment using the following command:
    kubectl patch domain <domain> -n <domain_namespace> --type=merge --patch "{\"spec\": {\"serverStartPolicy\": \"Never\"}}"
    For example:
    kubectl patch domain governancedomain -n oigns --type=merge --patch "{\"spec\": {\"serverStartPolicy\": \"Never\"}}"
    The output will look similar to the following:
    domain.weblogic.oracle/governancedomain patched
  2. Run the following kubectl command to view the pods:
    kubectl get pods -n <domain_namespace>
    For example:
    kubectl get pods -n oigns
    The output will look similar to the following:
    NAME                           READY   STATUS        RESTARTS   AGE
governancedomain-adminserver   1/1     Terminating   0          16m
governancedomain-soa-server1   1/1     Terminating   0          13m
governancedomain-oim-server1   1/1     Terminating   0          5m22s

    The Administration Server pods and Managed Server pods will move to a STATUS of Terminating.

    After a few minutes, run the command again and make sure the pods should have disappeared before continuing.

  3. Restore the persistent volume from the backup taken before the upgrade:
    sudo cp -rp <persistent_volume>/governancedomainpv <persistent_volume>/governancedomain_bkp14c
    sudo rm -rf <persistent_volume>/governancedomainpv
    sudo cp -rp <persistent_volume>/governancedomainpv_bkp12c <persistent_volume>/governancedomain
    For example:
    sudo cp -rp /nfs_volumes/oig/governancedomainpv /nfs_volumes/oig/governancedomain_bkp14c
    sudo rm -rf /nfs_volumes/oig/governancedomainpv
    sudo cp -rp /nfs_volumes/oig/governancedomainpv_bkp12c /nfs_volumes/oig/governancedomain
  4. Restore the Oracle Database from the backup taken before the upgrade.
  5. Run the following command to update the deployment with the OIG 12c container image used previously: 
    kubectl patch domain %DOMAIN_UID% -n %NAMESPACE% --type merge  -p '{"spec":{"image":"%WEBLOGIC_IMAGE%"}}'
    For example:
    • If using Oracle Container Registry or your own container registry for your OIG container image:
      kubectl patch domain governancedomain -n oigns \
--type merge  -p '{"spec":{"image":"container-registry.oracle.com/middleware/oig_cpu:12.2.1.4-jdk8-ol8-<YYMMDD>"}}'
    • If you are not using a container registry and have loaded the image on each of the worker nodes:
      kubectl patch domain governancedomain -n oigns \
--type merge  -p '{"spec":{"image":"oracle/oig:12.2.1.4.0"}}'
      The output will look similar to the following:
      domain.weblogic.oracle/governancedomain patched
  6. Run the following command to start the OIG domain:
    kubectl patch domain.v9.weblogic.oracle "%DOMAIN_UID%" -n "%NAMESPACE%" \
--type=merge --patch "{\"spec\": {\"serverStartPolicy\": \"IfNeeded\"}}"
    For example:
    kubectl patch domain.v9.weblogic.oracle "governancedomain" -n "oigns" \
--type=merge --patch "{\"spec\": {\"serverStartPolicy\": \"IfNeeded\"}}"
    The output will look similar to the following:
    domain.weblogic.oracle/governancedomain patched
  7. Downgrade the WebLogic Kubernetes Operator to a version supported by OIG 12c. Follow the instructions at Updating the WebLogic Kubernetes Operator, but use a supported operator for OIG 12c.
  8. Run the following command to view the pods:
    kubectl get pods -n %NAMESPACE%
    For example:
    kubectl get pods -n oigns
    The output will look similar to the following:
    NAME                                                     READY   STATUS        RESTARTS   AGE
governancedomain-introspector-jwqxw                          1/1     Running     0          10s
    The introspect job will start, followed by the Administration Server pod, and then the OIG server pods. This process will take several minutes, so keep executing the command until all the pods are running with READY status 1/1:

    Note:

    Alternatively, you can add the watch flag, -w, which allows you watch the status of the pods as they change. 
    NAME                           READY   STATUS    RESTARTS   AGE
governancedomain-adminserver   1/1     Running   0          16m
governancedomain-soa-server1   1/1     Running   0          13m
governancedomain-oim-server1   1/1     Running   0          5m22s
  9. Once everything is running, check the consoles are accessible as per Validating the Domain URLs.