If the OIG domain creation fails, run the following to diagnose the issue:

kubectl logs <domain_job> -n <domain_namespace>

For example:

kubectl logs governancedomain-create-fmw-infra-sample-domain-job-c6vfb -n oigns

Also run:

kubectl describe pod <domain_job> -n <domain_namespace>

For example:

kubectl describe pod governancedomain-create-fmw-infra-sample-domain-job-c6vfb -n oigns

Using the output you should be able to diagnose the problem and resolve the issue.

Failed to start container "create-fmw-infra-sample-domain-job": Error response from daemon: error while creating mount source path
'/nfs_volumes/oig/governancedomainpv ': mkdir /nfs_volumes/oig/governancedomainpv : permission denied

Once you have diagnosed the problem, clean down the failed domain creation by following:

• Deleting the OIG Domain
• Deleting RCU Schemas
• Deleting Persistent Volume Contents

Then follow the instructions again in Creating OIG Domains Using WLST Offline Scripts

If the domain creation fails while creating domain resources using the domain.yaml file, run the following steps to diagnose the issue:

1. Check the domain events, by running the following command:

   kubectl describe domain <domain name> -n <domain_namespace>
   

   For example:

   kubectl describe domain governancedomain -n oigns
   

   Using the output, you should be able to diagnose the problem and resolve the issue.
2. If the introspect job fails due to validation errors, then you can recreate the domain resources using the commands:

   kubectl delete -f domain.yaml

   kubectl create -f domain.yaml
   

3. If the domain creation fails because of database issues:
   1. Create a helper pod:

       kubectl run --image=container-registry.oracle.com/middleware/oig_cpu:14.1.2.1.0-jdk17-ol8-<YYMMDD> --image-pull-policy="IfNotPresent" --overrides='{"apiVersion": "v1","spec":{"imagePullSecrets": [{"name": "orclcred"}]}}' helper -n oigns -- sleep infinity
      

   2. Once you have diagnosed the problem, clean down the failed domain creation by following:
      • Deleting the OIG Domain
      • Deleting RCU Schemas
      • Deleting Persistent Volume Contents
   3. Execute the steps in Creating OIG Domains Using WDT Models again.

   Note:

   You might need to recreate the domain creation image depending upon the errors. Domain creation logs are stored in <persistent_volume>/domains/wdt-logs.
4. If there is any issues bringing up the administration server, or OIG managed server pods, you can run the following to check the logs:

   kubectl logs <pod> -n <domain_namespace>

   For example:

   kubectl logs governancedomain-adminserver -n oigns

   If the above does not give any information you can also run:

   kubectl describe pod <pod> -n <domain_namespace>

   For example:

   kubectl describe pod governancedomain-adminserver -n oigns

For more details related to debugging issues, refer to Domain Debugging.

If the restart is due to resource (CPU in this case) limit issue, then the CPU parameter need to be adjusted as needed:

kubectl edit cluster <cluster> -n <domain_namespace>

For example:

kubectl edit cluster governancedomain-oim-cluster -n oigns

In the edit session change the CPU parameter as follows:

spec:
  clusterName: oim_cluster
  replicas: 1
  serverPod:
    env:
    - name: USER_MEM_ARGS
      value: '-Djava.security.egd=file:/dev/./urandom -Xms4096m -Xmx10240m -Dweblogic.rjvm.allowUnknownHost=true '
    livenessProbe:
      failureThreshold: 3
      periodSeconds: 60
      timeoutSeconds: 60
    readinessProbe:
      failureThreshold: 3
      periodSeconds: 60
      timeoutSeconds: 60
    resources:
      limits:
        cpu: 1700m
        memory: 6Gi
      requests:
        cpu: 500m
        memory: 4Gi

If the OIG domain patching fails when running patch_oig_domain.sh, run the following to diagnose the issue:

kubectl describe domain <domain name> -n <domain_namespace>

For example:

kubectl describe domain governancedomain -n oigns

If the script is stuck at the following message for a long time:

[INFO] Waiting for weblogic pods to be ready..This may take several minutes, do not close the window. Check log /scratch/OIGK8Slatest/fmw-kubernetes/OracleIdentityGovernance/kubernetes/domain-lifecycle/log/oim_patch_log-<DATE>/monitor_weblogic_pods.log for progress

Run the following command to diagnose the issue:

kubectl get pods -n <domain_namespace>

For example:

kubectl get pods -n oigns

Run the following to check the logs of the Admininstration Server, SOA server or OIM server pods, as there may be an issue that is not allowing the domain pods to start properly:

kubectl logs <pod> -n oigns

If the above does not glean any information you can also run:

kubectl describe pod <pod> -n oigns

Further diagnostic logs can also be found under the $WORKDIR/kubernetes/domain-lifecycle.