UIM communicates with the Topology Service using REST APIs and Kafka Message Bus. UIM is the Producer for Create, Update and Delete operations from UIM that impact Topology. UIM uses REST APIs to communicate directly with the UTIA Service while building the messages and can also continue processing when the Topology Service is unavailable.

The UTIA service is a consumer for inventory system and assurance system messages. UTIA processes multiple message events including TopologyNodeCreate, TopologyNodeUpdate, TopologyNodeDelete, TopologyEdgeCreate, TopologyEdgeUpdate, TopologyEdgeDelete, TopologyFaultEventCreate, TopologyFaultEventUpdate, TopologyPerformanceEventCreate, TopologyPerformanceEventUpdate.

The service information is updated using the TopologyProfileCreate, TopologyProfileUpdate, and TopologyProfileDelete events.

The UTIA Service communicates to the Oracle Databases using the Oracle Property Graph feature with PGQL and standard SQL. It can communicate directly to the database or with the In-Memory Graph for high performance operations. This converged database feature of Oracle Database makes it possible to utilize the optimal processing method with a single database. The Graph Database is isolated and a separate Pluggable Database (PDB) from the UIM Database but runs on the same 19c version for simplified licensing.

The UTIA Service also uses the Oracle Labs Parallel Graph AnalytiX (PGX) In-Memory database. The PGX server is used for Path Analysis and is configured for periodic updates.

UTIA provides a graphical representation of topology where you can see your inventory and its relationships at the level of detail that meets your needs. UTIA is built using Oracle Redwood Design System.

You require the following prerequisites for creating UTIA images:

• Podman on the build machine if Linux version is greater than or equal to 8.
• Docker on the build machine if Linux version is lesser than 8
• Unified Topology Builder Toolkit (ref about the deliverables)
• Install Maven and update path variable with Maven Home.

  Set PATH variable export PATH=$PATH:$MAVEN_HOME/bin

• Java, installed with JAVA_HOME set in the environment.

  Set PATH variable export PATH=$PATH:$JAVA_HOME/bin

• Bash, to enable the `<tab>` command complete feature.

See UIM Compatibility Matrix for details about the required and supported versions of these prerequisite software.

The dependency manifest file describes the input that goes into the Unified Topology images. It is consumed by the image build process. The default configuration in the latest manifest file provides the necessary components for creating the Unified Topology images easily. See "About the Manifest File" for more information.

The Unified Topology image builder creates images with names and tags based on the settings in the manifest file. By default, this results in the following images:

• uim-7.5.1.2.0-unified-topology-base-1.0.0.2.0:latest
• uim-7.5.1.2.0-unified-topology-api-1.0.0.2.0:latest
• uim-7.5.1.2.0-unified-pgx-1.0.0.2.0:latest
• uim-7.5.1.2.0-unified-topology-ui-1.0.0.2.0:latest
• uim-7.5.1.2.0-unified-topology-dbinstaller-1.0.0.2.0:latest
• uim-7.5.1.2.0-unified-topology-consumer-1.0.0.2.0:latest

Build container images for the following using the Unified Topology cloud native Image Builder:

• Unified Topology Core application
• Unified PGX application
• Unified Topology User Interface application
• Unified Topology database installer

See "Deployment Toolkits" to download the Common cloud native toolkit archive file. Set the variable for the installation directory by running the following command, where $WORKSPACEDIR is the installation directory of the COMMON cloud native toolkit:

export COMMON_CNTK=$WORKSPACEDIR/common-cntk

Unified Topology Service relies on access to certain environment variables to run seamlessly. Ensure the following variables are set in your environment:

• Path to your common cloud native toolkit
• Traefik namespace

To set the environment variables:

1. Set the COMMON_CNTK variable to the path of directory where common cloud native toolkit is extracted as follows:

   $ export COMMON_CNTK=$WORKSPACEDIR/common-cntk

2. Set the TRAEFIK_NS variable for Traefik namespace as follows:

   $ export TRAEFIK_NS=Treafik Namespace

3. Set the TRAEFIK_CHART_VERSION variable for Traefik helm chart version. Refer UIM Compatibility Matrix for appropriate version. The following is a sample for Traefik chart version 15.1.0.

   $ export TRAEFIK_CHART_VERSION=15.1.0

4. Set SPEC_PATH variable to the location where application and database yamls are copied as follows:

   $ export SPEC_PATH=$WORKSPACEDIR/utia_spec_dir

After you set the environment variables, register the namespace.

To register the namespace, run the following command:

$COMMON_CNTK/scripts/register-namespace.sh -p sr -t targets
# For example, $COMMON_CNTK/scripts/register-namespace.sh -p sr -t traefik
# Where the targets are separated by a comma without extra spaces

You must store sensitive data and credential information in the form of Kubernetes Secrets that the scripts and Helm charts in the toolkit consume. Managing secrets is out of the scope of the toolkit and must be implemented while adhering to your organization's corporate policies. Additionally, Unified Topology service does not establish password policies.

As a prerequisite to use the toolkit for either installing the Unified Topology database or creating a Unified Topology instance, you must create secrets to access the following:

• UTIA Database
• UIM Instance Credentials
• Secret for UTIA API
• Secret for UTIA UI
• OAM Authentication server details
• Truststore secret for OAM server

The toolkit provides sample scripts to perform this. These scripts should be used for manual and faster creation of an instance. It does not support any automated process for creating instances. The scripts also illustrate both the naming of the secret and the layout of the data within the secret that Unified Topology requires. You must create secrets before running the install-database.sh or create-applications.sh scripts.

To install the Unified Topology schema:

1. Update values under unified-topology-dbinstaller in $SPEC_PATH/sr/quick/database.yaml file with values required for unified topology schema creation.

   Note:

   • The YAML formatting is case-sensitive. Use a YAML editor to ensure that you do not make any syntax errors while editing. Follow the indentation guidelines for YAML.
   • Before changing the default values provided in the specification file, verify that they align with the values used during PDB creation. For example, the default tablespace name should match the value used when PDB is created.
2. Edit the database.yaml file and update the DB installer image to point to the location of your image as follows:

   unified-topology-dbinstaller:
     dbinstaller:
       image:  DB_installer_image_in_your_repo
       tag: DB_installer image tag in your repo
   

3. If your environment requires a password to download the container images from your repository, create a Kubernetes secret with the Docker pull credentials. See "Kubernetes documentation" for details. Refer the secret name in the database.yaml. Provide image pull secret and image pull policy details.

   unified-topology-dbinstaller:
     imagePullPolicy: Never
   # The image pull access credentials for the "docker login" into Docker repository, as a Kubernetes secret.
   # Uncomment and set if required.
   #  imagePullSecret: ""
   

4. Run the following script to start the Unified Topology DB installer, which instantiates a Kubernetes pod resource. The pod resource lives until the DB installation operation completes.

   $COMMON_CNTK/scripts/install-database.sh -p sr -i quick -f $SPEC_PATH/sr/quick/database.yaml -a unified-topology -c 1

5. You can run the script with -h to see the available options.
6. Check the console to see if the DB installer is installed successfully.
7. If the installation has failed, run the following command to review the error message in the log:

   kubectl logs -n sr sr-quick-unified-topology-dbinstaller

8. Clear the failed pod by running the following command:

   helm uninstall sr-quick-unified-topology-dbinstaller -n sr

9. Run the install-database script again to install the Unified Topology DB installer.

The applications.yaml file is a Helm override values file to override default values of unified topology chart. Update values under chart unified-topology in $SPEC_PATH/<PROJECT>/<INSTANCE>/applications.yaml to override the default values.

The applications.yaml provides a section for values that are common for all microservices. Provide Values under that common section and it is reflected for all services.

To configure the project specification:

1. Edit the applications.yaml to provide the image in your repository (name and tag) by running the following command:

   vi $SPEC_PATH/<PROJECT>/<INSTANCE>/applications.yaml
    
   ** edit the topologyAPiName, pgxName, uiName to reflect the Unified Topology image names and location in your docker repository
   ** edit the topologyAPiTag, pgxTag, uiTag to reflect the Unified Topology image names and location in your docker repository
    
   unified-topology:
     	image:     
       topologyApiName: uim-7.5.1.2.0-unified-topology-api-1.0.0.2.0
       pgxName: uim-7.5.1.2.0-unified-pgx-1.0.0.2.0
       uiName: uim-7.5.1.2.0-unified-topology-ui-1.0.0.2.0
       topologyConsumerName: uim-7.5.1.2.0-unified-topology-consumer-1.0.0.2.0
       topologyApiTag: latest
       pgxTag: latest
       uiTag: latest
       topologyConsumerTag: latest

2. If your environment requires a password to download the container images from your repository, create a Kubernetes secret with the Docker pull credentials. See the "Kubernetes documentation" for details. See the secret name in the applications.yaml for more information.

   # The image pull access credentials for the "docker login" into Docker repository, as a Kubernetes secret.
   # uncomment and set if required.
    
   unified-topology:
   #  imagePullSecret:
   #    imagePullSecrets:
   #      - name: regcred
   

3. Set Pull Policy for unified topology images in applications.yaml. Set pullPolicy to Always in case image is updated.

   unified-topology:
     image:
       pullPolicy: Never
   

4. Update loadbalancerhost, loadbalancerpost in applications.yaml. If there is no external loadbalancer configured for the instance change the value of loadbalancerport to the default Traefik NodePort and loadbalancerhost to the worker node IP. If TLS is enabled on Unified Topology Traefik NodePort is 30443 and if TLS is disabled it is 30305.

   If you use Oracle Cloud Infrastructure LBaaS, or any other external load balancer, if TLS is enabled set loadbalancerport to 443 else set loadbalancerport to 80 and update the value for loadbalancerhost appropriately. loadbalancerhost and loadbalancerport are common values for all services.

   #provide loadbalancer host and post
    
   loadbalancerhost: 100.76.135.13
   loadbalancerport: 30305
   

5. To enable authentication, set flag authentication.enabled to true. Provide OHS server hostname If authentication is enabled. authentication.enabled flag and ohsHostname are common values for all services.

   # The enabled flag is to enable or disable authentication
   authentication:
     enabled: true
    
   #Provide ohs server hostname
   ohsHostname: <instance>.<project>.ohs.<oam-host-suffix>
   

6. If Authentication is not enabled on UTIA and want to integrate UTIA with traditonal SSL enabled UIM, you have to create inventorySSL secret and enable the inventorySSL flag in applications.yaml as shown below:

   # make it true if using on prem inventory with ssl port enabled and authentication is not enabled on topology
   # always false for Cloud Native inventory
   # not required in production environment
   isInventorySSL: true

Sample configuration files topology-static-config.yaml.sample, topology-dynamic-config.yaml.sample are provided as follows:

• The sample files for Topology API service are added in $COMMON_CNTK/charts/unified-topology-app/charts/unified-topology/config/topology-api.
• The sample files for Topology Consumer service are added in $COMMON_CNTK/charts/unified-topology-app/charts/unified-topology/config/topology-consumer.

To override configuration properties, copy the sample static property file to topology-static-config.yaml and sample dynamic property file to topology-dynamic-config.yaml. Provide key value to override the default value provided out-of-the-box for any specific system configuration property. The properties defined in property files are fed into the container using Kubernetes configuration maps. Any changes to these properties require the instance to be upgraded. Pods are restarted after configuration changes to topology-static-config.yaml.

To integrate Unified Topology API service with Message Bus service:

1. In the file $SPEC_PATH/sr/quick/applications.yaml, uncomment the section messagingBusConfig.
2. Provide namespace and instance name on which the Messaging Bus service is deployed.
3. Security protocol is SASL_PLAINTEXT if authentication is enabled on Message bus service. If authentication is not enabled on the Message Bus service, the security protocol is PLAINTEXT.

A sample configuration when authentication is enabled and Messaging Bus is deployed on instance 'quick' and namespace 'sr' is as follows:

applications.yaml

authentication:
  enabled: true 

messagingBusConfig:
  namespace: sr
  instance: quick

To create a Unified Topology instance in your environment using the scripts that are provided with the toolkit:

1. Run the following command to create a UTIA instance:

   $COMMON_CNTK/scripts/create-applications.sh -p sr -i quick -f $SPEC_PATH/sr/quick/applications.yaml -a unified-topology

   The create-applications script uses the helm chart located in $COMMON_CNTK/charts/unified-topology-app to create and deploy a unified-topology service.

2. If the scripts fail, see the Troubleshooting Issues section at the end of this topic, before you make additional attempts.

To validate the UTIA instance:

1. Run the following to check the status of unified-topology instance deployed.

   $COMMON_CNTK/scripts/application-status.sh -p sr -i quick -f $SPEC_PATH/sr/quick/applications.yaml -a unified-topology

   The application-status script returns the status of unified topology service deployments and pods status.

2. Run the following endpoint to monitor health of unified-topology:

   https://<loadbalancerhost>:<loadbalancerport>/unified-topology/health

3. Run the following Unified Topology service endpoints to add entry in /etc/hosts <k8s cluster ip or external loadbalancer ip> quick.sr.topology.uim.org:
   • Unified Topology UI endpoint: https://quick.sr.topology.uim.org:30443/apps/unified-topology-ui
   • Unified Topology API endpoint: https://quick.sr.topology.uim.org:30443/topology/v2/vertex

Once the instance is created succesfully, cronjob needs to schedule for unified-topology-pgx pod restarts. For a scheduled period of time, one of the unified-topology-pgx pod is restarted and all incoming requests are routed to other unfified-topology-pgx pod seamlessly.

Update the script $COMMON_CNTK/samples/cronjob-scripts/pgx-restart.sh to include required environment variables - KUBECONFIG, pgx_ns, pgx_instance. For a basic instance, pgx_ns is sr and pgx_instance is quick.

export KUBECONFIG=<kube config path>
export pgx_ns=<unified-topology project name>
export pgx_instance=<unified-topology instance name>
pgx_pods=`kubectl get pods -n $pgx_ns --sort-by=.status.startTime -o name | awk -F "/" '{print $2}' | grep $pgx_instance-unified-pgx`
pgx_pod_arr=( $pgx_pods )
echo "Deleting pod - ${pgx_pod_arr[0]}"
kubectl delete pod ${pgx_pod_arr[0]} -n $pgx_ns --grace-period=0

The following crontab is scheduled for every day midnight. Scheduled time may vary depending on the volume of data.

Variable $COMMON_CNTK should be set in environment where cronjob runs or replace $COMMON_CNTK with complete path.

crontab –e 0 0 * * * $COMMON_CNTK/samples/cronjob-scripts/pgx-restart.sh > $COMMON_CNTK/samples/cronjob-scripts/pgx-restart.log

If multiple PGX pods are scheduled on the same worker node, the memory consumption by these PGX pods becomes very high. To address this, include the following affinity rule in applications.yaml, under the unified-topology chart to avoid scheduling of multiple PGX pods on the same worker node.

The following podantiaffinity rule uses the app= <topology-project>-<topology-instance>-unified-pgx label. Update the label with the corresponding project and instance names for UTIA service. For example: sr-quick-unified-pgx.

unified-topology:
  affinity:
    podAntiAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        - labelSelector:
            matchExpressions:
              - key: app
                operator: In
                values:
                - <topology-project>-<topology-instance>-unified-pgx
          topologyKey: "kubernetes.io/hostname"

When Unified Topology service is involved in secure communication with other systems, either as the server or as the client, you should additionally configure SSL/TLS.The procedures for setting up TLS use self-signed certificates for demonstration purposes. However, replace the steps as necessary to use signed certificates.

To setup secure communication using TLS:

1. Generate keystore by passing commoncert.pem and commonkey.pem generated while OAM setup for inputs. Provide -name "param".

   openssl pkcs12 -export -in $COMMON_CNTK/certs/commoncert.pem -inkey $COMMON_CNTK/certs/commonkey.pem -out $COMMON_CNTK/certs/keyStore.p12 -name "topology"

2. Edit the $SPEC_PATH/sr/quick/applications.yaml and set tls enabled to true. Provide tls strategy to be used either terminate or reencrypt. Tls strategy should be RENCRYPT If authetication is enabled using OHS service enabled with SSL.

   tls:
     # The enabled flag is to enable or disable the TLS support for the unified topology m-s end points
     enabled: true
     # valid values are TERMINATE, REENCRYPT
     strategy: "REENCRYPT"

   Note:

   TLS terminate strategy requires ingressTLS secret and TLS reencrypt requires both ingressTLS and appkeystore secrets to be created.

3. Create IngressTLS secret to pass the generated certificate and key pem files.

   $COMMON_CNTK/scripts/manage-app-credentials.sh -p sr -i quick -f $SPEC_PATH/sr/quick/applications.yaml -a unified-topology create ingressTLS

4. The script prompts for the following detail:
   1. Ingress TLS Certificate Path (PEM file): <path_to_cert.pem>
   2. Ingress TLS Key file Path (PEM file): <path_to_key.pem>
5. Create appkeystore secret to pass the generated keystore file.

   $COMMON_CNTK/scripts/manage-app-credentials.sh -p sr -i quick -f $SPEC_PATH/sr/quick/applications.yaml -a unified-topology create appKeystore

6. The script prompts for the following detail:
   • App TLS Keystore Passphrase: <export password value passed while creating keyStrore.p12 key>
   • App TLS Keystore Key Alias: <-name "param" passed while creating keyStore.p12 key>
   • App TLS Keystore PrivateKey Path: <path to keyStore.p12>
7. Verify that the following secrets are created successfully.

   sr-quick-unified-topology-ingress-tls-cert-secret
   sr-quick-unified-topology-keystore

8. Create Unified Topology Instance as usual. Access Topology endpoints using hostname <topology-instance>.<topology-instance>.topology.uim.org
9. Add entry in /etc/hosts <k8s cluster ip or external loadbalancer ip> quick.sr.topology.uim.org
10. Unified Topology UI endpoint: https://quick.sr.topology.uim.org:30443/apps/unified-topology-ui
11. Unified Topology API endpoint: https://quick.sr.topology.uim.org:30443/topology/v2/vertex

openssl req -x509 -newkey rsa:2048 -days 365 -keyout key.pem -out cert.pem -nodes -subj "/CN=<hostname> /ST=TL /L=HYD /O=ORACLE /OU=CAGBU" -extensions san -config <(echo '[req]'; echo 'distinguished_name=req'; echo '[san]';echo 'subjectAltName=@alt_names';echo '[alt_names]';echo 'DNS.1=<hostname>';echo 'DNS.2=localhost';echo 'DNS.3=svc.cluster.local';)

By default, Unified Topology has its pods scheduled on all worker nodes in the Kubernetes cluster in which it is installed. However, in some situations, you may want to choose a subset of nodes where pods are scheduled.

For example:

Limitation on the deployment of Unified Topology on specific worker nodes per each team for reasons such as capacity management, chargeback, budgetary reasons, and so on.

To choose a subset of nodes where pods are scheduled, you can use the configuration in the applications.yaml file.

Sample node affinity configuration(requiredDuringSchedulingIgnoredDuringExecution) for unified topology service:

applications.yaml

unified-topology:
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
        - matchExpressions:
          - key: name
            operator: In
            values:
            - south_zone

Kubernetes pod is scheduled on the node with label name as south_zone. If node with label name: south_zone is not available, pod will not be scheduled.

Sample node affinity configuration (preferredDuringSchedulingIgnoredDuringExecution:) for unified topology service:

applications.yaml

unified-topology:
  affinity:
    nodeAffinity:
      preferredDuringSchedulingIgnoredDuringExecution:
      - weight: 1
        preference:
          matchExpressions:
          - key: name
            operator: In
            values:
            - south_zone

Kubernetes pod is scheduled on the node with label name as south_zone. If node with label name: south_zone is not available, pod will still be scheduled on another node.

Follow the instructions mentioned in UIM Cloud Native Deployment guide for configuring Kubernetes persistent volumes.

To create persistent storage:

1. Update applications.yaml to enable storage volume for unified topology service and provide the persistent volume name.

   storageVolume:
     enabled: true
     pvc: sr-nfs-pvc #Specify the storage-volume name
   

2. Update database.yaml to enable storage volume for unified topology dbinstaller and provide the persistent volume name.

   storageVolume:
     enabled: true
     type: pvc
     pvc: sr-nfs-pvc #Specify the storage-volume name
   

After the instance is created, you must see the directories unified-topology and unified-topology-dbinstaller in your PV mount point, if you have enabled logs.

To customize and enable logging, update the logging configuration files for the application.

1. Customize unified-topology-api service logs:
   • For service level logs update file $COMMON_CNTK/charts/unified-topology-app/charts/unified-topology/config/topology-api/logging-config.xml
   • For Helidon-specific logs update file $COMMON_CNTK/charts/unified-topology-app/charts/unified-topology/config/topology-api/logging.properties. By default console handler is used, you can provide filehandler as well uncomment below lines and provide <project> and <instance> names for location to save logs

     handlers=io.helidon.common.HelidonConsoleHandler,java.util.logging.FileHandler
     java.util.logging.FileHandler.formatter=java.util.logging.SimpleFormatter
     java.util.logging.FileHandler.pattern=/logMount/sr-quick/unified-topology/unified-topology-api/logs/TopologyJULMS-%g-%u.log

2. Customize unified-topology-pgx service logs:

   Update file $COMMON_CNTK/charts/unified-topology-app/charts/unified-topology/config/pgx/logging-config.xml

3. Customize unified-topology-ui service logs:

   Update file $COMMON_CNTK/charts/unified-topology-app/charts/unified-topology/config/topology-ui/logging.properties

4. Update the logging configuration files and upgrade the unified-topology m-s application:

   $COMMON_CNTK/scripts/upgrade-applications.sh -p sr -i quick -f $SPEC_PATH/applications.yaml -a unified-topology

You can view and analyze the Unified Topology service logs using Elastic Stack.

The logs are generated as follows:

• Fluentd collects the text logs that are generated during Unified Topology deployment and sends them to Elasticsearch.
• Elasticsearch collects all types of logs and converts them into a common format so that Kibana can read and display the data.
• Kibana reads the data and presents it in a simplified view.

See "Setting Up Elastic Stack" for more information.

You can view and analyze the Application logs using OpenSearch.

The logs are generated as follows:

1. Fluentd collects the application logs that are generated during cloud native deployments and sends them to OpenSearch.
2. OpenSearch collects all types of logs and converts them into a common format so that OpenSearch Dashboard can read and display the data.
3. OpenSearch Dashboard reads the data and presents it in a simplified view.

See "Setting Up OpenSearch" for more information.

Run the following endpoint to monitor metrics of unified topology:

https://<loadbalancerhost>:<loadbalancerport>/sr/quick/unified-topology/metrics

Prometheus and Grafana setup

See "Setting Up Prometheus and Grafana" for more information.

Adding scrape Job in Prometheus

Add the following Scrape job in Prometheus Server. This can be added by editing the config map used by the Prometheus server:

- job_name: 'topologyApiSecuredMetrics'
      oauth2:
        client_id: <client-id>
        client_secret: <client-secret>
        scopes:
          - <Scope>
        token_url: <OAUTH-TOKEN-URL>
        tls_config:
          insecure_skip_verify: true
      kubernetes_sd_configs:
      - role: pod
      relabel_configs:
      - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]
        action: keep
        regex: true
      - source_labels: [__meta_kubernetes_pod_label_app]
        action: keep
        regex: (<project>-<instance>-unified-topology-api)
      - source_labels: [__meta_kubernetes_pod_container_port_number]
        action: keep
        regex: (8080)
      - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_path]
        action: replace
        target_label: __metrics_path__
        regex: (.+)
      - source_labels: [__address__, __meta_kubernetes_pod_annotation_prometheus_io_port]
        action: replace
        regex: ([^:]+)(?::\d+)?;(\d+)
        replacement: $1:$2
        target_label: __address__
      - action: labelmap
        regex: __meta_kubernetes_pod_label_(.+)
      - source_labels: [__meta_kubernetes_namespace]
        action: replace
        target_label: kubernetes_namespace
      - source_labels: [__meta_kubernetes_pod_name]
        action: replace
        target_label: kubernetes_pod_name

To increase performance of the service, applications.yaml has configuration to provide JVM memory settings and pod resources for Unified Topology Service.

There are separate configurations provided for topology-api, topology-consumer, pgx and topology-ui services. Provide required values under the service name under unified-topology application.

unified-topology:
  topologyApi:
    apiName: "unified-topology-api"
    replicaCount: 3
    java:
      user_mem_args: "-Xms2000m -Xmx2000m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/logMount/$(APP_PREFIX)/unified-topology/unified-topology-api/"
      gc_mem_args: "-XX:+UseG1GC"
      options:
    resources:
      limits:
        cpu: "2"
        memory: 3Gi
      requests:
        cpu: 2000m
        memory: 3Gi

Provide replica count in applications.yaml to scale up or scale down the unified topology pods. Replica count can be configured for topology-api, topology-consumer, pgx and topology-ui pods individually by updating applications.yaml.

Update applications.yaml to increase replica count to 3 for topology-api deployment.

unified-topology:
  topologyApi:
    replicaCount: 3

Apply the change in replica count to the running Helm release by running the upgrade-applications script.

$COMMON_CNTK/scripts/upgrade-applications.sh -p sr -i quick -f $SPEC_PATH/sr/quick/applications.yaml -a unified-topology

By default, GC logs are disabled, you can enable them and view the logs at the corresponding folders inside location /logMount/sr-quick/unified-topology.

To Enable GC logs, update $SPEC_PATH/sr/quick/applications.yaml file as follows:

1. Under gcLogs make enabled as true you can uncomment gcLogs options under unified-topology to override the common values.
2. To configure the maximum size of each file and limit for number of files you need to set fileSize and noOfFiles inside gcLogs as follows:

   gcLogs:
     enabled: true
     fileSize: 10M
     noOfFiles: 10

The disaster recovery when the data center completely goes down is maintained with another passive data center.

Figure 5-2 documents the disaster recovery plan for the data center. A parallel passive data center is maintained, where the runtime data is periodically replicated from the active data center to the passive data center. In the event of any catastrophic failures in the primary (or active) data center, the load must be switched to secondary (or passive) data center. Before switching the load to secondary data center, you should shutdown all the services in the primary data center and start all the services in the secondary data center.

Figure 5-2 Disaster Recovery Plan for Data Center

Description of "Figure 5-2 Disaster Recovery Plan for Data Center"

The purpose of a geographically redundant deployment is to provide resiliency in the event of a complete loss of service in the primary site, due to a natural disaster or other unrecoverable failure in the primary UIM site. This resiliency is achieved by creating one or more passive standby sites that can take the load when the primary site becomes unavailable. The role reversal from the standby site to the primary site can be accomplished in any of the following ways:

• Switchover, in which the operator performs a controlled shutdown of the primary site before activating the standby site. This is primarily intended for planned service interruptions in the primary UIM site. Following a switchover, the former primary site becomes the standby site. The site roles of primary site and standby site can be restored by performing a second switchover operation, which is switchback.
• Failover, in which the primary site becomes unavailable due to unanticipated reasons and cannot be recovered. The operator then transitions the standby site to the primary role. The primary site that is down cannot act as a standby site and will require reconstruction of the database as a standby database before restoring the site roles.

Kafka's Mirror Maker functionality makes it possible to maintain a replica of an existing Kafka cluster (which is used in Message Bus service). This mirrors a source Kafka cluster into a target (mirror) Kafka cluster. To use this mirror, it is a requirement that the source and target Kafka clusters (that is, Message Bus service) are up and running. If the target Kafka cluster is down or offline, we cannot mirror into the target cluster.

Oracle Data Guard

Oracle Data Guard is responsible for replicating transactions from the Active DB to the Standby DB. It is included as a part of every Oracle DB Enterprise Edition installation.

If UTIA is disabled in UIM Cloud Native then it is not required to deploy Message Bus, UTIA and Mirror Maker Services in the clusters. These commands are intended to be used as samples. For detailed documentation on deploying UIM, see UIM Cloud Native Deployment Guide.

To set up the primary (active) instance:

1. Provision Databases one for the primary site and another for the secondary site.
2. Set up Data Guard between primary site and secondary site. Primary site should be in ACTIVE role. Secondary site should be in STANDBY role. Refer to Oracle 19c Documentation.
3. Deploy UIM Cloud Native.
   1. Create image pull secrets (if required).
   2. Create UIM secrets for WLS admin, OPSS, WLS RTE, RCU DB and UIM DB.

      Note:

      uimprimary here refers to the Kubernetes namespace where the primary instance will be deployed. Replace this with the desired namespace.

      $UIM_CNTK/scripts/manage-instance-credentials.sh -p uimprimary -i dr create wlsadmin,opssWP,wlsRTE,rcudb,uimdb

   3. Create Weblogic encrypted password.

      $UIM_CNTK/scripts/install-uimdb.sh -p uimprimary -i dr -s $SPEC_PATH -c 8

   4. Create UIM users secrets.

      $UIM_CNTK/samples/credentials/manage-uim-credentials.sh -p uimprimary -i dr -c create -f "/home/spec_dir/users.txt"

   5. Create DB schemas.

      $UIM_CNTK/scripts/install-uimdb.sh -p uimprimary -i dr -s $SPEC_PATH -c 1
      $UIM_CNTK/scripts/install-uimdb.sh -p uimprimary -i dr -s $SPEC_PATH -c 2

   6. Create UIM instance.

      $UIM_CNTK/scripts/create-ingress.sh -p uimprimary -i dr -s $SPEC_PATH
      $UIM_CNTK/scripts/create-instance.sh -p uimprimary -i dr -s $SPEC_PATH

   7. Add UIM user roles.

      $UIM_CNTK/samples/credentials/assign-role.sh -p uimprimary -i dr -f uim-users-roles.txt

4. Deploy Message Bus.

   $COMMON_CNTK/scripts/create-applications.sh -p uimprimary -i dr -f $SPEC_PATH/<project>/<instance>/applications.yaml -a messaging-bus

5. Deploy UTIA:
   1. Create Topology DB secrets:

      $COMMON_CNTK/scripts/manage-app-credentials.sh -p uimprimary -i dr -f $SPEC_PATH/<project>/<instance>/applications.yaml -a unified-topology create database

   2. Create Topology UIM secrets:

      $COMMON_CNTK/scripts/manage-app-credentials.sh -p uimprimary -i dr -f $SPEC_PATH/<project>/<instance>/applications.yaml -a unified-topology create uim

   3. Create Topology users:

      $COMMON_CNTK/scripts/manage-app-credentials.sh -p uimprimary -i dr -f $SPEC_PATH/<project>/<instance>/applications.yaml -a unified-topology create appUsers

   4. Create Topology UI users:

      $COMMON_CNTK/scripts/manage-app-credentials.sh -p uimprimary -i dr -f $SPEC_PATH/<project>/<instance>/applications.yaml -a unified-topology create appUIUsers

   5. Create DB schemas:

      $COMMON_CNTK/scripts/install-database.sh -p uimprimary -i dr -f $SPEC_PATH/<proejct>/<instance>/database.yaml -a unified-topology -c 1

   6. Deploy Topology:

      $COMMON_CNTK/scripts/create-applications.sh -p uimprimary -i dr -f $SPEC_PATH/<project>/<instance>/applications.yaml -a unified-topology

See "Deploying Unified Operations Message Bus" for deploying Message Bus, "Deploying the Unified Topology for Inventory and Automation Service" for deploying UTIA.

See UIM Cloud Native Deployment Guide for deploying UIM.

To set up the secondary (standby) instance:

1. Perform switchover operation on active (primary site) DB. Now secondary site DB should be in ACTIVE role and primary site DB should be in PASSIVE role. Refer to Oracle 19c Documentation.
2. Deploy UIM Cloud Native:
   1. Export OPSS wallet file secret from primary instance and recreate in secondary instance.

      Note:

      Where, uimsecondary refers to the Kubernetes namespace where the secondart instance will be deployed. Replace this with the desired namespace.

      kubectl -n uimprimary get configmap uimprimary-dr-weblogic-domain-introspect-cm -o jsonpath='{.data.ewallet\.p12}' > ./primary_ewallet.p12
      $UIM_CNTK/scripts/manage-instance-credentials.sh -p uimsecondary -i dr create opssWF

   2. (Optional) Create image pull secrets.
   3. Create UIM secrets for WLS admin, OPSS, WLS RTE, RCU DB and UIM DB:

      $UIM_CNTK/scripts/manage-instance-credentials.sh -p uimsecondary -i quick create wlsadmin,opssWP,wlsRTE,rcudb,uimdb

   4. Create Weblogic encrypted password:

      $UIM_CNTK/scripts/install-uimdb.sh -p uimsecondary -i dr -s $SPEC_PATH -c 8

   5. Create UIM users secrets:

      $UIM_CNTK/samples/credentials/manage-uim-credentials.sh -p uimsecondary -i dr -c create -f "/home/spec_dir/users.txt"

   6. Create UIM instance:

      $UIM_CNTK/scripts/create-ingress.sh -p uimsecondary -i dr -s $SPEC_PATH
      $UIM_CNTK/scripts/create-instance.sh -p uimsecondary -i dr -s $SPEC_PATH

   7. Add UIM user roles:

      $UIM_CNTK/samples/credentials/assign-role.sh -p uimsecondary -i dr -f uim-users-roles.txt

3. Deploy message bus:

   $COMMON_CNTK/scripts/create-applications.sh -p uimsecondary -i dr -f $SPEC_PATH/<project>/<instance>/applications.yaml  -a messaging-bus

4. Deploy UTIA:
   1. Create Topology DB secrets:

      $COMMON_CNTK/scripts/manage-app-credentials.sh -p uimsecondary -i dr -f $SPEC_PATH/<project>/<instance>/applications.yaml  -a unified-topology create database

   2. Create Topology UIM secrets:

      $COMMON_CNTK/scripts/manage-app-credentials.sh -p uimsecondary -i dr -f $SPEC_PATH/<project>/<instance>/applications.yaml  -a unified-topology create uim

   3. Create Topology users:

      $COMMON_CNTK/scripts/manage-app-credentials.sh -p uimsecondary -i dr -f $SPEC_PATH/<project>/<instance>/applications.yaml  -a unified-topology create appUsers

   4. Create Topology UI users:

      $COMMON_CNTK/scripts/manage-app-credentials.sh -p uimsecondary -i dr -f $SPEC_PATH/<project>/<instance>/applications.yaml  -a unified-topology create appUIUsers

   5. Deploy Topology:

      $COMMON_CNTK/scripts/create-applications.sh -p uimsecondary -i dr -f $SPEC_PATH/<project>/<instance>/applications.yaml  -a unified-topology

5. Deploy Mirror Maker. See "Installing and Configuring Mirror Maker 2.0" for more information.
6. After the secondary instance has been setup, switchover back to the primary (active) site.

To perform a switchover between site A (active) and site B (standby):

1. Bring down instances in site A. These include UIM and UTIA. Message Bus must be enabled to perform the replication using Mirror Maker.

   #Disable topology
   $COMMON_CNTK/scripts/delete-applications.sh -p uimprimary -i dr -f $SPEC_PATH/<project>/<instance>/applications.yaml -a unified-topology
   #Disable UIM
   $UIM_CNTK/scripts/delete-instance.sh -p uimprimary -i dr -s $SPEC_PATH

2. Perform switchover on DB. Site B DB will now become Primary. Site B DB will assume Standby role. Refer to Oracle 19c Documentation.
3. Bring up instances in site B. This includes UIM and UTIA. Message Bus should already be active:

   #EnableUIM
   $UIM_CNTK/scripts/create-instance.sh -p uimsecondary -i dr -s $SPEC_PATH
   #Enable topology
   $COMMON_CNTK/scripts/create-applications.sh -p uimsecondary -i dr -f $SPEC_PATH/<project>/<instance>/applications.yaml -a unified-topology

4. Perform DNS switching to route all traffic to site B.

In case of any irrecoverable failure in the primary site, perform a failover operation on the standby site. To do so:

1. Perform failover on DB. Standby (secondary) DB will now become Primary. Primary site DB will assume Deactivated Standby role. Refer to Oracle 19c Documentation.
2. Bring up instances in standby. This includes UIM and Topology. Message Bus should already be active:

   #EnableUIM
   $UIM_CNTK/scripts/create-instance.sh -p uimsecondary -i dr -s $SPEC_PATH
   #Enable topology
   $COMMON_CNTK/scripts/create-applications.sh -p uimsecondary -i dr -f $SPEC_PATH/<project>/<instance>/applications.yaml -a unified-topology

3. Perform DNS switching to route all traffic to secondary instances.

Once the primary site to restored, establish a synchronization between secondary and primary site. To do so:

1. Bring up Message Bus and DB in primary site:

   #Enable message bus
   $COMMON_CNTK/scripts/create-applications.sh -p uimprimary-i dr -f $SPEC_PATH/<project>/<instance>/applications.yaml -a messaging-bus

2. Setup Kafka Mirror Maker with secondary Message Bus as source and primary Message Bus as target. See "About Kafka Mirror Maker" for more information.
3. Switch primary DB role from Deactivated Standby → Standby. See Deploying Unified Operations Message Bus for more information.