Upgrading Unified Assurance to a New Linux System

You can upgrade Oracle Communications Unified Assurance onto a new Linux 8 system by first installing the new Linux 8 environment, and then synchronizing and updating your Unified Assurance installation onto the new system. This is also known as a forklift upgrade.

About the Forklift Upgrade Procedure

Review the details in this section before beginning the upgrade procedure.

Forklift Upgrade Overview

A forklift upgrade involves the following high-level steps, repeated for each server in the environment:

1. Installing a new Linux 8 environment. For information about installing Linux 8, see Oracle Linux 8 Installing Oracle Linux.

2. Preparing for the Unified Assurance upgrade by backing up the Unified Assurance home directory and copying the backup file over to the new Linux 8 server.

3. Synchronizing Unified Assurance files and data to the new Linux 8 server.

4. Updating Unified Assurance, including removing and replacing important configurations.

Forklift Upgrade Active User

You run most commands in this procedure as the root user. For some steps, such as backing up and restoring the Graph database, you will switch to the assure1 user. These switches are specified in the commands.

Forklift Upgrade Placeholders

The procedures use the following placeholders, marked with angled brackets in the commands (<>):

• <UA_home>: Your Unified Assurance home directory. For example, /opt/assure1.

• <installation_file_directory>: The directory to store installation files. For example, /opt/install.

• <Primary_Linux_7_FQDN>: The fully-qualified domain name (FQDN) of the original Linux 7 Unified Assurance primary server.

• <Primary_Linux_8_FQDN>: The FQDN of the new Linux 8 Unified Assurance primary server.

• <Secondary_Linux_7_FQDN>: The fully-qualified domain name (FQDN) of the original Linux 7 Unified Assurance secondary partner server.

• <Secondary_Linux_8_FQDN>: The fully-qualified domain name (FQDN) of the original Linux 8 Unified Assurance secondary partner server.

• <Primary_Linux_8_web_FQDN>: The web FQDN of the new Linux 8 primary server (the same as secondary, if the WebFQDN is shared).

• <Secondary_Linux_8_web_FQDN>: The web FQDN of the new Linux 8 secondary partner server (the same as primary, if the WebFQDN is shared).

To make the process easier, the procedures instructs you to create environment variables for these placeholder values at the appropriate point. For example:

export A1BASEDIR=/opt/assure1
export FILESDIR=/opt/install
export OL7FQDN=primary.linux7.server.fqdn
export OL8FQDN=primary.linux8.server.fqdn
export SECOL7FQDN=secondary.linux7.server.fqdn
export OL8WEBFQDN=ol8.server.webfqdn
export RPMVERSION=6.0.4.0.2

Server Upgrade Order

Perform the upgrade on each server in the following order:

1. The primary presentation server. See Task 1: Upgrading the Primary Presentation Server.

2. The secondary presentation server. See Task 2: Upgrading the Secondary Presentation Server.

3. Each database server pair, starting with primary, then secondary. See Task 3: Upgrading Database Servers.

   To ensure minimal downtime, upgrade each database server one pair at a time. For example, if you have two database server pairs, upgrade them in this order:

   1. Pair 1: Primary database server

   2. Pair 1: Secondary database server

   3. Pair 2: Primary database server

   4. Pair 2: Secondary database server

4. Each collection server. See Task 4: Upgrading Collection Servers.

5. Each external presentation server. See Task 5: Upgrading External Presentation Servers.

When you have upgraded all your servers, perform the tasks described in Task 6: Finalizing the Upgrade.

Forklift Upgrade Prerequisites

Before starting a forklift upgrade:

1. Set up your Linux 8 servers and prepare them for the Unified Assurance installation. See Linux Prerequisites for information about preparing your Linux system for a Unified Assurance installation or upgrade.

2. Download the latest Unified Assurance release package for Linux 8 from Oracle Software Delivery Cloud, and upload to your Linux 8 servers:

   https://edelivery.oracle.com

   The package contains the Oracle-Communications-Unified-Assurance-6.X.X.X.X-el8.zip collection file for installation.

3. If your existing Linux 7 system uses Keepalived, disable and stop the service. You can configure and enable the service again once the upgrade has been completed.

4. Verify that your existing Linux 7 primary and secondary databases are synchronized.

Task 1: Upgrading the Primary Presentation Server

On your existing Linux 7 presentation servers:

1. Starting with the primary and then repeating on the secondary Linux 7 presentation servers:

   1. Connect to MySQL as the root user.

   2. Get the binlog file name and position by running one of the following commands, depending on your MySQL version:

      • For MySQL versions less than 8.4.0:

        SHOW SLAVE STATUS \G
        

      • For MySQL versions 8.4.0 or greater:

        SHOW REPLICA STATUS \G
        

   3. Copy the values for the following parameters, depending on your MySQL version, and save them somewhere you can refer to them later:

      • For MySQL versions less than 8.4.0: Relay_Master_Log_File and Exec_Master_Log_Pos

        These will be referred to later as:

        • <Pri_L7_Pres_Relay_Master_Log_File>

        • <Pri_L7_Pres_Exec_Master_Log_Pos>

        • <Sec_L7_Pres_Relay_Master_Log_File>

        • <Sec_L7_Pres_Exec_Master_Log_Pos>

      • For MySQL versions 8.4.0 or greater: Relay_Source_Log_File and Exec_Source_Log_Pos

        These will be referred to later as:

        • <Pri_L7_Pres_Relay_Source_Log_File>

        • <Pri_L7_Pres_Exec_Source_Log_Pos>

        • <Sec_L7_Pres_Relay_Source_Log_File>

        • <Sec_L7_Pres_Exec_Source_Log_Pos>

   4. Stop replication by running one of the following commands, depending on your MySQL version:

      • For MySQL versions less than 8.4.0:

        STOP SLAVE;
        

      • For MySQL versions 8.4.0 or greater:

        STOP REPLICA;
        

2. On your primary and secondary Linux 7 presentation servers, create the $A1BASEDIR environment variable for <UA_home>, replacing <UA_home> with the appropriate value. This is usually opt/assure1:

   export A1BASEDIR=<UA_home>
   

3. For presentation servers that also have the Graph role (for Neo4j database servers):

   1. On your primary and secondary Linux 7 presentation-database servers, disable Neo4j replication by moving the graph-repl.jar and schema-repl.jar files to the $A1BASEDIR/vendor/neo4j/disabled_plugins/ directory:

      mv $A1BASEDIR/vendor/neo4j/plugins/graph-repl.jar $A1BASEDIR/vendor/neo4j/disabled_plugins/
      mv $A1BASEDIR/vendor/neo4j/plugins/schema-repl.jar $A1BASEDIR/vendor/neo4j/disabled_plugins/
      

      This prevents issues resulting from the old plugins from being run alongside the new plugins.

      Note:

      Depending on the version of Unified Assurance you have installed, graph-repl.jar and schema-repl.jar may no longer exist or may have already been moved to a backup directory. If so, continue to the next step.

   2. If your version of Unified Assurance on Linux 7 is 6.0.4 or higher, do the following on the primary and secondary Linux 7 presentation-database servers:

      1. In the $A1BASEDIR/etc/neo4j/streams.conf file, change the value of the streams.source.enabled property to false. For example:

         streams.source.enabled=false
         

      2. Disable and stop the Neo4j Kafka Connect service:

         1. In the Unified Assurance UI, from the Configuration menu, select Broker Control, then select Services.

         2. Select the Neo4j Kafka Connect service.

            The service details appear.

         3. Set Status to Disabled and click Submit.

            In the All Services table, the value in the Status column for the service is updated to Disabled.

         4. In the All Services table, click Stop.

            In the All Services table, the value in the State column for the service is updated to Stopped.

      3. Verify that the Neo4j processes have stopped:

         ps -ef | grep neo4j
         

         The output should not show any running Neo4j processes.

   3. On your secondary presentation-database server, restart the Neo4j service:

      1. In to the Unified Assurance UI, from the Configuration menu, select Broker Control, then select Services.

      2. Select the Neo4j service and click Restart.

4. On your primary Linux 7 presentation server:

   1. Stop all services, the broker, the message bus, the web server, and the assure1 database:

      source $A1BASEDIR/.assure1_bashrc
      $A1BASEDIR/bin/BrokerControl --batch stopall
      systemctl stop assure1-broker
      systemctl stop assure1-bus
      systemctl stop assure1-web
      systemctl stop assure1-db
      

   2. Verify that all processes have stopped:

      ps aux | grep assure1
      systemctl status assure1-broker
      systemctl status assure1-bus
      systemctl status assure1-web
      systemctl status assure1-db
      

   3. Create a file called tar-excludes.txt that specifies what to exclude from the backup:

      cat <<EOF > tar-excludes.txt
      $A1BASEDIR/distrib/packages
      $A1BASEDIR/var/chartmuseum
      $A1BASEDIR/var/docker
      $A1BASEDIR/var/registry
      $A1BASEDIR/var/run/docker
      $A1BASEDIR/var/run/docker.pid
      $A1BASEDIR/vendor/docker
      $A1BASEDIR/vendor/helm
      $A1BASEDIR/vendor/kubernetes
      $A1BASEDIR/vendor/rke
      $A1BASEDIR/vendor/submariner
      EOF
      

   4. Back up the $A1BASEDIR directory:

      tar --exclude-from=tar-excludes.txt -czf UA-Linux7-Primary-Presentation-Backup.tgz $A1BASEDIR
      

   5. Copy the UA-Linux7-Primary-Presentation-Backup.tgz tar backup file to the new primary Linux 8 presentation server. You can use any secure transfer protocol. For example, using scp:

      scp UA-Linux7-Primary-Presentation-Backup.tgz <user>@<hostname>:/home/<user>/
      

5. On your new primary Linux 8 presentation server:

   1. Move the UA-Linux7-Primary-Presentation-Backup.tgz file to <installation_file_directory>:

      mv /home/<user>/UA-Linux7-Primary-Presentation-Backup.tgz <installation_file_directory>
      

   2. Create the following environment variables, replacing the placeholders with the appropriate values:

      export A1BASEDIR=<UA_home>
      export FILESDIR=<installation_file_directory>
      export OL7FQDN=<Primary_Linux_7_FQDN>
      export OL8FQDN=<Primary_Linux_8_FQDN>
      export SECOL7FQDN=<Secondary_Linux_7_FQDN>
      export OL8WEBFQDN=<Primary_Linux_8_web_FQDN>
      

   3. Extract the UA-Linux7-Primary-Presentation-Backup.tgz tar backup file:

      tar -xzf $FILESDIR/UA-Linux7-Primary-Presentation-Backup.tgz -C /
      

   4. Replace the old server FQDN with the new FQDN:

      grep -rl --exclude-dir={etc,legal,lib,logs,tmp,var} "$OL7FQDN" $A1BASEDIR/* | xargs sed -i "s/$OL7FQDN/$OL8FQDN/g"
      grep -rl --exclude-dir={ssl,priv} "$OL7FQDN" $A1BASEDIR/etc/* | xargs sed -i "s/$OL7FQDN/$OL8FQDN/g"
      

   5. Update the rules checkout paths to point to the new FQDN:

      for CHECKOUT in config core diagrams files
      do
        if [ -d $A1BASEDIR/var/checkouts/$CHECKOUT ] ; then
            cd $A1BASEDIR/var/checkouts/$CHECKOUT &&  $A1BASEDIR/vendor/svn/bin/svn --config-dir $A1BASEDIR/etc/svn relocate https://OL8WEBFQDN/repos/$CHECKOUT/
        fi
      done
      

   6. For servers with the Database.Historical role (for Elasticsearch database servers), replace the old server FQDN in the Kibana configuration file with the new FQDN:

      sed -i "s/$OL7FQDN/$OL8FQDN/g" $A1BASEDIR/vendor/kibana/config/kibana.yml
      

   7. Remove the old SSL certificates:

      rm -f $A1BASEDIR/etc/ssl/priv/*.pem
      rm -f $A1BASEDIR/etc/ssl/priv/*.old
       > $A1BASEDIR/etc/ssl/priv/index.txt
       > $A1BASEDIR/etc/ssl/priv/index.txt.attr
       > $A1BASEDIR/etc/ssl/priv/serial
      cd $A1BASEDIR/etc/ssl && ls | grep -xv ".*CA.*" | grep -xv priv | xargs rm -rf
      rm -rf $A1BASEDIR/var/rabbitmq/mnesia/*
      

   8. Check the contents of the $A1BASEDIR/etc/ssl directory:

      ls -l $A1BASEDIR/etc/ssl/
      

      The response should look like this:

      -rw-r--r-- 1 60 assure1 1342 Jun 17 14:31 Assure1CA.crt
      -rw-r--r-- 1 60 assure1 5626 Jun 17 14:32 BundleCA.crt
      -rw-r--r-- 1 60 assure1 2294 Jun 17 14:32 BundleCA.jks
      drwxr-xr-x 2 60 assure1   59 Jun 20 08:23 priv
      -rw-r--r-- 1 60 assure1 4284 Jun 17 14:32 SiteCA.crt
      -rw------- 1 60 assure1 1854 Jun 17 14:32 SiteCA.key
      

      Remove any additional files.

   9. Check the contents of the $A1BASEDIR/etc/ssl/priv directory:

      ls -l $A1BASEDIR/etc/ssl/priv/
      

      The response should look like this:

      -rw-r--r-- 1 60 assure1 0 Jun 20 08:23 index.txt
      -rw-r--r-- 1 60 assure1 0 Jun 20 08:23 index.txt.attr
      -rw-r--r-- 1 60 assure1 0 Jun 20 08:23 serial
      

      Confirm that index.txt, index.txt.attr and serial are empty files and remove any additional files.

   10. Extract the collection file contents from the Unified Assurance release package to <installation_file_directory>:

       unzip -d $FILESDIR Oracle-Communications-Unified-Assurance-6.X.X.X.X-el8.zip
       

   11. Install the latest Unified Assurance RPM for Linux 8:

       dnf -y install $FILESDIR/Assure1-6.X.X.X.X-X.el8.x86_64.rpm
       

   12. Create an environment variable for the RPM version number. Do not include the package number or el8, and replace each X with the appropriate number:

       export RPMVERSION=6.X.X.X.X
       

       For example:

       export RPMVERSION=6.0.4.0.2
       

   13. Add the Linux 8 host FQDN to the <UA_home>/etc/Assure1.conf file:

       sed -i "s/\"DBHost\": \"\"/\"DBHost\": \"$OL8FQDN\"/g" $A1BASEDIR/etc/Assure1.conf
       

   14. Generate new SSL certificates:

       $A1BASEDIR/bin/CreateSSLCertificate --Type Host -c $OL8FQDN
       $A1BASEDIR/bin/CreateSSLCertificate --Type SSH
       $A1BASEDIR/bin/CreateSSLCertificate --Type User -c assure1
       $A1BASEDIR/bin/CreateSSLCertificate --Type User -c root
       $A1BASEDIR/bin/CreateSSLCertificate --Type User -c api
       $A1BASEDIR/bin/CreateSSLCertificate --Type User -c kibana
       $A1BASEDIR/bin/CreateSSLCertificate --Type User -c repl
       $A1BASEDIR/bin/CreateSSLCertificate --Type Web -c $OL8WEBFQDN
       

   15. For servers with the Database.Historical role (for Elasticsearch database servers), copy the BundleCA.crt, Host.crt and Host.key files to your Elasticsearch certs directory:

       /bin/cp $A1BASEDIR/etc/ssl/BundleCA.crt $A1BASEDIR/vendor/elasticsearch/config/certs/
       /bin/cp $A1BASEDIR/etc/ssl/Host.crt $A1BASEDIR/vendor/elasticsearch/config/certs/
       /bin/cp $A1BASEDIR/etc/ssl/Host.key $A1BASEDIR/vendor/elasticsearch/config/certs/
       chown assure1:assure1 $A1BASEDIR/vendor/elasticsearch/config/certs/*
       

   16. Copy the following Unified Assurance systemd scripts from the /etc/systemd/system/ directory on the original Linux 7 server to the /etc/systemd/system/ directory on the Linux 8 server:

       • assure1-db.service

       • assure1-bus.service

       • assure1-web.service

   17. Reload the systemctl daemon:

       systemctl daemon-reload
       

   18. Start the assure1 database:

       systemctl start assure1-db
       

   19. On the primary Linux 8 presentation server's MySQL database, set up the new primary Linux 8 presentation server as a temporary replication partner for the old secondary Linux 7 presentation server. This synchronizes any missing data that may have been added to old secondary server while the primary server was down during the upgrade process.

       To set this up, run the following commands as the root MySQL user:

       • For MySQL versions less than 8.4.0:

         STOP SLAVE;
         RESET SLAVE;
         CHANGE MASTER TO MASTER_HOST='<SECONDARY-LINUX7-SERVER-FQDN>', MASTER_LOG_FILE='<Pri_L7_Pres_Relay_Master_Log_File>', MASTER_LOG_POS=<Pri_L7_Pres_Exec_Master_Log_Pos>;
         START SLAVE;
         

         where:

         • <SECONDARY-LINUX7-SERVER-FQDN> is the FQDN of the original secondary Linux 7 presentation server.

         • <Pri_L7_Pres_Relay_Master_Log_File> is the Relay_Master_Log_File value saved earlier for the primary Linux 7 presentation server.

         • <Pri_L7_Pres_Exec_Master_Log_Pos> is the Exec_Master_Log_Pos value saved earlier for the primary Linux 7 presentation server.

       • For MySQL versions 8.4.0 or greater:

         STOP REPLICA;
         RESET REPLICA;
         CHANGE REPLICATION SOURCE TO SOURCE_HOST='<SECONDARY-LINUX7-SERVER-FQDN>', SOURCE_LOG_FILE='<Pri_L7_Pres_Relay_Source_Log_File>', SOURCE_LOG_POS=<Pri_L7_Pres_Exec_Source_Log_Pos>;
         START REPLICA;
         

         where:

         • <SECONDARY-LINUX7-SERVER-FQDN> is the FQDN of the original secondary Linux 7 presentation server.

         • <Pri_L7_Pres_Relay_Source_Log_File> = the Relay_Source_Log_File value saved earlier for the primary Linux 7 presentation server.

         • <Pri_L7_Pres_Exec_Source_Log_Pos> = the Exec_Source_Log_Pos value saved earlier for the primary Linux 7 presentation server.

6. On the secondary Linux 7 presentation server's MySQL database, run the following commands as the root MySQL user:

   • For MySQL versions less than 8.4.0:

     CHANGE MASTER TO MASTER_HOST='<PRIMARY-LINUX8-SERVER-FQDN>', MASTER_LOG_FILE='<Sec_L7_Pres_Relay_Master_Log_File>', MASTER_LOG_POS=<Sec_L7_Pres_Exec_Master_Log_Pos>;
     START SLAVE;
     

     where:

     • <PRIMARY-LINUX8-SERVER-FQDN> is the FQDN of the new primary Linux 8 presentation server.

     • <Sec_L7_Pres_Relay_Master_Log_File> is the Relay_Master_Log_File value saved earlier for the secondary Linux 7 presentation server.

     • <Sec_L7_Pres_Exec_Master_Log_Pos> is the Exec_Master_Log_Pos value saved earlier for the secondary Linux 7 presentation server.

   • For MySQL versions 8.4.0 or greater:

     CHANGE REPLICATION SOURCE TO SOURCE_HOST='<PRIMARY-LINUX8-SERVER-FQDN>', SOURCE_LOG_FILE='<Sec_L7_Pres_L7_Relay_Source_Log_File>', SOURCE_LOG_POS=<Sec_L7_Pres_Exec_Source_Log_Pos>;
     START REPLICA;
     

     The placeholder values in the above command should be replaced with the relevant info as follows:

     • <PRIMARY-LINUX8-SERVER-FQDN> is the FQDN of the new Linux 8 primary presentation server

     • <Sec_L7_Pres_Relay_Source_Log_File> is the Relay_Source_Log_File value saved earlier for the secondary Linux 7 presentation server.

     • <Sec_L7_Pres_Exec_Source_Log_Pos> is the Exec_Source_Log_Pos value saved earlier for the secondary Linux 7 presentation server.

7. On the primary Linux 8 presentation server:

   1. Create an environment variable for the primary Linux 7 presentation server ID:

      PRISERVERID=$($A1BASEDIR/vendor/mysql/bin/mysql --defaults-file=$A1BASEDIR/etc/my.cnf Assure1 -e "SELECT ServerID FROM Servers WHERE ServerHostFQDN = '$OL7FQDN'" | cut -f1 | grep -v 'ServerID')
      

   2. Set the correct Server FQDN in the database, and reset packages to their base 6.0.0.0.0 versions:

      $A1BASEDIR/vendor/mysql/bin/mysql --defaults-file=$A1BASEDIR/etc/my.cnf Assure1 -e "UPDATE Servers SET ServerHostFQDN='$OL8FQDN', WebFQDN='$OL8WEBFQDN', Version='$RPMVERSION' WHERE ServerID = $PRISERVERID"
      $A1BASEDIR/vendor/mysql/bin/mysql --defaults-file=$A1BASEDIR/etc/my.cnf Assure1 -e "UPDATE CoreDatabases SET DBHost = '$OL8FQDN' WHERE DBHost = '$OL7FQDN'"
      $A1BASEDIR/vendor/mysql/bin/mysql --defaults-file=$A1BASEDIR/etc/my.cnf Assure1 -e "UPDATE AuthVendorSystems SET SystemHost = '$OL8FQDN' WHERE SystemHost = '$OL7FQDN'"
      $A1BASEDIR/vendor/mysql/bin/mysql --defaults-file=$A1BASEDIR/etc/my.cnf Assure1 -e "UPDATE Packages SET PackageVersion = '6.0.0.0.0.0' WHERE ServerID = $PRISERVERID"
      

   3. Start RabbitMQ:

      systemctl start assure1-bus
      

   4. Create a new assure1 RabbitMQ user:

      source $A1BASEDIR/.assure1_bashrc
      $A1BASEDIR/bin/Package add-rabbit-user
      

   5. Remove the old Linux 7 packages from the $A1BASEDIR/distrib/packages directory:

      rm -f $A1BASEDIR/distrib/packages/*
      

8. On the secondary Linux 7 presentation server:

   1. Edit the $A1BASEDIR/etc/Assure1.conf file to replace the old Linux 7 primary presentation server FQDN with the new Linux 8 primary presentation server FQDN.

   2. Restart the Broker service:

      systemctl restart assure1-broker
      

9. On the primary Linux 8 presentation server:

   1. Start the Broker service:

      systemctl start assure1-broker
      

   2. Verify the Broker service has started:

      systemctl status assure1-broker
      

   3. Remove the old Linux 7 SSL and crypto libraries:

      rm -f $A1BASEDIR/lib/libcrypto.s* $A1BASEDIR/lib/libssl.s*
      

10. For presentation servers that also have the Graph role (for Neo4j database servers):

    1. On the secondary Linux 7 presentation-database server:

       1. Switch to the assure1 user:

          su - assure1
          

       2. Set the Graph database to READ ONLY to enable backing up the schema and data:

          a1neo4jroot
          ALTER DATABASE graph SET ACCESS READ ONLY;
          :exit
          

       3. Create the $A1BASEDIR environment variable for <UA_home>, replacing <UA_home> with the appropriate value. This is usually opt/assure1:

          export A1BASEDIR=<UA_home>
          

       4. Back up the Graph database by running one of the following sets of commands:

          • For Unified Assurance version 6.0.3 or lower:

            mkdir $A1BASEDIR/tmp/neo4j-dump
            NEO4J_CONF=$A1BASEDIR/etc/neo4j/ $A1BASEDIR/vendor/neo4j/bin/neo4j-admin backup --database=graph --backup-dir=$A1BASEDIR/tmp/neo4j-dump/ --include-metadata=all
            

          • For Unified Assurance version 6.0.4 or higher:

            mkdir $A1BASEDIR/tmp/neo4j-dump
            NEO4J_CONF=$A1BASEDIR/etc/neo4j/ $A1BASEDIR/vendor/neo4j/bin/neo4j-admin database backup --to-path=$A1BASEDIR/tmp/neo4j-dump --include-metadata=all --type=full graph
            

    2. On the primary Linux 8 presentation server:

       1. Switch to the assure1 user:

          su - assure1
          

       2. Connect to Neo4j and drop the Graph database:

          a1neo4jroot
          DROP DATABASE Graph;
          :exit
          

       3. Create the $A1BASEDIR environment variable for <UA_home>, replacing <UA_home> with the appropriate value. This is usually opt/assure1:

          export A1BASEDIR=<UA_home>
          

       4. Copy the backup files from the secondary Linux 7 presentation server to the primary Linux 8 presentation server.

       5. Restore the Graph database from the backup file by running one of the following sets of commands:

          • For Unified Assurance version 6.0.3 or lower:

            sudo chown -R assure1:assure1 $A1BASEDIR/tmp/neo4j-dump
            NEO4J_CONF=$A1BASEDIR/etc/neo4j/ $A1BASEDIR/vendor/neo4j/bin/neo4j-admin restore --database=graph --from=$A1BASEDIR/tmp/neo4j-dump/graph
            

          • For Unified Assurance version 6.0.4 or higher:

            sudo chown -R assure1:assure1 $A1BASEDIR/tmp/neo4j-dump
            NEO4J_CONF=$A1BASEDIR/etc/neo4j/ $A1BASEDIR/vendor/neo4j/bin/neo4j-admin database restore --from-path=$A1BASEDIR/tmp/neo4j-dump/<backup-file-name> graph
            

            where <backup-file-name> is the name of your Graph database backup file.

       6. Check if the output of the restore command contains the following line:

          You need to execute .../var/neo4j/scripts/graph/restore_metadata.cypher. To execute the file use cypher-shell command with parameter `graph`
          

          If so, run the following command to restore the metadata:

          cat $A1BASEDIR/var/neo4j/scripts/graph/restore_metadata.cypher | A1_JAVA_HOME=$A1BASEDIR/vendor/java NEO4J_PASSWORD=$($A1BASEDIR/bin/JWT -a neo4j -s neo4j) $A1BASEDIR/vendor/neo4j/bin/cypher-shell -a neo4j+ssc://$(hostname):7687 -u neo4j -d system --param "database => 'graph'"
          

          If not, continue.

       7. Connect to Neo4j as the admin user, switch to the system database, and create the Graph database:

          a1neo4jroot
          :use system
          CREATE DATABASE GRAPH;
          :exit
          

          Note:

          If the Graph database already exists, a warning message appears. Ignore it and continue.

       8. Switch back to the root user:

          su - root
          

    3. In the Unified Assurance UI:

       1. From the Configuration menu, select Broker Control, then Services.

       2. Select the Neo4j service for the secondary Linux 7 presentation server.

          The service details appear.

       3. Set Status to Disabled and click Submit.

          In the All Services table, the value in the Status column for the service is updated to Disabled.

       4. In the All Services table, click Stop.

          In the All Services table, the value in the State column for the service is updated to Stopped.

11. On the primary Linux 8 presentation server:

    1. Wait for the Broker to start the Unified Assurance services, then update the Unified Assurance package:

       source $A1BASEDIR/.assure1_bashrc
       $A1BASEDIR/bin/Package update --depot $FILESDIR
       

    2. Start the web server:

       systemctl start assure1-web
       

    3. In the $A1BASEDIR/etc/Assure1.conf file, remove the InstallIncomplete entry:

       sed -i '/InstallIncomplete/c\' $A1BASEDIR/etc/Assure1.conf
       

The primary presentation server upgrade is complete. Continue to upgrade your remaining servers in the order described in Server Upgrade Order.

Task 2: Upgrading the Secondary Presentation Server

1. Verify that all Unified Assurance services are running and all data collection is happening on the Linux 8 primary presentation server.

2. Starting with your existing secondary Linux 7 presentation server and then on your new primary Linux 8 presentation server:

   1. Connect to MySQL as the root user.

   2. Get the binlog file name and position by running one of the following commands, depending on your MySQL version:

      • For MySQL versions less than 8.4.0:

        SHOW SLAVE STATUS \G
        

      • For MySQL versions 8.4.0 or greater:

        SHOW REPLICA STATUS \G
        

   3. Copy the values for the following parameters, depending on your MySQL version, and save them somewhere you can refer to them later:

      • For MySQL versions less than 8.4.0: Relay_Master_Log_File and Exec_Master_Log_Pos

        These will be referred to later as:

        • <Sec_L7_Pres_Relay_Master_Log_File>

        • <Sec_L7_Pres_Exec_Master_Log_Pos>

        • <Pri_L8_Pres_Relay_Master_Log_File>

        • <Pri_L8_Pres_Exec_Master_Log_Pos>

      • For MySQL versions 8.4.0 or greater: Relay_Source_Log_File and Exec_Source_Log_Pos

        These will be referred to later as:

        • <Sec_L7_Pres_Relay_Source_Log_File>

        • <Sec_L7_Pres_Exec_Source_Log_Pos>

        • <Pri_L8_Pres_Relay_Source_Log_File>

        • <Pri_L8_Pres_Exec_Source_Log_Pos>

   4. Stop replication by running one of the following commands, depending on your MySQL version:

      • For MySQL versions less than 8.4.0:

        STOP SLAVE;
        

      • For MySQL versions 8.4.0 or greater:

        STOP REPLICA;
        

3. On your secondary Linux 7 presentation server:

   1. Create the $A1BASEDIR environment variable for <UA_home>, replacing <UA_home> with the appropriate value. This is usually opt/assure1:

      export A1BASEDIR=<UA_home>
      

   2. Stop all Unified Assurance services, the broker, the message bus, the web server, and the assure1 database:

      source $A1BASEDIR/.assure1_bashrc
      $A1BASEDIR/bin/BrokerControl --batch stopall
      systemctl stop assure1-broker
      systemctl stop assure1-bus
      systemctl stop assure1-web
      systemctl stop assure1-db
      

   3. Verify that all processes have stopped:

      ps aux | grep assure1
      systemctl status assure1-broker
      systemctl status assure1-bus
      systemctl status assure1-web
      systemctl status assure1-db
      

   4. Create a file called tar-excludes.txt that specifies what to exclude from the backup:

      cat <<EOF > tar-excludes.txt
      $A1BASEDIR/distrib/packages
      $A1BASEDIR/var/chartmuseum
      $A1BASEDIR/var/docker
      $A1BASEDIR/var/registry
      $A1BASEDIR/var/run/docker
      $A1BASEDIR/var/run/docker.pid
      $A1BASEDIR/vendor/docker
      $A1BASEDIR/vendor/helm
      $A1BASEDIR/vendor/kubernetes
      $A1BASEDIR/vendor/rke
      $A1BASEDIR/vendor/submariner
      EOF
      

   5. Back up the $A1BASEDIR directory:

      tar --exclude-from=tar-excludes.txt -czf UA-Linux7-Secondary-Presentation-Backup.tgz $A1BASEDIR
      

   6. Copy the UA-Linux7-Secondary-Presentation-Backup.tgz tar backup file to the new secondary Linux 8 presentation server. You can use any secure transfer protocol. For example, using scp:

      scp UA-Linux7-Secondary-Presentation-Backup.tgz <user>@<hostname>:/home/<user>/
      

4. On your new secondary Linux 8 presentation server:

   1. Move the UA-Linux7-Secondary-Presentation-Backup.tgz file to the <installation_file_directory>:

      mv /home/<user>/UA-Linux7-Secondary-Presentation-Backup.tgz <installation_file_directory>

   2. Create environment the following environment variables, replacing the placeholders with the appropriate values:

      export A1BASEDIR=<UA_home>
      export FILESDIR=<installation_file_directory>
      export OL7FQDN=<Secondary_Linux_7_FQDN>
      export OL8FQDN=<Secondary_Linux_8_FQDN>
      export PRIOL7FQDN=<Primary_Linux_7_FQDN>
      export PRIOL8FQDN=<Primary_Linux_8_FQDN>
      export OL8WEBFQDN=<Secondary_Linux_8_web_FQDN>
      export PRIOL7WEBFQDN=<Primary_Linux_7_web_FQDN>
      export PRIOL8WEBFQDN=<Primary_Linux_8_web_FQDN>
      

   3. Extract the UA-Linux7-Secondary-Presentation-Backup.tgz tar backup file:

      tar -xzf $FILESDIR/UA-Linux7-Secondary-Presentation-Backup.tgz -C /
      

   4. Replace the old server FQDN with the new FQDN:

      grep -rl --exclude-dir={etc,legal,lib,logs,tmp,var} "$OL7FQDN" $A1BASEDIR/* | xargs sed -i "s/$OL7FQDN/$OL8FQDN/g"
      grep -rl --exclude-dir={ssl,priv} "$OL7FQDN" $A1BASEDIR/etc/* | xargs sed -i "s/$OL7FQDN/$OL8FQDN/g"
      

   5. Update the rules checkout paths to point to the new FQDN:

      for CHECKOUT in config core diagrams files
      do
        if [ -d $A1BASEDIR/var/checkouts/$CHECKOUT ] ; then
            cd $A1BASEDIR/var/checkouts/$CHECKOUT &&  $A1BASEDIR/vendor/svn/bin/svn --config-dir $A1BASEDIR/etc/svn relocate https://$PRIOL8WEBFQDN/repos/$CHECKOUT/
        fi
      done
      

   6. For servers with the Database.Historical role (for Elasticsearch database servers), replace the old server FQDN in the Kibana configuration file with the new FQDN:

      sed -i "s/$OL7FQDN/$OL8FQDN/g" $A1BASEDIR/vendor/kibana/config/kibana.yml
      

   7. Remove the old SSL certificates:

      rm -f $A1BASEDIR/etc/ssl/priv/*.pem
      rm -f $A1BASEDIR/etc/ssl/priv/*.old
      > $A1BASEDIR/etc/ssl/priv/index.txt
      > $A1BASEDIR/etc/ssl/priv/index.txt.attr
      > $A1BASEDIR/etc/ssl/priv/serial
      cd $A1BASEDIR/etc/ssl && ls | grep -xv ".*CA.*" | grep -xv priv | xargs rm -rf
      rm -rf $A1BASEDIR/var/rabbitmq/mnesia/*
      

   8. Check the contents of the $A1BASEDIR/etc/ssl directory:

      ls -l $A1BASEDIR/etc/ssl/
      

      The response should look like this:

      -rw-r--r-- 1 60 assure1 1342 Jun 17 14:31 Assure1CA.crt
      -rw-r--r-- 1 60 assure1 5626 Jun 17 14:32 BundleCA.crt
      -rw-r--r-- 1 60 assure1 2294 Jun 17 14:32 BundleCA.jks
      drwxr-xr-x 2 60 assure1   59 Jun 20 08:23 priv
      -rw-r--r-- 1 60 assure1 4284 Jun 17 14:32 SiteCA.crt
      -rw------- 1 60 assure1 1854 Jun 17 14:32 SiteCA.key
      

      Remove any additional files.

   9. Check the contents of the $A1BASEDIR/etc/ssl/priv directory:

      ls -l $A1BASEDIR/etc/ssl/priv/
      

      The response should look like this:

      -rw-r--r-- 1 60 assure1 0 Jun 20 08:23 index.txt
      -rw-r--r-- 1 60 assure1 0 Jun 20 08:23 index.txt.attr
      -rw-r--r-- 1 60 assure1 0 Jun 20 08:23 serial
      

      Confirm that index.txt, index.txt.attr and serial are empty files and remove any additional files.

   10. Extract the collection file contents from the Unified Assurance release package to <installation_file_directory>:

       unzip -d $FILESDIR Oracle-Communications-Unified-Assurance-6.X.X.X.X-el8.zip
       

   11. Install the latest Unified Assurance RPM for Linux 8:

       dnf -y install $FILESDIR/Assure1-6.X.X.X.X-X.el8.x86_64.rpm
       

   12. Create an environment variable for the RPM version number. Do not include the package number or el8, and replace each X with the appropriate number:

       export RPMVERSION=6.X.X.X.X
       

       For example:

       export RPMVERSION=6.0.4.0.2
       

   13. Add the host FQDN for the primary and secondary Linux 8 presentation servers to the <UA_home>/etc/Assure1.conf file:

       sed -i "s/\"DBHost\": \"\"/\"DBHost\": \"$PRIOL8FQDN\"/g" $A1BASEDIR/etc/Assure1.conf
       sed -i "s/\"DBBackupHost\": \"\"/\"DBBackupHost\": \"$OL8FQDN\"/g" $A1BASEDIR/etc/Assure1.conf
       

   14. Generate new SSL certificates:

       $A1BASEDIR/bin/CreateSSLCertificate --Type Host -c $OL8FQDN
       $A1BASEDIR/bin/CreateSSLCertificate --Type SSH
       $A1BASEDIR/bin/CreateSSLCertificate --Type User -c assure1
       $A1BASEDIR/bin/CreateSSLCertificate --Type User -c root
       $A1BASEDIR/bin/CreateSSLCertificate --Type User -c api
       $A1BASEDIR/bin/CreateSSLCertificate --Type User -c kibana
       $A1BASEDIR/bin/CreateSSLCertificate --Type User -c repl
       $A1BASEDIR/bin/CreateSSLCertificate --Type Web -c $OL8WEBFQDN
       

   15. For servers with the Database.Historical role (for Elasticsearch database servers), copy the BundleCA.crt, Host.crt and Host.key files to your Elasticsearch certs directory:

       /bin/cp $A1BASEDIR/etc/ssl/BundleCA.crt $A1BASEDIR/vendor/elasticsearch/config/certs/
       /bin/cp $A1BASEDIR/etc/ssl/Host.crt $A1BASEDIR/vendor/elasticsearch/config/certs/
       /bin/cp $A1BASEDIR/etc/ssl/Host.key $A1BASEDIR/vendor/elasticsearch/config/certs/
       chown assure1:assure1 $A1BASEDIR/vendor/elasticsearch/config/certs/*
       

   16. Copy the following Unified Assurance systemd scripts from the /etc/systemd/system/ directory on the original Linux 7 server to the /etc/systemd/system/ directory on the Linux 8 server:

       • assure1-db.service

       • assure1-bus.service

       • assure1-web.service

   17. Reload the systemctl daemon:

       systemctl daemon-reload
       

   18. Start the assure1 database:

       systemctl start assure1-db
       

   19. On the secondary Linux 8 presentation server's MySQL database, synchronize the new secondary Linux 8 presentation server's MySQL database with the new primary Linux 8 presentation server's database by running one of the following sets of commands as the root MySQL user:

       • For MySQL versions less than 8.4.0:

         STOP SLAVE;
         RESET SLAVE;
         CHANGE MASTER TO MASTER_HOST='<PRIMARY-LINUX8-SERVER-FQDN>', MASTER_LOG_FILE='<Sec_L7_Pres_Relay_Master_Log_File>', MASTER_LOG_POS=<Sec_L7_Pres_Exec_Master_Log_Pos>;
         START SLAVE;
         

         where:

         • <PRIMARY-LINUX8-SERVER-FQDN> is the FQDN of the new primary Linux 8 presentation server.

         • <Sec_L7_Pres_Relay_Master_Log_File> is the Relay_Master_Log_File value saved earlier for the secondary Linux 7 presentation server.

         • <Sec_L7_Pres_Exec_Master_Log_Pos> = the Exec_Master_Log_Pos value saved earlier for the secondary Linux 7 presentation server.

       • For MySQL versions 8.4.0 or greater:

         STOP REPLICA;
         RESET REPLICA;
         CHANGE REPLICATION SOURCE TO SOURCE_HOST='<PRIMARY-LINUX8-SERVER-FQDN>', SOURCE_LOG_FILE='<Sec_L7_Pres_Relay_Source_Log_File>', SOURCE_LOG_POS=<Sec_L7_Pres_Exec_Source_Log_Pos>;
         START REPLICA;
         

         where:

         • <PRIMARY-LINUX8-SERVER-FQDN> is the FQDN of the new primary Linux 8 presentation server.

         • <Sec_L7_Pres_Relay_Source_Log_File> is the Relay_Source_Log_File value saved earlier for the secondary Linux 7 presentation server.

         • <Sec_L7_Pres_Exec_Source_Log_Pos> = the Exec_Source_Log_Pos value saved earlier for the secondary Linux 7 presentation server.

5. On the primary Linux 8 presentation server's MySQL database, run one of the following sets of commands as the root MySQL user:

   • For MySQL versions less than 8.4.0:

     CHANGE MASTER TO MASTER_HOST='<SECONDARY-LINUX8-SERVER-FQDN>', MASTER_LOG_FILE='<Pri_L8_Pres_Relay_Master_Log_File>', MASTER_LOG_POS=<Pri_L8_Pres_Exec_Master_Log_Pos>;
     START SLAVE;
     

     where:

     • <SECONDARY-LINUX8-SERVER-FQDN> is the FQDN of the new secondary Linux 8 presentation server.

     • <Pri_L8_Pres_Relay_Master_Log_File> is the Relay_Master_Log_File value saved earlier for the primary Linux 8 presentation server.

     • <Pri_L8_Pres_Exec_Master_Log_Pos> is the Exec_Master_Log_Pos value saved earlier for the primary Linux 8 presentation server.

   • For MySQL versions 8.4.0 or greater:

     CHANGE REPLICATION SOURCE TO SOURCE_HOST='<SECONDARY-LINUX8-SERVER-FQDN>', SOURCE_LOG_FILE='<Pri_L8_Pres_Relay_Source_Log_File>', SOURCE_LOG_POS=<Pri_L8_Pres_Exec_Source_Log_Pos>;
     START REPLICA;
     

     The placeholder values in the above command should be replaced with the relevant info as follows:

     • <SECONDARY-LINUX8-SERVER-FQDN> is the FQDN of the new secondary Linux 8 presentation server.

     • <Pri_L8_Pres_Relay_Source_Log_File> is the Relay_Source_Log_File value saved earlier for the primary Linux 8 presentation server.

     • <Pri_L8_Pres_Exec_Source_Log_Pos> is the Exec_Source_Log_Pos value saved earlier for the primary Linux 8 presentation server.

6. On the secondary Linux 8 presentation server:

   1. Create an environment variable for the secondary Linux 7 presentation server ID:

      SECSERVERID=$($A1BASEDIR/vendor/mysql/bin/mysql --defaults-file=$A1BASEDIR/etc/my.cnf Assure1 -e "SELECT ServerID FROM Servers WHERE ServerHostFQDN = '$OL7FQDN'" | cut -f1 | grep -v 'ServerID')
      

   2. Set the correct server FQDN in the database, and reset packages to their base 6.0.0.0.0 versions:

      $A1BASEDIR/vendor/mysql/bin/mysql --defaults-file=$A1BASEDIR/etc/my.cnf Assure1 -e "UPDATE Servers SET ServerHostFQDN='$OL8FQDN', WebFQDN='$OL8WEBFQDN', Version='$RPMVERSION' WHERE ServerID = $SECSERVERID"
      $A1BASEDIR/vendor/mysql/bin/mysql --defaults-file=$A1BASEDIR/etc/my.cnf Assure1 -e "UPDATE CoreDatabases SET DBHost = '$OL8FQDN' WHERE DBHost = '$OL7FQDN'"
      $A1BASEDIR/vendor/mysql/bin/mysql --defaults-file=$A1BASEDIR/etc/my.cnf Assure1 -e "UPDATE AuthVendorSystems SET SystemHost = '$OL8FQDN' WHERE SystemHost = '$OL7FQDN'"
      $A1BASEDIR/vendor/mysql/bin/mysql --defaults-file=$A1BASEDIR/etc/my.cnf Assure1 -e "UPDATE Packages SET PackageVersion = '6.0.0.0.0.0' WHERE ServerID = $SECSERVERID"
      

   3. Start RabbitMQ:

      systemctl start assure1-bus
      

   4. Create a new assure1 RabbitMQ user:

      source $A1BASEDIR/.assure1_bashrc
      $A1BASEDIR/bin/Package add-rabbit-user
      

   5. Remove the old Linux 7 packages from the $A1BASEDIR/distrib/packages/ directory:

      rm -f $A1BASEDIR/distrib/packages/*
      

7. On the primary Linux 7 presentation server:

   1. Edit the $A1BASEDIR/etc/Assure1.conf file to replace the old Linux 7 secondary presentation server FQDN with the new Linux 8 secondary presentation server FQDN.

   2. Restart the Broker service:

      systemctl restart assure1-broker
      

8. On the secondary Linux 8 presentation server:

   1. Start the Broker:

      systemctl start assure1-broker
      

   2. Verify that the Broker service has started:

      systemctl status assure1-broker
      

   3. Remove the old Linux 7 ssl and crypto libraries:

      rm -f $A1BASEDIR/lib/libcrypto.s* $A1BASEDIR/lib/libssl.s*
      

   4. Wait for the Broker to start the Unified Assurance services, then update the Unified Assurance package:

      source $A1BASEDIR/.assure1_bashrc
      $A1BASEDIR/bin/Package update --depot $FILESDIR
      

9. For presentation servers that also have the Graph role (for Neo4j database servers):

   1. On the primary Linux 8 presentation-database server:

      1. Switch to the assure1 user:

         su - assure1
         

      2. Set the Graph database to READ ONLY to enable backing up the schema and data:

         a1neo4jroot
         ALTER DATABASE graph SET ACCESS READ ONLY;
         :exit
         

      3. Create the $A1BASEDIR environment variable for <UA_home>, replacing <UA_home> with the appropriate value. This is usually opt/assure1:

         export A1BASEDIR=<UA_home>
         

      4. Back up the Graph database:

         mkdir $A1BASEDIR/tmp/neo4j-dump
         NEO4J_CONF=$A1BASEDIR/etc/neo4j/ $A1BASEDIR/vendor/neo4j/bin/neo4j-admin database backup --to-path=$A1BASEDIR/tmp/neo4j-dump --include-metadata=all --type=full graph
         

   2. On the secondary Linux 8 presentation-database server:

      1. Switch to the assure1 user:

         su - assure1
         

      2. Connect to Neo4j and drop the Graph database:

         a1neo4jroot
         DROP DATABASE Graph;
         :exit
         

      3. Create the $A1BASEDIR environment variable for <UA_home>, replacing <UA_home> with the appropriate value. This is usually opt/assure1:

         export A1BASEDIR=<UA_home>
         

      4. Copy the backup files from the primary Linux 8 presentation-database server to the secondary Linux 8 presentation-database server.

      5. Restore the Graph database from the backup file:

         sudo chown -R assure1:assure1 $A1BASEDIR/tmp/neo4j-dump
         NEO4J_CONF=$A1BASEDIR/etc/neo4j/ $A1BASEDIR/vendor/neo4j/bin/neo4j-admin database restore --from-path=$A1BASEDIR/tmp/neo4j-dump/<backup-file-name> graph
         

         where <backup-file-name> is the name of your Graph database backup file.

      6. Check if the output of the restore command contains the following line:

         You need to execute .../var/neo4j/scripts/graph/restore_metadata.cypher. To execute the file use cypher-shell command with parameter `graph`
         

         If so, run the following command to restore the metadata:

         cat $A1BASEDIR/var/neo4j/scripts/graph/restore_metadata.cypher | A1_JAVA_HOME=$A1BASEDIR/vendor/java NEO4J_PASSWORD=$($A1BASEDIR/bin/JWT -a neo4j -s neo4j) $A1BASEDIR/vendor/neo4j/bin/cypher-shell -a neo4j+ssc://$(hostname):7687 -u neo4j -d system --param "database => 'graph'"
         

         If not, continue.

      7. Connect to Neo4j as the admin user, switch to the system database, and create the Graph database:

         a1neo4jroot
         :use system
         CREATE DATABASE GRAPH;
         :exit
         

         Note:

         If the Graph database already exists, a warning message appears. Ignore it and continue.

   3. On both the primary and secondary Linux 8 presentation-database servers, in the $A1BASEDIR/etc/neo4j/streams.conf file, change the value of the streams.source.enabled property to true. For example:

      streams.source.enabled=true
      

   4. In the Unified Assurance UI:

      1. From the Configuration menu, select Broker Control, then Services.

      2. Select the Neo4j service for the secondary Linux 8 presentation-database server.

         The service details appear.

      3. Set Status to Enabled and click Submit.

      4. Select the Neo4j Kafka Connect service for the primary Linux 8 presentation-database server.

      5. Set Status to Enabled and click Submit.

      6. Select the Neo4j Kafka Connect service for the secondary Linux 8 presentation-database server.

      7. Set Status to Enabled and click Submit.

   5. On the primary Linux 8 presentation-database server, set the Graph database access back to READ WRITE:

      a1neo4jroot
      ALTER DATABASE graph SET ACCESS READ WRITE;
      :exit
      

   6. On the secondary Linux 8 presentation-database server, switch back to the root user:

      su - root
      

10. On the secondary Linux 8 presentation server:

    1. Start the web server:

       systemctl start assure1-web
       

    2. In the $A1BASEDIR/etc/Assure1.conf file, remove the InstallIncomplete entry:

       sed -i '/InstallIncomplete/c\' $A1BASEDIR/etc/Assure1.conf
       

The secondary presentation server upgrade is complete. Continue to upgrade your remaining servers in the order described in Server Upgrade Order.

Task 3: Upgrading Database Servers

Upgrade each database server pair one pair at a time.

Upgrading a Primary Database Server

1. For database server pairs with the Database.Event role (for MySQL database servers), starting with the primary and then repeating on the secondary Linux 7 database server:

   1. Connect to MySQL as the root user.

   2. Get the binlog file name and position by running one of the following commands, depending on your MySQL version:

      • For MySQL versions less than 8.4.0:

        SHOW SLAVE STATUS \G
        

      • For MySQL versions 8.4.0 or greater:

        SHOW REPLICA STATUS \G
        

   3. Copy the values for the following parameters, depending on your MySQL version, and save them somewhere you can refer to them later:

      • For MySQL versions less than 8.4.0: Relay_Master_Log_File and Exec_Master_Log_Pos

        These will be referred to later as:

        • <Pri_L7_DB_Relay_Master_Log_File>

        • <Pri_L7_DB_Exec_Master_Log_Pos>

        • <Sec_L7_DB_Relay_Master_Log_File>

        • <Sec_L7_DB_Exec_Master_Log_Pos>

      • For MySQL versions 8.4.0 or greater: Relay_Source_Log_File and Exec_Source_Log_Pos

        These will be referred to later as:

        • <Pri_L7_DB_Relay_Source_Log_File>

        • <Pri_L7_DB_Exec_Source_Log_Pos>

        • <Sec_L7_DB_Relay_Source_Log_File>

        • <Sec_L7_DB_Exec_Source_Log_Pos>

   4. Stop replication by running one of the following commands, depending on your MySQL version:

      • For MySQL versions less than 8.4.0:

        STOP SLAVE;
        

      • For MySQL versions 8.4.0 or greater:

        STOP REPLICA;
        

2. On your primary and secondary Linux 7 database servers, create the $A1BASEDIR environment variable for <UA_home>, replacing <UA_home> with the appropriate value. This is usually opt/assure1:

   export A1BASEDIR=<UA_home>
   

3. For database server pairs with the Graph role (for Neo4j database servers):

   1. On your primary and secondary Linux 7 database servers, disable Neo4j replication by moving the graph-repl.jar and schema-repl.jar files to the $A1BASEDIR/vendor/neo4j/disabled_plugins/ directory:

      mv $A1BASEDIR/vendor/neo4j/plugins/graph-repl.jar $A1BASEDIR/vendor/neo4j/disabled_plugins/
      mv $A1BASEDIR/vendor/neo4j/plugins/schema-repl.jar $A1BASEDIR/vendor/neo4j/disabled_plugins/
      

      This prevents issues resulting from the old plugins from being run alongside the new plugins.

      Note:

      Depending on the version of Unified Assurance you have installed, graph-repl.jar and schema-repl.jar may no longer exist or may have already been moved to a backup directory. If so, continue to the next step.

   2. If your version of Unified Assurance on Linux 7 is 6.0.4 or higher, do the following on the primary and secondary Linux 7 database servers:

      1. In the $A1BASEDIR/etc/neo4j/streams.conf file, change the value of the streams.source.enabled property to false. For example:

         streams.source.enabled=false
         

      2. Disable and stop the Neo4j Kafka Connect service:

         1. In the Unified Assurance UI, from the Configuration menu, select Broker Control, then select Services.

         2. Select the Neo4j Kafka Connect service.

            The service details appear.

         3. Set Status to Disabled and click Submit.

            In the All Services table, the value in the Status column for the service is updated to Disabled.

         4. In the All Services table, click Stop.

            In the All Services table, the value in the State column for the service is updated to Stopped.

      3. Verify that the Neo4j processes have stopped:

         ps -ef | grep neo4j
         

         The output should not show any running Neo4j processes.

   3. On your secondary Linux 7 database server, restart the Neo4j service:

      1. In to the Unified Assurance UI, from the Configuration menu, select Broker Control, then select Services.

      2. Select the Neo4j service and click Restart.

4. On your primary Linux 7 database server:

   1. Stop all services, the broker, the message bus, the web server, and the assure1 database:

      source $A1BASEDIR/.assure1_bashrc
      $A1BASEDIR/bin/BrokerControl --batch stopall
      systemctl stop assure1-broker
      systemctl stop assure1-db
      

   2. Verify that all processes have stopped:

      ps aux | grep assure1
      systemctl status assure1-broker
      systemctl status assure1-db
      

   3. Create a file called tar-excludes.txt that specifies what to exclude from the backup:

      cat <<EOF > tar-excludes.txt
      $A1BASEDIR/distrib/packages
      $A1BASEDIR/var/chartmuseum
      $A1BASEDIR/var/docker
      $A1BASEDIR/var/registry
      $A1BASEDIR/var/run/docker
      $A1BASEDIR/var/run/docker.pid
      $A1BASEDIR/vendor/docker
      $A1BASEDIR/vendor/helm
      $A1BASEDIR/vendor/kubernetes
      $A1BASEDIR/vendor/rke
      $A1BASEDIR/vendor/submariner
      EOF
      

   4. Back up the $A1BASEDIR directory:

      tar --exclude-from=tar-excludes.txt -czf UA-Linux7-Primary-Database-Backup.tgz $A1BASEDIR
      

   5. Copy the UA-Linux7-Primary-Database-Backup.tgz tar backup file to the new primary Linux 8 database server. You can use any secure transfer protocol. For example, using scp:

      scp UA-Linux7-Primary-Database-Backup.tgz <user>@<hostname>:/home/<user>/
      

5. On your new primary Linux 8 database server:

   1. Move the UA-Linux7-Primary-Database-Backup.tgz file to <installation_file_directory>:

      mv /home/<user>/UA-Linux7-Primary-Database-Backup.tgz <installation_file_directory>
      

   2. Create the following environment variables, replacing the placeholders with the appropriate values:

      export A1BASEDIR=<UA_home>
      export FILESDIR=<installation_file_directory>
      export OL7FQDN=<Primary_DB_Linux_7_FQDN>
      export OL8FQDN=<Primary_DB_Linux_8_FQDN>
      export OL8PRIMARYPRESFQDN=<Primary_Pres_Linux_8_FQDN>
      export OL8SECONDARYPRESFQDN=<Secondary_Pres_Linux_8_FQDN>
      export PRIPRESOL7WEBFQDN=<Primary_Pres_Linux_7_web_FQDN>
      export PRIPRESOL8WEBFQDN=<Primary_Pres_Linux_8_web_FQDN>
      export SECOL7FQDN=<Secondary_DB_Linux_7_FQDN>
      

   3. Extract the UA-Linux7-Primary-Database-Backup.tgz tar backup file:

      tar -xzf $FILESDIR/UA-Linux7-Primary-Database-Backup.tgz -C /
      

   4. Replace the old server FQDN with the new FQDN:

      grep -rl --exclude-dir={etc,legal,lib,logs,tmp,var} "$OL7FQDN" $A1BASEDIR/* | xargs sed -i "s/$OL7FQDN/$OL8FQDN/g"
      grep -rl --exclude-dir={ssl,priv} "$OL7FQDN" $A1BASEDIR/etc/* | xargs sed -i "s/$OL7FQDN/$OL8FQDN/g"
      

   5. Update the rules checkout paths to point to the new FQDN:

      for CHECKOUT in config core diagrams files
      do
        if [ -d $A1BASEDIR/var/checkouts/$CHECKOUT ] ; then
            cd $A1BASEDIR/var/checkouts/$CHECKOUT &&  $A1BASEDIR/vendor/svn/bin/svn --config-dir $A1BASEDIR/etc/svn relocate https://$PRIPRESOL8WEBFQDN/repos/$CHECKOUT/
        fi
      done
      

      Note:

      Not all checkout directories are used on database servers.

   6. For servers with the Database.Historical role (for Elasticsearch database servers), replace the old server FQDN in the Kibana configuration file with the new FQDN:

      sed -i "s/$OL7FQDN/$OL8FQDN/g" $A1BASEDIR/vendor/kibana/config/kibana.yml
      

   7. Remove the old SSL certificates:

      rm -f $A1BASEDIR/etc/ssl/priv/*.pem
      rm -f $A1BASEDIR/etc/ssl/priv/*.old
      > $A1BASEDIR/etc/ssl/priv/index.txt
      > $A1BASEDIR/etc/ssl/priv/index.txt.attr
      > $A1BASEDIR/etc/ssl/priv/serial
      cd $A1BASEDIR/etc/ssl && ls | grep -xv ".*CA.*" | grep -xv priv | xargs rm -rf
      rm -rf $A1BASEDIR/var/rabbitmq/mnesia/*
      

   8. Check the contents of the $A1BASEDIR/etc/ssl directory:

      ls -l $A1BASEDIR/etc/ssl/
      

      The response should look like this:

      -rw-r--r-- 1 60 assure1 1342 Jun 17 14:31 Assure1CA.crt
      -rw-r--r-- 1 60 assure1 5626 Jun 17 14:32 BundleCA.crt
      -rw-r--r-- 1 60 assure1 2294 Jun 17 14:32 BundleCA.jks
      drwxr-xr-x 2 60 assure1   59 Jun 20 08:23 priv
      -rw-r--r-- 1 60 assure1 4284 Jun 17 14:32 SiteCA.crt
      -rw------- 1 60 assure1 1854 Jun 17 14:32 SiteCA.key
      

      Remove any additional files.

   9. Check the contents of the $A1BASEDIR/etc/ssl/priv directory:

      ls -l $A1BASEDIR/etc/ssl/priv/
      

      The response should look like this:

      -rw-r--r-- 1 60 assure1 0 Jun 20 08:23 index.txt
      -rw-r--r-- 1 60 assure1 0 Jun 20 08:23 index.txt.attr
      -rw-r--r-- 1 60 assure1 0 Jun 20 08:23 serial
      

      Confirm that index.txt, index.txt.attr and serial are empty files and remove any additional files.

   10. Extract the collection file contents from the Unified Assurance release package to <installation_file_directory>:

       unzip -d $FILESDIR Oracle-Communications-Unified-Assurance-6.X.X.X.X-el8.zip
       

   11. Install the latest Unified Assurance RPM for Linux 8:

       dnf -y install $FILESDIR/Assure1-6.X.X.X.X-X.el8.x86_64.rpm
       

   12. Create an environment variable for the RPM version number. Do not include the package number or el8, and replace each X with the appropriate number:

       export RPMVERSION=6.X.X.X.X
       

       For example:

       export RPMVERSION=6.0.4.0.2
       

   13. Add the new primary and secondary Linux 8 presentation server FQDNs to the $A1BASEDIR/etc/Assure1.conf file:

       sed -i "s/\"DBHost\": \"\"/\"DBHost\": \"$OL8PRIMARYPRESFQDN\"/g" $A1BASEDIR/etc/Assure1.conf
       sed -i "s/\"DBBackupHost\": \"\"/\"DBBackupHost\": \"$OL8SECONDARYPRESFQDN\"/g" $A1BASEDIR/etc/Assure1.conf
       

   14. Generate new SSL certificates:

       $A1BASEDIR/bin/CreateSSLCertificate --Type Host -c $OL8FQDN
       $A1BASEDIR/bin/CreateSSLCertificate --Type SSH
       $A1BASEDIR/bin/CreateSSLCertificate --Type User -c assure1
       $A1BASEDIR/bin/CreateSSLCertificate --Type User -c root
       $A1BASEDIR/bin/CreateSSLCertificate --Type User -c api
       $A1BASEDIR/bin/CreateSSLCertificate --Type User -c kibana
       $A1BASEDIR/bin/CreateSSLCertificate --Type User -c repl
       

   15. For servers with the Database.Historical role (for Elasticsearch database servers), copy the BundleCA.crt, Host.crt and Host.key files to your Elasticsearch certs directory:

       /bin/cp $A1BASEDIR/etc/ssl/BundleCA.crt $A1BASEDIR/vendor/elasticsearch/config/certs/
       /bin/cp $A1BASEDIR/etc/ssl/Host.crt $A1BASEDIR/vendor/elasticsearch/config/certs/
       /bin/cp $A1BASEDIR/etc/ssl/Host.key $A1BASEDIR/vendor/elasticsearch/config/certs/
       chown assure1:assure1 $A1BASEDIR/vendor/elasticsearch/config/certs/*
       

   16. Copy the following Unified Assurance systemd scripts from the /etc/systemd/system/ directory on the original Linux 7 server to the /etc/systemd/system/ directory on the Linux 8 server:

       • assure1-db.service

       • assure1-bus.service

   17. Reload the systemctl daemon:

       systemctl daemon-reload
       

   18. For database server pairs with the Database.Event role (for MySQL database servers):

       1. Start the assure1 database:

          systemctl start assure1-db
          

       2. Set up the new primary Linux 8 database server as a temporary replication partner for the old secondary Linux 7 database server. This synchronizes any missing data that may have been added to old secondary server while the primary server was down during the upgrade process.

          To set this up, run the following commands as the root MySQL user:

          • For MySQL versions less than 8.4.0:

            STOP SLAVE;
            RESET SLAVE;
            CHANGE MASTER TO MASTER_HOST='<SECONDARY-LINUX7-SERVER-FQDN>', MASTER_LOG_FILE='<Pri_L7_DB_Relay_Master_Log_File>', MASTER_LOG_POS=<Pri_L7_DB_Exec_Master_Log_Pos>;
            START SLAVE;
            

            where:

            • <SECONDARY-LINUX7-SERVER-FQDN> is the FQDN of the original secondary Linux 7 database server.

            • <Pri_L7_DB_elay_Master_Log_File> is the Relay_Master_Log_File value saved earlier for the primary Linux 7 database server.

            • <Pri_L7_DB_Exec_Master_Log_Pos> is the Exec_Master_Log_Pos value saved earlier for the primary Linux 7 database server.

          • For MySQL versions 8.4.0 or greater:

            STOP REPLICA;
            RESET REPLICA;
            CHANGE REPLICATION SOURCE TO SOURCE_HOST='<SECONDARY-LINUX7-SERVER-FQDN>', SOURCE_LOG_FILE='<Pri_L7_DB_Relay_Source_Log_File>', SOURCE_LOG_POS=<Pri_L7_DB_Exec_Source_Log_Pos>;
            START REPLICA;
            

            where:

            • <SECONDARY-LINUX7-SERVER-FQDN> is the FQDN of the original secondary Linux 7 database server.

            • <Pri_L7_DB_Relay_Source_Log_File> is the Relay_Source_Log_File value saved earlier for the primary Linux 7 database server.

            • <Pri_L7_DB_Exec_Source_Log_Pos> is the Exec_Source_Log_Pos value saved earlier for the primary Linux 7 database server.

       3. On the secondary Linux 7 database server's MySQL database, run the following commands as the root MySQL user:

          • For MySQL versions less than 8.4.0:

            CHANGE MASTER TO MASTER_HOST='<PRIMARY-LINUX8-SERVER-FQDN>', MASTER_LOG_FILE='<Sec_L7_DB_Relay_Master_Log_File>', MASTER_LOG_POS=<Sec_L7_DB_Exec_Master_Log_Pos>;
            START SLAVE;
            

            where:

            • <PRIMARY-LINUX8-SERVER-FQDN> is the FQDN of the new Linux 8 primary database server.

            • <Sec_L7_DB_Relay_Master_Log_File> is the Relay_Master_Log_File value saved earlier for the secondary Linux 7 database server.

            • <Sec_L7_DB_Exec_Master_Log_Pos> is the Exec_Master_Log_Pos value saved earlier for the secondary Linux 7 database server.

          • For MySQL versions 8.4.0 or greater:

            CHANGE REPLICATION SOURCE TO SOURCE_HOST='<PRIMARY-LINUX8-SERVER-FQDN>', SOURCE_LOG_FILE='<Sec_L7_DB_Relay_Source_Log_File>', SOURCE_LOG_POS=<Sec_L7_DB_Exec_Source_Log_Pos>;
            START REPLICA;
            

            where

            • <PRIMARY-LINUX8-SERVER-FQDN> is the FQDN of the new Linux 8 primary database server.

            • <Sec_L7_DB_Relay_Source_Log_File> is the Relay_Master_Log_File value saved earlier for the secondary Linux 7 database server.

            • <Sec_L7_DB_Exec_Source_Log_Pos> is the Exec_Master_Log_Pos value saved earlier for the secondary Linux 7 database server.

6. On the primary Linux 8 presentation server:

   1. Create environment variables for the FQDNs of the primary Linux 7 and Linux 8 database servers:

      export OL7FQDN=<Linux_7_Primary_Database_Server_FQDN>
      export OL8FQDN=<Linux_8_Primary_Database_Server_FQDN>
      

   2. Create an environment variable for the primary Linux 7 database server ID:

      PRISERVERID=$($A1BASEDIR/vendor/mysql/bin/mysql --defaults-file=$A1BASEDIR/etc/my.cnf Assure1 -e "SELECT ServerID FROM Servers WHERE ServerHostFQDN = '$OL7FQDN'" | cut -f1 | grep -v 'ServerID')
      

   3. Set the correct server FQDN in the database, and reset packages to their base 6.0.0.0.0 versions:

      $A1BASEDIR/vendor/mysql/bin/mysql --defaults-file=$A1BASEDIR/etc/my.cnf Assure1 -e "UPDATE Servers SET ServerHostFQDN='$OL8FQDN', Version='$RPMVERSION' WHERE ServerID = $PRISERVERID"
      $A1BASEDIR/vendor/mysql/bin/mysql --defaults-file=$A1BASEDIR/etc/my.cnf Assure1 -e "UPDATE CoreDatabases SET DBHost = '$OL8FQDN' WHERE DBHost = '$OL7FQDN'"
      $A1BASEDIR/vendor/mysql/bin/mysql --defaults-file=$A1BASEDIR/etc/my.cnf Assure1 -e "UPDATE AuthVendorSystems SET SystemHost = '$OL8FQDN' WHERE SystemHost = '$OL7FQDN'"
      $A1BASEDIR/vendor/mysql/bin/mysql --defaults-file=$A1BASEDIR/etc/my.cnf Assure1 -e "UPDATE Packages SET PackageVersion = '6.0.0.0.0.0' WHERE ServerID = $PRISERVERID"
      

7. On the primary Linux 8 database server:

   1. Remove the old Linux 7 packages from the $A1BASEDIR/distrib/packages directory:

      rm -f $A1BASEDIR/distrib/packages/*
      

   2. Start the Broker service:

      systemctl start assure1-broker
      

   3. Verify the Broker service has started:

      systemctl status assure1-broker
      

   4. Remove the old Linux 7 SSL and crypto libraries:

      rm -f $A1BASEDIR/lib/libcrypto.s* $A1BASEDIR/lib/libssl.s*
      

8. For database server pairs with the Graph role (for Neo4j database servers):

   1. On the secondary Linux 7 database server:

      1. Switch to the assure1 user:

         su - assure1
         

      2. Set the Graph database to READ ONLY to enable backing up the schema and data:

         a1neo4jroot
         ALTER DATABASE graph SET ACCESS READ ONLY;
         :exit
         

      3. Create the $A1BASEDIR environment variable for <UA_home>, replacing <UA_home with the appropriate value. This is usually opt/assure1:

         export A1BASEDIR=<UA_home>
         

      4. Back up the Graph database:

         • For Unified Assurance version 6.0.3 or lower:

           mkdir $A1BASEDIR/tmp/neo4j-dump
           NEO4J_CONF=$A1BASEDIR/etc/neo4j/ $A1BASEDIR/vendor/neo4j/bin/neo4j-admin backup --database=graph --backup-dir=$A1BASEDIR/tmp/neo4j-dump/ --include-metadata=all
           

         • For Unified Assurance version 6.0.4 or higher:

           mkdir $A1BASEDIR/tmp/neo4j-dump
           NEO4J_CONF=$A1BASEDIR/etc/neo4j/ $A1BASEDIR/vendor/neo4j/bin/neo4j-admin database backup --to-path=$A1BASEDIR/tmp/neo4j-dump --include-metadata=all --type=full graph
           

   2. On the primary Linux 8 database server:

      1. Switch to the assure1 user:

         su - assure1
         

      2. Connect to Neo4j and drop the Graph database:

         a1neo4jroot
         DROP DATABASE Graph;
         :exit
         

         3. Create an environment variable for $A1BASEDIR, replacing <UA_home> with the appropriate value. This is usually opt/assure1:

         export A1BASEDIR=<UA_home>
         

      3. Copy the backup files from the secondary Linux 7 database server to the primary Linux 8 database server.

      4. Restore the Graph database from the backup file by running one of the following sets of commands:

         • For Unified Assurance version 6.0.3 or lower:

           sudo chown -R assure1:assure1 $A1BASEDIR/tmp/neo4j-dump
           NEO4J_CONF=$A1BASEDIR/etc/neo4j/ $A1BASEDIR/vendor/neo4j/bin/neo4j-admin restore --database=graph --from=$A1BASEDIR/tmp/neo4j-dump/graph
           

         • For Unified Assurance version 6.0.4 or higher:

           sudo chown -R assure1:assure1 $A1BASEDIR/tmp/neo4j-dump
           NEO4J_CONF=$A1BASEDIR/etc/neo4j/ $A1BASEDIR/vendor/neo4j/bin/neo4j-admin database restore --from-path=$A1BASEDIR/tmp/neo4j-dump/<backup-file-name> graph
           

           where <backup-file-name> is the name of your Graph database backup file.

      5. Check if the output of the restore command contains the following line:

         You need to execute .../var/neo4j/scripts/graph/restore_metadata.cypher. To execute the file use cypher-shell command with parameter `graph`
         

         If so, run the following command to restore the metadata:

         cat $A1BASEDIR/var/neo4j/scripts/graph/restore_metadata.cypher | A1_JAVA_HOME=$A1BASEDIR/vendor/java NEO4J_PASSWORD=$($A1BASEDIR/bin/JWT -a neo4j -s neo4j) $A1BASEDIR/vendor/neo4j/bin/cypher-shell -a neo4j+ssc://$(hostname):7687 -u neo4j -d system --param "database => 'graph'"
         

         If not, continue.

      6. Connect to Graph as the admin user, switch to the system DB, and create the Graph database:

         a1neo4jroot
         :use system
         CREATE DATABASE GRAPH;
         :exit
         

         Note:

         If the Graph database already exists, a warning message appears. Ignore it and continue.

      7. Switch back to the root user:

         su - root
         

   3. In the Unified Assurance UI:

      1. From the Configuration menu, select Broker Control, then Services.

      2. Select the Neo4j service for the secondary Linux 7 database server.

         The service details appear.

      3. Set Status to Disabled and click Submit.

         In the All Services table, the value in the Status column for the service is updated to Disabled.

      4. In the All Services table, click Stop.

         In the All Services table, the value in the State column for the service is updated to Stopped.

9. On the primary Linux 8 database server:

   1. Wait for the Broker to start the Unified Assurance services, then update the Unified Assurance package:

      source $A1BASEDIR/.assure1_bashrc
      $A1BASEDIR/bin/Package update --depot $FILESDIR
      

   2. In the $A1BASEDIR/etc/Assure1.conf file, remove the InstallIncomplete entry:

      sed -i '/InstallIncomplete/c\' $A1BASEDIR/etc/Assure1.conf
      

The primary database server upgrade is complete. Upgrade the secondary database server in the pair and continue to upgrade your remaining servers in the order described in Server Upgrade Order.

Upgrading a Secondary Database Server

1. For database server pairs with the Database.Event role (for MySQL database servers), starting with the secondary Linux 7 database server and then repeating on the new primary Linux 8 database server:

   1. Connect to MySQL as the root user.

   2. Get the binlog file name and position by running one of the following commands, depending on your MySQL version:

      • For MySQL versions less than 8.4.0:

        SHOW SLAVE STATUS \G
        

      • For MySQL versions 8.4.0 or greater:

        SHOW REPLICA STATUS \G
        

   3. Copy the values for the following parameters, depending on your MySQL version, and save them somewhere you can refer to them later:

      • For MySQL versions less than 8.4.0: Relay_Master_Log_File and Exec_Master_Log_Pos

        These will be referred to later as:

        • <Sec_L7_DB_Relay_Master_Log_File>

        • <Sec_L7_DB_Exec_Master_Log_Pos>

        • <Pri_L8_DB_Relay_Master_Log_File>

        • <Pri_L8_DB_Exec_Master_Log_Pos>

      • For MySQL versions 8.4.0 or greater: Relay_Source_Log_File and Exec_Source_Log_Pos

        These will be referred to later as:

        • <Sec_L7_DB_Relay_Source_Log_File>

        • <Sec_L7_DB_Exec_Source_Log_Pos>

        • <Pri_L8_DB_Relay_Source_Log_File>

        • <Pri_L8_DB_Exec_Source_Log_Pos>

   4. Stop replication by running one of the following commands, depending on your MySQL version:

      • For MySQL versions less than 8.4.0:

        STOP SLAVE;
        

      • For MySQL versions 8.4.0 or greater:

        STOP REPLICA;
        

2. On your secondary Linux 7 database server:

   1. Create the $A1BASEDIR environment variable for <UA_home>, replacing <UA_home> with the appropriate value. This is usually opt/assure1:

      export A1BASEDIR=<UA_home>
      

   2. Stop all Unified Assurance services, the broker, and the assure1 database:

      source $A1BASEDIR/.assure1_bashrc
      $A1BASEDIR/bin/BrokerControl --batch stopall
      systemctl stop assure1-broker
      systemctl stop assure1-db
      

   3. Verify that all processes have stopped:

      ps aux | grep assure1
      systemctl status assure1-broker
      systemctl status assure1-db
      

   4. Create a file called tar-excludes.txt that specifies what to exclude from the backup:

      cat <<EOF > tar-excludes.txt
      $A1BASEDIR/distrib/packages
      $A1BASEDIR/var/chartmuseum
      $A1BASEDIR/var/docker
      $A1BASEDIR/var/registry
      $A1BASEDIR/var/run/docker
      $A1BASEDIR/var/run/docker.pid
      $A1BASEDIR/vendor/docker
      $A1BASEDIR/vendor/helm
      $A1BASEDIR/vendor/kubernetes
      $A1BASEDIR/vendor/rke
      $A1BASEDIR/vendor/submariner
      EOF
      

   5. Back up the $A1BASEDIR directory:

      tar --exclude-from=tar-excludes.txt -czf UA-Linux7-Secondary-Database-Backup.tgz $A1BASEDIR
      

   6. Copy the UA-Linux7-Secondary-Database-Backup.tgz tar backup file to the new secondary Linux 8 database server. You can use any secure transfer protocol. For example, using scp:

      scp UA-Linux7-Secondary-Database-Backup.tgz <user>@<hostname>:/home/<user>/
      

3. On your new secondary Linux 8 database server:

   1. Move the UA-Linux7-Secondary-Database-Backup.tgz file to <installation_file_directory>:

      mv /home/<user>/UA-Linux7-Secondary-Database-Backup.tgz <installation_file_directory>
      

   2. Create the following environment variables, replacing the placeholders with the appropriate values:

      export A1BASEDIR=<UA_home>
      export FILESDIR=<installation_file_directory>
      export OL7FQDN=<Secondary_DB_Linux_7_FQDN>
      export OL8FQDN=<Secondary_DB_Linux_8_FQDN>
      export OL8PRIMARYPRESFQDN=<Primary_Pres_Linux_8_FQDN>
      export OL8SECONDARYPRESFQDN=<Secondary_Pres_Linux_8_FQDN>
      export PRIPRESOL7WEBFQDN=<Primary_Pres_Linux_7_web_FQDN>
      export PRIPRESOL8WEBFQDN=<Primary_Pres_Linux_8_web_FQDN>
      export PRIOL7FQDN=<Primary_DB_Linux_7_FQDN>
      export PRIOL8FQDN=<Primary_DB_Linux_7_FQDN>
      

   3. Extract the UA-Linux7-Secondary-Database-Backup.tgz tar backup file:

      tar -xzf $FILESDIR/UA-Linux7-Secondary-Database-Backup.tgz -C /
      

   4. Replace the old server FQDN with the new FQDN:

      grep -rl --exclude-dir={etc,legal,lib,logs,tmp,var} "$OL7FQDN" $A1BASEDIR/* | xargs sed -i "s/$OL7FQDN/$OL8FQDN/g"
      grep -rl --exclude-dir={ssl,priv} "$OL7FQDN" $A1BASEDIR/etc/* | xargs sed -i "s/$OL7FQDN/$OL8FQDN/g"
      

   5. Update the rules checkout paths to point to the new FQDN:

      for CHECKOUT in config core diagrams files
      do
        if [ -d $A1BASEDIR/var/checkouts/$CHECKOUT ] ; then
            cd $A1BASEDIR/var/checkouts/$CHECKOUT &&  $A1BASEDIR/vendor/svn/bin/svn --config-dir $A1BASEDIR/etc/svn relocate https://$PRIPRESOL8WEBFQDN/repos/$CHECKOUT/
        fi
      done
      

      Note:

      Not all checkout directories are used on database servers.

   6. For servers with the Database.Historical role (for Elasticsearch database servers), replace the old server FQDN in the Kibana configuration file with the new FQDN:

      sed -i "s/$OL7FQDN/$OL8FQDN/g" $A1BASEDIR/vendor/kibana/config/kibana.yml
      

   7. Remove the old SSL certificates:

      rm -f $A1BASEDIR/etc/ssl/priv/*.pem
      rm -f $A1BASEDIR/etc/ssl/priv/*.old
      > $A1BASEDIR/etc/ssl/priv/index.txt
      > $A1BASEDIR/etc/ssl/priv/index.txt.attr
      > $A1BASEDIR/etc/ssl/priv/serial
      cd $A1BASEDIR/etc/ssl && ls | grep -xv ".*CA.*" | grep -xv priv | xargs rm -rf
      rm -rf $A1BASEDIR/var/rabbitmq/mnesia/*
      

   8. Check the contents of the $A1BASEDIR/etc/ssl directory:

      ls -l $A1BASEDIR/etc/ssl/
      

      The response should look like this:

      -rw-r--r-- 1 60 assure1 1342 Jun 17 14:31 Assure1CA.crt
      -rw-r--r-- 1 60 assure1 5626 Jun 17 14:32 BundleCA.crt
      -rw-r--r-- 1 60 assure1 2294 Jun 17 14:32 BundleCA.jks
      drwxr-xr-x 2 60 assure1   59 Jun 20 08:23 priv
      -rw-r--r-- 1 60 assure1 4284 Jun 17 14:32 SiteCA.crt
      -rw------- 1 60 assure1 1854 Jun 17 14:32 SiteCA.key
      

      Remove any additional files.

   9. Check the contents of the $A1BASEDIR/etc/ssl/priv directory:

      ls -l $A1BASEDIR/etc/ssl/priv/
      

      The response should look like this:

      -rw-r--r-- 1 60 assure1 0 Jun 20 08:23 index.txt
      -rw-r--r-- 1 60 assure1 0 Jun 20 08:23 index.txt.attr
      -rw-r--r-- 1 60 assure1 0 Jun 20 08:23 serial
      

      Confirm that index.txt, index.txt.attr and serial are empty files and remove any additional files.

   10. Extract the collection file contents from the Unified Assurance release package to <installation_file_directory>:

       unzip -d $FILESDIR Oracle-Communications-Unified-Assurance-6.X.X.X.X-el8.zip
       

   11. Install the latest Unified Assurance RPM for Linux 8:

       dnf -y install $FILESDIR/Assure1-6.X.X.X.X-X.el8.x86_64.rpm
       

   12. Create an environment variable for the RPM version number. Do not include the package number or el8, and replace each X with the appropriate number:

       export RPMVERSION=6.X.X.X.X
       

       For example:

       export RPMVERSION=6.0.4.0.2
       

   13. Add the host FQDN for the Linux 8 primary and secondary presentation servers to the <UA_home>/etc/Assure1.conf file:

       sed -i "s/\"DBHost\": \"\"/\"DBHost\": \"$OL8PRIMARYPRESFQDN\"/g" $A1BASEDIR/etc/Assure1.conf
       sed -i "s/\"DBBackupHost\": \"\"/\"DBBackupHost\": \"$OL8SECONDARYPRESFQDN\"/g" $A1BASEDIR/etc/Assure1.conf
       

   14. Generate new SSL certificates:

       $A1BASEDIR/bin/CreateSSLCertificate --Type Host -c $OL8FQDN
       $A1BASEDIR/bin/CreateSSLCertificate --Type SSH
       $A1BASEDIR/bin/CreateSSLCertificate --Type User -c assure1
       $A1BASEDIR/bin/CreateSSLCertificate --Type User -c root
       $A1BASEDIR/bin/CreateSSLCertificate --Type User -c api
       $A1BASEDIR/bin/CreateSSLCertificate --Type User -c kibana
       $A1BASEDIR/bin/CreateSSLCertificate --Type User -c repl
       

   15. For servers with the Database.Historical role (for Elasticsearch database servers), copy the BundleCA.crt, Host.crt and Host.key files to your Elasticsearch certs directory:

       /bin/cp $A1BASEDIR/etc/ssl/BundleCA.crt $A1BASEDIR/vendor/elasticsearch/config/certs/
       /bin/cp $A1BASEDIR/etc/ssl/Host.crt $A1BASEDIR/vendor/elasticsearch/config/certs/
       /bin/cp $A1BASEDIR/etc/ssl/Host.key $A1BASEDIR/vendor/elasticsearch/config/certs/
       chown assure1:assure1 $A1BASEDIR/vendor/elasticsearch/config/certs/*
       

   16. Copy the following Unified Assurance systemd scripts from the /etc/systemd/system/ directory on the secondary Linux 7 database server to the /etc/systemd/system/ directory on the secondary Linux 8 database server:

       • assure1-db.service

       • assure1-bus.service

   17. Reload the systemctl daemon:

       systemctl daemon-reload
       

4. For database server pairs with the Database.Event role (for MySQL database servers):

   1. On the secondary Linux 8 database server, start the assure1 database:

      systemctl start assure1-db
      

   2. On the secondary Linux 8 database server's MySQL database, synchronize the new secondary Linux 8 database server's MySQL database with the new primary Linux 8 database server's database by running one of the following sets of commands as the root MySQL user:

      • For MySQL versions less than 8.4.0:

        STOP SLAVE;
        RESET SLAVE;
        CHANGE MASTER TO MASTER_HOST='<PRIMARY-LINUX8-SERVER-FQDN>', MASTER_LOG_FILE='<Sec_L7_DB_Relay_Master_Log_File>', MASTER_LOG_POS=<Sec_L7_DB_Exec_Master_Log_Pos>;
        START SLAVE;
        

        where:

        • <PRIMARY-LINUX8-SERVER-FQDN> is the FQDN of the new primary Linux 8 database server.

        • <Sec_L7_DB_Relay_Master_Log_File> is the Relay_Master_Log_File value saved earlier for the secondary Linux 7 database server.

        • <Sec_L7_DB_Exec_Master_Log_Pos> is the Exec_Master_Log_Pos value saved earlier for the secondary Linux 7 database server.

      • For MySQL versions 8.4.0 or greater:

        STOP REPLICA;
        RESET REPLICA;
        CHANGE REPLICATION SOURCE TO SOURCE_HOST='<PRIMARY-LINUX8-SERVER-FQDN>', SOURCE_LOG_FILE='<Relay_Source_Log_File>', SOURCE_LOG_POS=<Exec_Source_Log_Pos>;
        START REPLICA;
        

        where:

        • <PRIMARY-LINUX8-SERVER-FQDN> is the FQDN of the new primary Linux 8 database server.

        • <Sec_L7_DB_Relay_Source_Log_File> is the Relay_Source_Log_File value saved earlier for the secondary Linux 7 database server.

        • <Sec_L7_DB_Exec_Source_Log_Pos> is the Exec_Source_Log_Pos value saved earlier for the secondary Linux 7 database server.

   3. On the primary Linux 8 database server's MySQL database, run one of the following sets of commands as the root MySQL user:

      • For MySQL versions less than 8.4.0:

        CHANGE MASTER TO MASTER_HOST='<SECONDARY-LINUX8-SERVER-FQDN>', MASTER_LOG_FILE='<Pri_L8_DB_Relay_Master_Log_File>', MASTER_LOG_POS=<Pri_L8_DB_Exec_Master_Log_Pos>;
        START SLAVE;
        

        where:

        • <SECONDARY-LINUX8-SERVER-FQDN> is the FQDN of the new secondary Linux 8 database server.

        • <Pri_L8_DB_Relay_Master_Log_File> is the Relay_Master_Log_File value saved earlier for the primary Linux 8 database server.

        • <Pri_L8_DB_Exec_Master_Log_Pos> is the Exec_Master_Log_Pos value saved earlier for the primary Linux 8 database server.

      • For MySQL versions 8.4.0 or greater:

        CHANGE REPLICATION SOURCE TO SOURCE_HOST='<SECONDARY-LINUX8-SERVER-FQDN>', SOURCE_LOG_FILE='<Pri_L8_DB_Relay_Source_Log_File>', SOURCE_LOG_POS=<Pri_L8_DB_Exec_Source_Log_Pos>;
        START REPLICA;
        

        where:

        • <SECONDARY-LINUX8-SERVER-FQDN> is the FQDN of the new secondary Linux 8 database server.

        • <Pri_L8_DB_Relay_Source_Log_File> is the Relay_Source_Log_File value saved earlier for the primary Linux 8 database server.

        • <Pri_L8_DB_Exec_Source_Log_Pos> is the Exec_Source_Log_Pos value saved earlier for the primary Linux 8 database server.

5. On the primary Linux 8 presentation server:

   1. Create environment variables for the FQDNs of the secondary Linux 7 and Linux 8 database servers:

      export OL7FQDN=<Linux_7_Secondary_Database_Server_FQDN>
      export OL8FQDN=<Linux_8_Secondary_Database_Server_FQDN>
      

   2. Create an environment variable for the secondary Linux 7 database server ID:

      SECSERVERID=$($A1BASEDIR/vendor/mysql/bin/mysql --defaults-file=$A1BASEDIR/etc/my.cnf Assure1 -e "SELECT ServerID FROM Servers WHERE ServerHostFQDN = '$OL7FQDN'" | cut -f1 | grep -v 'ServerID')
      

   3. Set the correct server FQDN in the database, and reset packages to their base 6.0.0.0.0 versions:

      $A1BASEDIR/vendor/mysql/bin/mysql --defaults-file=$A1BASEDIR/etc/my.cnf Assure1 -e "UPDATE Servers SET ServerHostFQDN='$OL8FQDN', Version='$RPMVERSION' WHERE ServerID = $SECSERVERID"
      $A1BASEDIR/vendor/mysql/bin/mysql --defaults-file=$A1BASEDIR/etc/my.cnf Assure1 -e "UPDATE CoreDatabases SET DBHost = '$OL8FQDN' WHERE DBHost = '$OL7FQDN'"
      $A1BASEDIR/vendor/mysql/bin/mysql --defaults-file=$A1BASEDIR/etc/my.cnf Assure1 -e "UPDATE AuthVendorSystems SET SystemHost = '$OL8FQDN' WHERE SystemHost = '$OL7FQDN'"
      $A1BASEDIR/vendor/mysql/bin/mysql --defaults-file=$A1BASEDIR/etc/my.cnf Assure1 -e "UPDATE Packages SET PackageVersion = '6.0.0.0.0.0' WHERE ServerID = $SECSERVERID"
      

6. On the secondary Linux 8 secondary database server:

   1. Remove the old Linux 7 packages from $A1BASEDIR/distrib/packages/ directory:

      rm -f $A1BASEDIR/distrib/packages/*
      

   2. Start the Broker:

      systemctl start assure1-broker
      

   3. Verify that the Broker service has started:

      systemctl status assure1-broker
      

   4. Remove the old Linux 7 ssl and crypto libraries:

      rm -f $A1BASEDIR/lib/libcrypto.s* $A1BASEDIR/lib/libssl.s*
      

   5. Wait for the Broker to start the Unified Assurance services, then update the Unified Assurance package:

      source $A1BASEDIR/.assure1_bashrc
      $A1BASEDIR/bin/Package update --depot $FILESDIR
      

   6. Wait for the package update process to complete.

7. For database server pairs with the Graph role (for Neo4j database servers):

   1. On the primary Linux 8 database server:

      1. Switch to the assure1 user:

         su - assure1
         

      2. Set the Graph database to READ ONLY to enable backing up the schema and data:

         a1neo4jroot
         ALTER DATABASE graph SET ACCESS READ ONLY;
         :exit
         

      3. Create the $A1BASEDIR environment variable for <UA_home>, replacing <UA_home> with the appropriate value. This is usually opt/assure1:

         export A1BASEDIR=<UA_home>
         

      4. Back up the Graph database:

         mkdir $A1BASEDIR/tmp/neo4j-dump
         NEO4J_CONF=$A1BASEDIR/etc/neo4j/ $A1BASEDIR/vendor/neo4j/bin/neo4j-admin database backup --to-path=$A1BASEDIR/tmp/neo4j-dump --include-metadata=all --type=full graph
         

   2. On the secondary Linux 8 database server:

      1. Switch to the assure1 user:

         su - assure1
         

      2. Connect to Neo4j and drop the Graph database:

         a1neo4jroot
         DROP DATABASE Graph;
         :exit
         

      3. Create the $A1BASEDIR environment variable for <UA_home>, replacing <UA_home> with the appropriate value. This is usually opt/assure1:

         export A1BASEDIR=<UA_home>
         

      4. Copy the backup files from the primary Linux 8 database server onto the Linux 8 secondary database server.

      5. Restore the Graph database from the backup file:

         sudo chown -R assure1:assure1 $A1BASEDIR/tmp/neo4j-dump
         NEO4J_CONF=$A1BASEDIR/etc/neo4j/ $A1BASEDIR/vendor/neo4j/bin/neo4j-admin database restore --from-path=$A1BASEDIR/tmp/neo4j-dump/<backup-file-name> graph
         

         where <backup-file-name> is the name of your Graph database backup file.

      6. Check if the output of the restore command contains the following line:

         You need to execute .../var/neo4j/scripts/graph/restore_metadata.cypher. To execute the file use cypher-shell command with parameter `graph`
         

         If so, run the following command to restore the metadata:

         cat $A1BASEDIR/var/neo4j/scripts/graph/restore_metadata.cypher | A1_JAVA_HOME=$A1BASEDIR/vendor/java NEO4J_PASSWORD=$($A1BASEDIR/bin/JWT -a neo4j -s neo4j) $A1BASEDIR/vendor/neo4j/bin/cypher-shell -a neo4j+ssc://$(hostname):7687 -u neo4j -d system --param "database => 'graph'"
         

         If not, continue.

      7. Connect to Neo4j as the admin user, switch to the system database, and create the Graph database:

         a1neo4jroot
         :use system
         CREATE DATABASE GRAPH;
         :exit
         

         Note:

         If the Graph database already exists, a warning message appears. Ignore it and continue.

   3. On both the primary and secondary Linux 8 database servers, in the $A1BASEDIR/etc/neo4j/streams.conf file, change the value of the streams.source.enabled property to true. For example:

      streams.source.enabled=true
      

   4. In the Unified Assurance UI:

      1. From the Configuration menu, select Broker Control, then Services.

      2. Select the Neo4j service for the secondary Linux 8 database server.

         The service details appear.

      3. Set Status to Enabled and click Submit.

      4. Select the Neo4j Kafka Connect service for the primary Linux 8 database server.

      5. Set Status to Enabled and click Submit.

      6. Select the Neo4j Kafka Connect service for the secondary Linux 8 database server.

      7. Set Status to Enabled and click Submit.

   5. On the primary Linux 8 database server, set the Graph database access back to READ WRITE:

      a1neo4jroot
      ALTER DATABASE graph SET ACCESS READ WRITE;
      :exit
      

   6. On the secondary Linux 8 database server, switch back to the root user:

      su - root
      

8. On the secondary Linux 8 database server, in the $A1BASEDIR/etc/Assure1.conf file, remove the InstallIncomplete entry:

   sed -i '/InstallIncomplete/c\' $A1BASEDIR/etc/Assure1.conf
   

The secondary database server upgrade is complete.

Repeat the primary and secondary database server upgrade process for any remaining database pairs, then continue to upgrade your remaining servers in the order described in Server Upgrade Order.

Task 4: Upgrading Collection Servers

Repeat the following steps on each of your collection servers:

1. On your existing Linux 7 collection server:

   1. Create the $A1BASEDIR environment variable for <UA_home>, replacing <UA_home> with the appropriate value. This is usually opt/assure1:

      export A1BASEDIR=<UA_home>
      

   2. Stop all services and the broker:

      source $A1BASEDIR/.assure1_bashrc
      $A1BASEDIR/bin/BrokerControl --batch stopall
      systemctl stop assure1-broker
      

   3. Verify that all processes have stopped:

      ps aux | grep assure1
      systemctl status assure1-broker
      

   4. Create a file called tar-excludes.txt that specifies what to exclude from the backup:

      cat <<EOF > tar-excludes.txt
      $A1BASEDIR/distrib/packages
      $A1BASEDIR/var/chartmuseum
      $A1BASEDIR/var/docker
      $A1BASEDIR/var/registry
      $A1BASEDIR/var/run/docker
      $A1BASEDIR/var/run/docker.pid
      $A1BASEDIR/vendor/docker
      $A1BASEDIR/vendor/helm
      $A1BASEDIR/vendor/kubernetes
      $A1BASEDIR/vendor/rke
      $A1BASEDIR/vendor/submariner
      EOF
      

   5. Back up the $A1BASEDIR directory:

      tar --exclude-from=tar-excludes.txt -czf UA-Linux7-Collection-Backup.tgz $A1BASEDIR
      

   6. Copy the UA-Linux7-Collection-Backup.tgz tar backup file to the new Linux 8 collection server. You can use any secure transfer protocol. For example, using scp:

      scp UA-Linux7-Collection-Backup.tgz <user>@<hostname>:/home/<user>/
      

2. On the new Linux 8 collection server:

   1. Move the UA-Linux7-Collection-Backup.tgz file to <installation_file_directory>:

      mv /home/<user>/UA-Linux7-Collection-Backup.tgz <installation_file_directory>
      

   2. Create the following environment variables, replacing the placeholders with the appropriate values:

      export A1BASEDIR=<UA_home>
      export FILESDIR=<installation_file_directory>
      export OL7FQDN=<Collection_Linux_7_FQDN>
      export OL8FQDN=<Collection_Linux_8_FQDN>
      export OL8PRIMARYPRESFQDN=<Primary_Pres_Linux_8_FQDN>
      export OL8SECONDARYPRESFQDN=<Secondary_Pres_Linux_8_FQDN>
      export PRIPRESOL7WEBFQDN=<Primary_Pres_Linux_7_web_FQDN>
      export PRIPRESOL8WEBFQDN=<Primary_Pres_Linux_8_web_FQDN>
      

   3. Extract the UA-Linux7-Collection-Backup.tgz tar backup file:

      tar -xzf $FILESDIR/UA-Linux7-Collection-Backup.tgz -C /
      

   4. Replace the old server FQDN with the new FQDN:

      grep -rl --exclude-dir={etc,legal,lib,logs,tmp,var} "$OL7FQDN" $A1BASEDIR/* | xargs sed -i "s/$OL7FQDN/$OL8FQDN/g"
      grep -rl --exclude-dir={ssl,priv} "$OL7FQDN" $A1BASEDIR/etc/* | xargs sed -i "s/$OL7FQDN/$OL8FQDN/g"
      

   5. Update the rules checkout paths to point to the new FQDN:

      for CHECKOUT in config core diagrams files
      do
        if [ -d $A1BASEDIR/var/checkouts/$CHECKOUT ] ; then
            cd $A1BASEDIR/var/checkouts/$CHECKOUT &&  $A1BASEDIR/vendor/svn/bin/svn --config-dir $A1BASEDIR/etc/svn relocate https://$PRIPRESOL8WEBFQDN/repos/$CHECKOUT/
        fi
      done
      

      Note:

      Not all checkout directories are used on collection servers.

   6. Remove the old SSL certificates:

      rm -f $A1BASEDIR/etc/ssl/priv/*.pem
      rm -f $A1BASEDIR/etc/ssl/priv/*.old
      > $A1BASEDIR/etc/ssl/priv/index.txt
      > $A1BASEDIR/etc/ssl/priv/index.txt.attr
      > $A1BASEDIR/etc/ssl/priv/serial
      cd $A1BASEDIR/etc/ssl && ls | grep -xv ".*CA.*" | grep -xv priv | xargs rm -rf
      rm -rf $A1BASEDIR/var/rabbitmq/mnesia/*
      

      The cert directories should now look like this:

      ls -l $A1BASEDIR/etc/ssl/
      -rw-r--r-- 1 60 assure1 1342 Jun 17 14:31 Assure1CA.crt
      -rw-r--r-- 1 60 assure1 5626 Jun 17 14:32 BundleCA.crt
      -rw-r--r-- 1 60 assure1 2294 Jun 17 14:32 BundleCA.jks
      drwxr-xr-x 2 60 assure1   59 Jun 20 08:23 priv
      -rw-r--r-- 1 60 assure1 4284 Jun 17 14:32 SiteCA.crt
      -rw------- 1 60 assure1 1854 Jun 17 14:32 SiteCA.key
      
      ls -l $A1BASEDIR/etc/ssl/priv/
      -rw-r--r-- 1 60 assure1 0 Jun 20 08:23 index.txt
      -rw-r--r-- 1 60 assure1 0 Jun 20 08:23 index.txt.attr
      -rw-r--r-- 1 60 assure1 0 Jun 20 08:23 serial
      

   7. Confirm that index.txt, index.txt.attr and serial are empty and remove any additional files in the ssl and ssl/priv directories.

   8. Extract the collection file contents from the Unified Assurance release package to <installation_file_directory>:

      unzip -d $FILESDIR Oracle-Communications-Unified-Assurance-6.X.X.X.X-el8.zip
      

   9. Install the latest Unified Assurance RPM for Linux 8:

      dnf -y install $FILESDIR/Assure1-6.X.X.X.X-X.el8.x86_64.rpm
      

   10. Create an environment variable for the RPM version number. Do not include the package number or el8, and replace each X with the appropriate number:

       export RPMVERSION=6.X.X.X.X
       

       For example:

       export RPMVERSION=6.0.4.0.2
       

   11. Add the new primary and secondary Linux 8 presentation server FQDNs to the $A1BASEDIR/etc/Assure1.conf file:

       sed -i "s/\"DBHost\": \"\"/\"DBHost\": \"$OL8PRIMARYPRESFQDN\"/g" $A1BASEDIR/etc/Assure1.conf
       sed -i "s/\"DBBackupHost\": \"\"/\"DBBackupHost\": \"$OL8SECONDARYPRESFQDN\"/g" $A1BASEDIR/etc/Assure1.conf
       

   12. Generate new SSL certificates:

       $A1BASEDIR/bin/CreateSSLCertificate --Type Host -c $OL8FQDN
       $A1BASEDIR/bin/CreateSSLCertificate --Type SSH
       $A1BASEDIR/bin/CreateSSLCertificate --Type User -c assure1
       $A1BASEDIR/bin/CreateSSLCertificate --Type User -c root
       $A1BASEDIR/bin/CreateSSLCertificate --Type User -c api
       $A1BASEDIR/bin/CreateSSLCertificate --Type User -c kibana
       $A1BASEDIR/bin/CreateSSLCertificate --Type User -c repl
       

3. On the primary Linux 8 primary presentation server:

   1. Create environment variables for the FQDNs of the Linux 7 and Linux 8 collection servers:

      export OL7FQDN=<Linux_7_Collection_Server_FQDN>
      export OL8FQDN=<Linux_8_Collection_Server_FQDN>
      

   2. Create an environment variable for the Linux 7 collection server ID:

      COLSERVERID=$($A1BASEDIR/vendor/mysql/bin/mysql --defaults-file=$A1BASEDIR/etc/my.cnf Assure1 -e "SELECT ServerID FROM Servers WHERE ServerHostFQDN = '$OL7FQDN'" | cut -f1 | grep -v 'ServerID')
      

   3. Set the correct server FQDN in the database, and reset Packages to their base 6.0.0.0.0 versions:

      $A1BASEDIR/vendor/mysql/bin/mysql --defaults-file=$A1BASEDIR/etc/my.cnf Assure1 -e "UPDATE Servers SET ServerHostFQDN='$OL8FQDN', Version='$RPMVERSION' WHERE ServerID = $COLSERVERID"
      $A1BASEDIR/vendor/mysql/bin/mysql --defaults-file=$A1BASEDIR/etc/my.cnf Assure1 -e "UPDATE Packages SET PackageVersion = '6.0.0.0.0.0' WHERE ServerID = $COLSERVERID"
      

4. On the Linux 8 collection server:

   1. Remove the old Linux 7 packages from the $A1BASEDIR/distrib/packages directory

      rm -f $A1BASEDIR/distrib/packages/*
      

   2. Start the Broker service:

      systemctl start assure1-broker
      

   3. Verify the Broker service has started:

      systemctl status assure1-broker
      

   4. Remove the old Linux 7 SSL and crypto libraries:

      rm -f $A1BASEDIR/lib/libcrypto.s* $A1BASEDIR/lib/libssl.s*
      

   5. Wait for the Broker to start the Unified Assurance services, then update the Unified Assurance package:

      source $A1BASEDIR/.assure1_bashrc
      $A1BASEDIR/bin/Package update --depot $FILESDIR
      

   6. Remove the "InstallIncomplete" entry from Assure1.conf:

      sed -i '/InstallIncomplete/c\' $A1BASEDIR/etc/Assure1.conf
      

The collection server upgrade is complete.

Repeat the process for any additional collection servers, then proceed to Task 5: Upgrading External Presentation Servers if relevant, or Task 6: Finalizing the Upgrade.

Task 5: Upgrading External Presentation Servers

Repeat the following steps on each of your external presentation servers:

1. On your existing Linux 7 external presentation server:

   1. Create the $A1BASEDIR environment variable for <UA_home>, replacing <UA_home> with the appropriate value. This is usually opt/assure1:

      export A1BASEDIR=<UA_home>
      

   2. Stop all services, the broker, and the web server:

      source $A1BASEDIR/.assure1_bashrc
      $A1BASEDIR/bin/BrokerControl --batch stopall
      systemctl stop assure1-broker
      systemctl stop assure1-web
      

   3. Verify that all processes have stopped:

      ps aux | grep assure1
      systemctl status assure1-broker
      systemctl status assure1-web
      

   4. Create a file called tar-excludes.txt that specifies what to exclude from the backup:

      cat <<EOF > tar-excludes.txt
      $A1BASEDIR/distrib/packages
      $A1BASEDIR/var/chartmuseum
      $A1BASEDIR/var/docker
      $A1BASEDIR/var/registry
      $A1BASEDIR/var/run/docker
      $A1BASEDIR/var/run/docker.pid
      $A1BASEDIR/vendor/docker
      $A1BASEDIR/vendor/helm
      $A1BASEDIR/vendor/kubernetes
      $A1BASEDIR/vendor/rke
      $A1BASEDIR/vendor/submariner
      EOF
      

   5. Back up the $A1BASEDIR directory:

      tar --exclude-from=tar-excludes.txt -czf UA-Linux7-External-Presentation-Backup.tgz $A1BASEDIR
      

   6. Copy the UA-Linux7-External-Presentation-Backup.tgz tar backup file to the new Linux 8 external presentation server. You can use any secure transfer protocol. For example, using scp:

      scp UA-Linux7-External-Presentation-Backup.tgz <user>@<hostname>:/home/<user>/
      

2. On the new Linux 8 external presentation server:

   1. Move the UA-Linux7-External-Presentation-Backup.tgz file to the <installation_file_directory>:

      mv /home/<user>/UA-Linux7-External-Presentation-Backup.tgz <installation_file_directory>
      

   2. Create the following environment variables, replacing the placeholders with the appropriate values:

      export A1BASEDIR=<UA_home>
      export FILESDIR=<installation_file_directory>
      export OL7FQDN=<External_Pres_Linux_7_FQDN>
      export OL8FQDN=<External_Pres_Linux_8_FQDN>
      export OL8WEBFQDN=<External_Pres_Linux_8_web_FQDN>
      export OL8PRIMARYPRESFQDN=<Primary_Pres_Linux_8_FQDN>
      export OL8SECONDARYPRESFQDN=<Secondary_Pres_Linux_8_FQDN>
      export PRIPRESOL7WEBFQDN=<Primary_Pres_Linux_7_web_FQDN>
      export PRIPRESOL8WEBFQDN=<Primary_Pres_Linux_8_web_FQDN>
      

   3. Extract the UA-Linux7-External-Presentation-Backup.tgz tar backup file:

      tar -xzf $FILESDIR/UA-Linux7-External-Presentation-Backup.tgz -C /
      

   4. Replace the old server FQDN with the new FQDN:

      grep -rl --exclude-dir={etc,legal,lib,logs,tmp,var} "$OL7FQDN" $A1BASEDIR/* | xargs sed -i "s/$OL7FQDN/$OL8FQDN/g"
      grep -rl --exclude-dir={ssl,priv} "$OL7FQDN" $A1BASEDIR/etc/* | xargs sed -i "s/$OL7FQDN/$OL8FQDN/g"
      

   5. Update the rules checkout paths to point to the new FQDN:

      for CHECKOUT in config core diagrams files
      do
        if [ -d $A1BASEDIR/var/checkouts/$CHECKOUT ] ; then
            cd $A1BASEDIR/var/checkouts/$CHECKOUT &&  $A1BASEDIR/vendor/svn/bin/svn --config-dir $A1BASEDIR/etc/svn relocate https://$PRIPRESOL8WEBFQDN/repos/$CHECKOUT/
        fi
      done
      

   6. Remove the old SSL certificates:

      rm -f $A1BASEDIR/etc/ssl/priv/*.pem
      rm -f $A1BASEDIR/etc/ssl/priv/*.old
      > $A1BASEDIR/etc/ssl/priv/index.txt
      > $A1BASEDIR/etc/ssl/priv/index.txt.attr
      > $A1BASEDIR/etc/ssl/priv/serial
      cd $A1BASEDIR/etc/ssl && ls | grep -xv ".*CA.*" | grep -xv priv | xargs rm -rf
      rm -rf $A1BASEDIR/var/rabbitmq/mnesia/*
      

   7. Check the contents of the $A1BASEDIR/etc/ssl directory:

      ls -l $A1BASEDIR/etc/ssl/
      

      The response should look like this:

      -rw-r--r-- 1 60 assure1 1342 Jun 17 14:31 Assure1CA.crt
      -rw-r--r-- 1 60 assure1 5626 Jun 17 14:32 BundleCA.crt
      -rw-r--r-- 1 60 assure1 2294 Jun 17 14:32 BundleCA.jks
      drwxr-xr-x 2 60 assure1   59 Jun 20 08:23 priv
      -rw-r--r-- 1 60 assure1 4284 Jun 17 14:32 SiteCA.crt
      -rw------- 1 60 assure1 1854 Jun 17 14:32 SiteCA.key
      

      Remove any additional files.

   8. Check the contents of the $A1BASEDIR/etc/ssl/priv directory:

      ls -l $A1BASEDIR/etc/ssl/priv/
      

      The response should look like this:

      -rw-r--r-- 1 60 assure1 0 Jun 20 08:23 index.txt
      -rw-r--r-- 1 60 assure1 0 Jun 20 08:23 index.txt.attr
      -rw-r--r-- 1 60 assure1 0 Jun 20 08:23 serial
      

      Confirm that index.txt, index.txt.attr and serial are empty files and remove any additional files.

   9. Extract the collection file contents from the Unified Assurance release package to <installation_file_directory>

      unzip -d $FILESDIR Oracle-Communications-Unified-Assurance-6.X.X.X.X-el8.zip
      

   10. Install the latest Unified Assurance RPM for Linux 8:

       dnf -y install $FILESDIR/Assure1-6.X.X.X.X-X.el8.x86_64.rpm
       

   11. Create an environment variable for the RPM version number. Do not include the package number or el8, and replace each X with the appropriate number:

       export RPMVERSION=6.X.X.X.X
       

       For example:

       export RPMVERSION=6.0.4.0.2
       

   12. Add the new primary and secondary Linux 8 presentation server FQDNs to the $A1BASEDIR/etc/Assure1.conf file:

       sed -i "s/\"DBHost\": \"\"/\"DBHost\": \"$OL8PRIMARYPRESFQDN\"/g" $A1BASEDIR/etc/Assure1.conf
       sed -i "s/\"DBBackupHost\": \"\"/\"DBBackupHost\": \"$OL8SECONDARYPRESFQDN\"/g" $A1BASEDIR/etc/Assure1.conf
       

   13. Generate new SSL certificates:

       $A1BASEDIR/bin/CreateSSLCertificate --Type Host -c $OL8FQDN
       $A1BASEDIR/bin/CreateSSLCertificate --Type SSH
       $A1BASEDIR/bin/CreateSSLCertificate --Type User -c assure1
       $A1BASEDIR/bin/CreateSSLCertificate --Type User -c root
       $A1BASEDIR/bin/CreateSSLCertificate --Type User -c api
       $A1BASEDIR/bin/CreateSSLCertificate --Type User -c kibana
       $A1BASEDIR/bin/CreateSSLCertificate --Type User -c repl
       $A1BASEDIR/bin/CreateSSLCertificate --Type Web -c $OL8WEBFQDN
       

   14. Copy the assure1-web.service systemd script from the /etc/systemd/system/ directory on the original Linux 7 server to the /etc/systemd/system/ directory on the Linux 8 server.

   15. Reload the systemctl daemon:

       systemctl daemon-reload
       

3. On the primary Linux 8 presentation server:

   1. Create an environment variable for the external presentation server ID:

      EXTPRESSERVERID=$($A1BASEDIR/vendor/mysql/bin/mysql --defaults-file=$A1BASEDIR/etc/my.cnf Assure1 -e "SELECT ServerID FROM Servers WHERE ServerHostFQDN = '$OL7FQDN'" | cut -f1 | grep -v 'ServerID')
      

   2. Set the correct server FQDN in the database, and reset packages to their base 6.0.0.0.0 versions:

      $A1BASEDIR/vendor/mysql/bin/mysql --defaults-file=$A1BASEDIR/etc/my.cnf Assure1 -e "UPDATE Servers SET ServerHostFQDN='$OL8FQDN', WebFQDN='$OL8WEBFQDN', Version='$RPMVERSION' WHERE ServerID = $EXTPRESSERVERID"
      $A1BASEDIR/vendor/mysql/bin/mysql --defaults-file=$A1BASEDIR/etc/my.cnf Assure1 -e "UPDATE Packages SET PackageVersion = '6.0.0.0.0.0' WHERE ServerID = $EXTPRESSERVERID"
      

4. On the Linux 8 external presentation server:

   1. Remove the old Linux 7 packages from the $A1BASEDIR/distrib/packages directory:

      rm -f $A1BASEDIR/distrib/packages/*
      

   2. Start the Broker service:

      systemctl start assure1-broker
      

   3. Verify the Broker service has started:

      systemctl status assure1-broker
      

   4. Remove the old Linux 7 SSL and crypto libraries:

      rm -f $A1BASEDIR/lib/libcrypto.s* $A1BASEDIR/lib/libssl.s*
      

   5. Wait for the Broker to start the Unified Assurance services, then update the Unified Assurance package:

      source $A1BASEDIR/.assure1_bashrc
      $A1BASEDIR/bin/Package update --depot $FILESDIR
      

   6. Start the web server:

      systemctl start assure1-web
      

   7. Remove the "InstallIncomplete" entry from Assure1.conf:

      sed -i '/InstallIncomplete/c\' $A1BASEDIR/etc/Assure1.conf
      

The external presentation server upgrade is complete.

Repeat the process for any additional external presentation servers, then proceed to Task 6: Finalizing the Upgrade.

Task 6: Finalizing the Upgrade

After upgrading all of your servers:

1. If you were using keepalived and stopped it during the upgrade, reinstall and configure it on your new Linux 8 system.

2. Copy any custom Docker images on your Linux 7 servers to your Linux 8 servers using the docker save and docker load commands:

   1. On the Linux 7 server, run docker save:

      docker save -o /<docker_backup_directory>/<docker_image_name>.tgz  <docker_image_name>:<tag>
      

      In the command:

      • <docker_backup_directory> is the directory where you want to save the backup.

      • <docker_image_name> is the name for the image.

      • <tag> is the tag to use to refer to the image.

   2. Copy <docker_image_name>.tgz to your Linux 8 server.

   3. On the Linux 8 server, run docker load:

      docker load -i /<docker_backup_directory>/<docker_image_name>.tgz
      

      See the Docker documentation for full descriptions of Docker commands.

3. Set up your microservice clusters as described in Microservice Cluster Setup in Unified Assurance Implementation Guide.

The forklift upgrade is complete.