To maintain Oracle Real Application Clusters (Oracle RAC) and Oracle Grid Infrastructure, you can use OPatchAuto.

To simplify the patching process, Oracle recommends that you use Oracle Fleet Patching and Provisioning. If you do not use Oracle Fleet Patching and Provisioning, then DBCA is a good option to use for Oracle Real Application Clusters (Oracle RAC). OPatchAuto is an additional option that you can use.

Note:

As a best practice to simplify maintenance and avoid potential issues, Oracle strongly recommends that you perform database maintenance using a new Oracle home and out-of-place patching. This is the preferred deployment choice for patching operations.

Checks and Prerequisites for Using OPatch and OPatchAuto

Download the patch, and before you perform software maintenance, review the guidelines and prerequisites for your planned deployment scenario.

Note:

Before starting to apply a maintenance Release Update or interim patch, Oracle strongly recommends that you take a backup of any Oracle home binaries, Grid home binaries, and the central Oracle Inventory (oraInventory that can be affected by the update). For further information, see:

How to Perform ORACLE_HOME Backup? (Doc ID 565017.1)

Download and Unzip the Patch

Oracle recommends that you download and use the latest released OPatch version. Download the latest version from My Oracle Support:

My Oracle Support patch 6880880

You unzip the patch as the owner of the Oracle software binaries. This is the Oracle user for Oracle Database and Oracle Real Application Clusters (Oracle RAC).

For Oracle Grid Infrastructure and Oracle RAC, Oracle recommends that you download the OPatch, OPatchAuto and the RU or Interim patch to a shared location so that they can be accessed from any node in the cluster for the patch application on each node. For each Oracle RAC database home and the Oracle Grid Infrastructure home that are being patched, as the respective home owner, extract the OPatch utility. For example:
ORACLE_HOME/OPatch

For exact instructions to install OPatch, follow the readme included with the tool download.

See Also:

For detailed OPatch documentation, including any known issues, see:

Primary Note For OPatch (Doc ID 293369.1)

Guidelines for OPatch or OPatchAuto Maintenance

Release Updates (RUs) from Automated Release Update (ARU) or My Oracle Support are cumulative. That is, the content of all previous RUs for a given software release are included in the latest RU. To increase performance by deleting inactive patches, Oracle recommends that you run OPatch with the listorderedinactivepatches and deleteinactivepatches options. For more information, see:

OPatch 12.2.0.1.37+ Introduces a New Feature to Delete Inactive Patches in the ORACLE_HOME/.patch_storage Directory (Doc ID 2942102.1)

To use OPatch or OPatchAuto to install the latest software update for your Oracle software, the software home must have the latest release software already installed. For example, to install the latest RU for Oracle Database 23ai, you must first have installed Oracle Database 23ai in the Oracle home where you want to apply the most recent RU. You can find more information about your patch requirements and options in the Download Reference for software updates:

Assistant: Download Reference for Oracle Database/GI Update, Revision, PSU, SPU(CPU), Bundle Patches, Patchsets and Base Releases (Doc ID 2118136.2)

Patches typically include Java Development Kit (JDK) fixes released since the previous RU, and will update the JDK in the Oracle home.

The UTL_URI.ESCAPE function is now compliant with RFC 3896 and will treat "#" as a reserved character. For more information, see:

UTL_URI.ESCAPE function is now compliant with RFC 3896 and will treat “#” as a reserved character (Doc ID 2981395.1)

Deployment Guidelines for Oracle RAC or Oracle Grid Infrastructure

To deploy RUs or patches manually, you can use OPatchAuto. These updates can be installed as rolling patches for Oracle Grid Infrastructure or Oracle RAC.

Caution:

Do not attempt to use OPatch to patch software in an Oracle Grid Infrastructure home (Grid home).

You no longer require a separate Oracle Database Embedded JVM (OJVM) RU bundle patch. OJVM fixes can now be applied as rolling updates to Oracle RAC, and are included in the Database RU bundles. For more information, See:

Primary Note for Database Proactive Patch Program (Doc ID 888.1)

Validating the Oracle Inventory on Oracle Grid Infrastructure and Oracle RAC Cluster Members

Before beginning patch application, check the consistency of inventory information for the Grid home and each Oracle home that you plan to patch. Run this command as the respective Oracle home owner to check the consistency, where <ORACLE_HOME> is the Oracle home or Grid home:

$ <ORACLE_HOME>/OPatch/opatch lsinventory -detail -oh <ORACLE_HOME>

If this command succeeds, then it lists the Oracle components that are installed in the home. Save the output so that you have the status of the inventory before you start the patch application.

If this command fails, then contact Oracle Support for assistance.

Deployment Guidelines for Oracle Data Guard

To deploy RUs and patches in Oracle homes deployed with Oracle Data Guard, you can use OPatchAuto. For information about how to apply updates in homes configured with Oracle Data Guard, and how to reduce downtime, see:

Oracle Patch Assurance - Data Guard Standby-First Patch Apply (Doc ID 1265700.1)

For the latest Update with Security Fixes that should be used on Client-Only installations, refer to the Critical Patch Update (CPU) Program Patch Availability Document (PAD) section on Oracle Database for the cycle in which you are interested.

Check System Requirements

Before you start OPatch, check to ensure that you have required environment dependencies.

Ensure that the $PATH definition has the following executables:

  • make
  • ar
  • ld
  • nm

The location of these executables depends on your operating system. On many operating systems, they are located in /usr/ccs/bin. Ensure that your path environment includes the location of the executables. For example:

export PATH=$PATH:/usr/ccs/bin

Detect and Resolve Interim Patch Conflicts in the Grid Home

You must detect and resolve any interim (one-off) patch conflicts before running OPatchAuto in the Oracle Real Application Clusters database homes, and in the Oracle Grid Infrastructure Grid home.

Oracle recommends that you use the "Patch Recommendations and Patch Plans" features on the Patches & Updates tab in the My Oracle Support search results page for a patch search. These features are the fastest and easiest way to determine whether you have interim patches in the Oracle home that conflict with a Release Update (RU), and to obtain the conflict resolution patches that you will require. The "Patch Recommendations and Patch Plans" features work in conjunction with the My Oracle Support Configuration Manager. You can find recorded training sessions for these features on My Oracle Support:

My Oracle Support How-to Video Training Series (Doc ID 603505.2)

If you choose not to use My Oracle Support Patch Plans, then My Oracle Support Conflict Checker tool enables you to upload an OPatch inventory and check the patches that you want to apply to your environment for conflicts.

For details about using this tool, see:

How to Use the My Oracle Support Conflict Checker Tool for Patches Installed with OPatch. (Doc ID 1091294.1)

The procedure to use the tool is as follows:

  1. Download the patch from My Oracle Support. Patches are zip archive file format files in the form <patchnumber>_<version>_<platform>.zip, where <patchnumber> is the eight-digit number of the patch, <version> is the release of the Oracle software for which the patch is provided, and <platform> is the operating system platform for which the patch can be applied. For example: p12345678_23.0.0.0.0_Linux-x86-64.

  2. Unzip the patch, and use the OPatch command opatch prereq CheckConflictAgainstOHWithDetail to find if any currently installed interim patches conflict with the patch being installed. For example, if the patch is p12345678_23.0.0.0.0_Linux-x86-64:

    unzip p12345678_23.0.0.0.0_Linux-x86-64.zip
    cd p12345678_23.0.0.0.0_Linux-x86-64
    opatch prereq CheckConflictAgainstOHWithDetail -ph ./
    
  3. Review the report. If there are conflicts, then the report indicates the patches that conflict and the patches that are a superset.

  4. Resolve the patch conflicts for each conflicting patch. Determine if a conflict resolution patch is already available, and if you need to request a new conflict resolution patch or if the conflict may be ignored. If you require additional conflict resolution patches, then you must wait until they become available before proceeding further.

    To resolve patch conflicts, use the following My Oracle Support document:

    Database Patch Conflict Resolution (Doc ID 1321267.1)

After you resolve patch conflicts, or receive any interim conflict resolution patches, you can proceed with updating your Oracle software.

Example 1-1 Manually Running Conflict Check for Oracle Grid Infrastructure

To run conflict checks manually, you have to run opatch with the prereq flag for each of the Grid home component installed interim patches. For example, suppose the patch name is p12345678_23.0.0.0.0_Linux-x86-64 and the unzipped patch location is /usr/home/oracle/OPatch. Checks for the Grid home can be similar to the following:

% $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /usr/home/oracle/patch/p12345678_23.0.0.0.0_Linux-x86-64/% ARU_Database_SubPatchTrackingBug %

% $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /usr/home/oracle/patch/p12345678_23.0.0.0.0_Linux-x86-64/% ARU_OCW_SubPatchTrackingBug %

% $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /usr/home/oracle/patch/p12345678_23.0.0.0.0_Linux-x86-64/% ARU_ACFS_SubPatchTrackingBug %

% $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /usr/home/oracle/patch/p12345678_23.0.0.0.0_Linux-x86-64/% ARU_RHP_SubPatchTrackingBug %

% $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /usr/home/oracle/patch/p12345678_23.0.0.0.0_Linux-x86-64/% ARU_TOMCAT_SubPatchTrackingBug %

% $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /usr/home/oracle/patch/p12345678_23.0.0.0.0_Linux-x86-64/% ARU_DBWLM_SubPatchTrackingBug %

Note:

  • For HP-UX Itanium and Linux on IBM System z platforms, the last two checks in the previous example do not need to be done.

  • For the Oracle or Grid home, as the software home user:

    % $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /usr/home/oracle/patch/p12345678_23.0.0.0.0_Linux-x86-64/% ARU_Database_SubPatchTrackingBug %
    % $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /usr/home/oracle/patch/p12345678_23.0.0.0.0_Linux-x86-64/% ARU_OCW_SubPatchTrackingBug %
    

The report will indicate the interim patches that conflict with the patch p12345678_23.0.0.0.0_Linux-x86-64 and the interim patches for which patch p12345678_23.0.0.0.0_Linux-x86-64 is a superset.

Run OPatch System Space Check

Check if enough free space is available on the ORACLE_HOME file system for the patches that you want to apply.

The UPGRADE CATALOG command must be entered twice to confirm the upgrade.

Enter the following commands:

  • For the Oracle Grid Infrastructure home (Grid home), as the Grid user:

    1. Create file /tmp/patch_list_gihome.txt with the following content (in this example, the patch location is /usr/home/oracle/patch/OPatch):

      % cat /tmp/patch_list_gihome.txt
      
      /usr/home/oracle/patch/OPatch/p12345678_23.0.0.0.0_Linux-x86-64/% ARU_Database_SubPatchTrackingBug %
      /usr/home/oracle/patch/OPatch/p12345678_23.0.0.0.0_Linux-x86-64/% ARU_OCW_SubPatchTrackingBug %
      /usr/home/oracle/patch/OPatch/p12345678_23.0.0.0.0_Linux-x86-64/% ARU_ACFS_SubPatchTrackingBug %
      /usr/home/oracle/patch/OPatch/p12345678_23.0.0.0.0_Linux-x86-64/% ARU_RHP_SubPatchTrackingBug %
      /usr/home/oracle/patch/OPatch/p12345678_23.0.0.0.0_Linux-x86-64/% ARU_TOMCAT_SubPatchTrackingBug %
      /usr/home/oracle/patch/OPatch/p12345678_23.0.0.0.0_Linux-x86-64/% ARU_DBWLM_SubPatchTrackingBug %
      

      Note:

      For HP-UX Itanium and Linux on IBM System z platforms, the last two rows in the previous example should not be added to the patch_list_gihome.txt file.

    2. Run the OPatch command to check if enough free space is available in the Grid home:

      % $ORACLE_HOME/OPatch/opatch prereq CheckSystemSpace -phBaseFile /tmp/patch_list_gihome.txt
      
  • In the Oracle software home, as the software home user:

    1. Create file /tmp/patch_list_dbhome.txt with the patch content in the location where you have unzipped the patch. For example, if the patch is p12345678_23.0.0.0.0_Linux-x86-64::

      % cat /tmp/patch_list_dbhome.txt
      <patch-location>/p12345678_23.0.0.0.0_Linux-x86-64/% ARU_Database_SubPatchTrackingBug %
      <Patch-location>/p12345678_23.0.0.0.0_Linux-x86-64/% ARU_OCW_SubPatchTrackingBug %
      
    2. Run the OPatch command to check if enough free space is available in the Oracle home:

      % $ORACLE_HOME/OPatch/opatch prereq CheckSystemSpace -phBaseFile /tmp/patch_list_dbhome.txt
      

The command output reports pass and fail messages in response to the system space availability:

  • If OPatch reports Prereq "checkSystemSpace" failed, then the required amount of space is not available, so you must free up space on the system.

  • If OPatch reports Prereq "checkSystemSpace" passed, then no action is needed. Proceed with patch installation.

Patch Installation Scenarios Using OPatchAuto

Use the instruction for your scenario. The patch instructions will differ based on the Oracle Grid infrastructure and Oracle RAC database homes.

For other configurations listed below, see:

Supplemental Readme - Grid Infrastructure Release Update 12.2.0.1.x / 18c /19c (Doc ID 2246888.1):

  • Grid home is not shared, the Oracle home is not shared, Oracle Advanced Cluster File System (ACFS) can be used.
  • Patching Oracle RAC database homes.
  • Patching Grid home alone.
  • Patching Grid home together with Oracle RAC One Node and clusterware-managed single-instance databases.
  • Patching Oracle Restart home.
  • Patching a software only Grid home installation or before the Grid home is configured.

Case 1: Unshared Oracle RAC and Grid home without Oracle ACFS

In this scenario you patch Oracle RAC, where the Grid home and the Oracle homes are not shared and Oracle ACFS file system is not configured.

Oracle recommends that you use the latest OPatchAutoversion for your Oracle software release.

If the system you are patching is configured with an Oracle Data Guard Standby database, then you must install this patch on both the primary patch and on the physical standby database, as described in How do you apply a Patchset,PSU or CPU in a Data Guard Physical Standby configuration (Doc ID 278641.1).

If the system you are patching is configured with Oracle Real Application Clusters, then you must install this patch on both the primary patch and on the physical standby database, as described in Rolling Patch - OPatch Support for RAC (Doc ID 244241.1).

Follow these steps:

  1. As the root user, use the following command syntax to run the patch on each node of the cluster, where: <GI_HOME> is the Oracle Grid Infrastructure home (Grid home); <UNZIPPED_PATCH_LOCATION> where the patch is unzipped; and <ARU_TrackingBug> is the patch:

    # <GI_HOME>/OPatch/opatchauto apply <UNZIPPED_PATCH_LOCATION>/<ARU_TrackingBug> -logLevel FINEST

    For example:

    /u01/app/grid/OPatch/opatchauto apply /usr/home/oracle/patch/p12345678_23.0.0.0.0_Linux-x86-64/ -logLevel FINEST

After installing the patch, complete these steps:

  1. Load modified SQL files into the database, as explained in "Load Modified SQL Files into the Database" .
  2. Upgrade Oracle Recovery Manager Catalog, as explained in "Upgrade Oracle Recovery Manager Catalog".
  3. Check bug fixes that may change an existing optimizer execution plan, as explained in Managing "installed but disabled" bug fixes in Database Release Updates using DBMS_OPTIM_BUNDLE (Doc ID 2147007.1)

If there are errors, refer to "Known Issues".

Case 2: Shared Oracle RAC Home Unshared Grid Home with Oracle ACFS

In this scenario you patch Oracle RAC, where the Grid home is not shared, the Oracle RAC home is shared, and Oracle ACFS can be used.

Oracle recommends that you use the latest OPatchAutoversion for your Oracle software release.

If the system you are patching is configured with an Oracle Data Guard Standby database, then you must install this patch on both the primary patch and on the physical standby database, as described in:

How do you apply a Patchset, PSU or CPU in a Data Guard Physical Standby configuration (Doc ID 278641.1).

If the system you are patching is configured with Oracle Real Application Clusters, then you must install this patch on both the primary patch and on the physical standby database, as described in:

Rolling Patch - OPatch Support for RAC (Doc ID 244241.1).

Follow these steps:

  1. From the Oracle RAC home, stop the Oracle RAC databases running on all nodes using the following command, where <RAC_HOME> is the Oracle RAC home and <db_unique_name> is the database unique name of the Oracle RAC database:

    $ <RAC_HOME>/bin/srvctl stop database -d <db-unique-name>
    
  2. On the first node, unmount the Oracle Advanced Cluster File System (ACFS). For detailed instructions for how to complete this task, see:

    How to Mount or Unmount ACFS File System While Applying GI Patches? (Doc ID 1494652.1).

  3. On the first node, apply the patch to the Grid home using the opatchauto command. As the root user, run the following command:

    # <GI_HOME>/OPatch/opatchauto apply <UNZIPPED_PATCH_LOCATION>/% ARU_TrackingBug % -oh <GI_HOME> -logLevel FINEST
    
  4. If the message, "A system reboot is recommended before using ACFS" is shown, then you must restart the system before continuing. Failure to do so will result in running with an unpatched ACFS\ADVM\OKS driver.

  5. On the first node, remount Oracle ACFS file systems. For instructions, see:

    How to Mount or Unmount ACFS File System While Applying GI Patches? (Doc ID 1494652.1).

  6. On the first node, apply the patch to the Oracle home using the opatchauto command. Since the Oracle home is shared, this operation will patch the Oracle home across the cluster. Note that a Universal Storage Manager patch (USM only patch) cannot be applied to an Oracle home. As the root user, run the following command:

    # <RAC_HOME>/OPatch/opatchauto apply <UNZIPPED_PATCH_LOCATION>/% ARU_TrackingBug % -oh <ORACLE_HOME> -logLevel FINEST -nonrolling
  7. On the first node only, restart the Oracle instance that you previously stopped in Step 1, using srvctl start instance:

    $ <ORACLE_HOME>/bin/srvctl start instance –d <db-unique-name> -n <nodename>
    
  8. On the second (next) node, unmount the Oracle ACFS file systems. For instructions, see:

    How to Mount or Unmount ACFS File System While Applying GI Patches? (Doc ID 1494652.1).

  9. On the second node, apply the patch to the Grid home using the opatchauto command. As the root user, run the following command:

    # <GI_HOME>/OPatch/opatchauto apply <UNZIPPED_PATCH_LOCATION>/% ARU_TrackingBug % -oh <GI_HOME> -logLevel FINEST
  10. If the message "A system reboot is recommended before using ACFS" is shown, then you must restart the system before you continue. Failing to restart the system will result in running with an unpatched ACFS\ADVM\OKS driver.

  11. On the second node, running the opatchauto command in Step 9 will restart the stack.

  12. On the second node, remount Oracle ACFS file systems. For instructions, see:

    How to Mount or Unmount ACFS File System While Applying GI Patches? (Doc ID 1494652.1).

  13. On the second node only, restart the Oracle instance, which you have previously stopped in Step 1. As the Oracle home owner run:

    $ <ORACLE_HOME>/bin/srvctl start instance –d <db-unique-name> -n <nodename>
    
  14. Repeat steps 8 through 13 for all remaining nodes of the cluster.

After installing the patches needed, complete these steps:

  1. Load modified SQL files into the database, as explained in "Load Modified SQL Files into the Database" .
  2. Upgrade Oracle Recovery Manager Catalog, as explained in "Upgrade Oracle Recovery Manager Catalog" .
  3. Check bug fixes that may change an existing optimizer execution plan, as explained in Managing "installed but disabled" bug fixes in Database Release Updates using DBMS_OPTIM_BUNDLE (Doc ID 2147007.1).

If there are errors, refer to "Known Issues"

Load Modified SQL Files into the Database

When you use OPatch to perform database maintenance, you must run the datapatch tool manually so that the tool can load modified SQL files into the database.

Datapatch is the patching tool that completes the post-patch SQL actions for RDBMS patches, such as running scripts to load modified SQL files into the database registry. Datapatch identifies the postinstallation steps required after the database is updated, and applies and removes or rolls back SQL changes in the database as required. Unlike a maintenance operation with OPatchAuto, which calls Datapatch automatically to complete post-patch actions after the new binary patch is installed and the database is restarted, you must run Datapatch manually after running OPatch.

For more information about Datapatch, see:

  • Datapatch: Database 12c or later Post Patch SQL Automation (Doc ID 1585822.1)
  • Datapatch User Guide (Doc ID 2680521.1)
  1. For your databases running on the same Oracle home in the multitenant environment being patched, run the datapatch tool. In this example, we run datapatch with the -sanity_checks option:

    % sqlplus /nolog
    SQL> Connect / as sysdba
    SQL> startup
    SQL> alter pluggable database all open; 
    SQL> quit
    % cd $ORACLE_HOME/OPatch
    % ./datapatch -sanity_checks
    % ./datapatch -verbose

    Oracle highly recommends that you run datapatch with the -sanity_checks option. When started with this option, datapatch runs a series of environment and database checks to validate if conditions are optimal for patching. Results are shown on screen with severity and recommendations for potential actions you can take. For more information about this feature, see:

    Datapatch User Guide (Doc ID 2680521.1)

    Oracle also recommends that you run datapatch on all pluggable databases (PDBs) in the multitenant environment at the same time. However, you can choose to run datapatch on individual pluggable databases. If you choose to do this, then datapatch is run only on the multitenant container database (CDB) and opened pluggable databases (PDBs). You then are required to update the PDBs that are not updated at this time by using the alter pluggable database statement, and running the datapatch tool again at a later time to complete post-patch SQL actions on those PDBs. See:

    Multitenant Unplug/Plug Best Practices (Doc ID 1935365.1)

    The datapatch utility runs apply scripts to load the modified SQL files into the database. An entry is added to the dba_registry_sqlpatch view, recording the patch application.

  2. In the dba_registry_sqlpatch view, verify that the Status for the APPLY is SUCCESS. For any other status, see:

    Troubleshooting Assistant:12c Datapatch Issues (Doc ID 2335899.2)

  3. After datapatch completes, check its output to see if it has reported any errors. The output of datapatch contains log file locations for the patches installed. You can find all the relevant log files of that particular datapatch process in the same directory as sqlpatch_invocation.log. Check the initial lines of the output for "Log file for this invocation:"

Upgrade Oracle Recovery Manager Catalog

If you are using the Oracle Recovery Manager, then the catalog needs to be upgraded.

The UPGRADE CATALOG command must be entered twice to confirm the upgrade.

Enter the following commands:

$ rman catalog username/password@alias
RMAN> UPGRADE CATALOG;
RMAN> UPGRADE CATALOG;
RMAN> EXIT;

Known Issues

Learn how to find if an issue you have encountered while running OPatch is a known issue.

For information about known OPatch issues, see:

Primary Note For OPatch (Doc ID 293369.1)

For issues documented after the release of the Release Update (RU) or patch version that you have installed, review the README for the Release Update (RU) or patch version that you have installed.

Documentation Accessibility

Access to Oracle Support