8.11.1 Monitoring and Troubleshooting Exadata Database Server Updates
You can use the log files generated by the update utility to troubleshoot updates.
The update utility orchestrates updating the Exadata database
servers. Updating database nodes with the patchmgr tool is less
verbose because it prints only minimal information to the
screen. If additional information is required, you can view the
patchmgr logs and the dbnodeupdate.sh logs
that patchmgr copies over from other servers, if available. The
log file (dbnodeupdate.log) and the diag
file (dbnodeupdate.<runid>.diag) will
eventually exist on two locations:
- On each updated database server, in the
/var/log/cellosdirectory - Consolidated on the node running the update utility.
On the node running the update utility, if the
-log_dir flag was set to
auto, the log files will be stored
in the log/<directory based on contents of nodes in
list file> directory, relative from the
directory where the update utility is started from. For example,
if the update utility is located in
/u01/dbserver.patch, then the log
directory may be
/u01/dbserver.patch/dm01db01_dm01db02_e8f1f753.
Important files found in the log directory are:
patchmgr.logcontains the consolidated screen output from running the remote update commands on the different database servers.<hostname>_dbnodeupdate.<runid>.diagis the diag file for the specific run on a database server.<hostname>_dbnodeupdate.logcontainsdbnodeupdate.logoutput appended from/var/log/cellosfrom the remote database server.
When a prerequisite check, backup, update, or rollback fails,
error messages on screen should provide information on which
step failed on which node. Consult the log files mentioned above
if more information is required. Search the log file for the
start of a new run (search for zzz).
Check if the time matches your run. If it matches, note the
runid for further reference. Then search for
ERROR.
If an update action fails before the actual YUM update, you can retry the update after resolving the error. If the update failed half-way, it is recommended that you roll back, resolve the error, and retry.
In rare cases, patchmgr may be unable to determine the status of an update, whether the update was successful or not. In such cases, it displays a message that the update failed. However, it is possible that the update still completed successfully. To determine the actual status of the update:
- Check the image status of the (database) node.
You can do this by running the
imageinfocommand. TheImage statusline displays the status. - Check the version of the Exadata software. This
can also be determined from the
imageinfocommand.
If the image status is success, and the Exadata version is
the new expected version, then the update was successful and you
can ignore the update failed message. Then:
- Run
dbnodeupdate.sh -cmanually on the particular node to perform the completion steps of the update. - Remove the completed node from the (database) node file.
- Rerun patchmgr to perform the update on the remaining nodes.
Other things to check if the update fails include:
- The correct syntax for using patchmgr to update database nodes can be found in the patchmgr online help.
- SSH equivalence must be configured before using patchmgr.
- Download the latest
dbserver.patch.zipfrom My Oracle Support document KB444935. - Open a service request with Oracle Support Services to analyze why the patchmgr orchestration failed.
After patchmgr completes, you can also check image status and history using the imageinfo and imagehistory commands on each database server. The following example contains typical imageinfo output showing the current image status on an Exadata database server.
# imageinfo
Kernel version: 5.15.0-308.179.6.16.el8uek.x86_64 #2 SMP Thu Sep 18 11:19:34 PDT 2025 x86_64
Uptrack kernel version: 5.15.0-314.193.5.3.el8uek.x86_64 #2 SMP Mon Nov 10 00:36:36 PST 2025 x86_64
Image kernel version: 5.15.0-308.179.6.16.el8uek
Image version: 25.2.5.0.0.251214
Image activated: 2026-01-15 15:00:42 -0800
Image status: success
Exadata software version: 25.2.5.0.0.251214
Node type: COMPUTE
System partition on device: /dev/mapper/VGExaDb-LVDbSys1The following imagehistory output shows that the database server was updated to Oracle Exadata System Software release 25.2.5 (from release 25.1.6).
# imagehistory
Version : 25.1.6.0.0.250622
Exadata Live Update Version : n/a
Image activation date : 2025-07-04 00:59:39 -0700
Imaging mode : fresh
Imaging status : success
Version : 25.2.5.0.0.251214
Exadata Live Update Version : n/a
Image activation date : 2026-01-15 15:00:42 -0800
Imaging mode : patch
Imaging status : successThe next example shows imageinfo output for an update to a guest VM using Exadata Live Update. In this example, the update has been applied but there is outstanding work that is yet to be applied.
# imageinfo
Kernel version: 5.15.0-300.163.18.7.el8uek.x86_64 #2 SMP Fri Nov 15 03:14:11 PST 2024 x86_64
Uptrack kernel version: 5.15.0-312.187.5.3.el8uek.x86_64 #2 SMP Sun Sep 21 08:53:14 PDT 2025 x86_64
Image kernel version: 5.15.0-300.163.18.7.el8uek
Image version: 25.1.1.0.0.250121
Image created: 2025-01-21 23:49:50 -0800
Image activated: 2025-10-28 11:42:36 -0700
Image image type: production
Image status: success
Image label: OSS_25.1.1.0.0_LINUX.X64_250121
Exadata software version: 25.1.1.0.0.250121
Exadata Live Update Type: high (CVSS 7-10)
Exadata Live Update Version: 25.1.10.0.0.251012.1 (Live Update applied. Reboot at any time to finalize outstanding items.)
Node type: GUEST
Install type: KVM Guest with ROCE and Secure Fabric
System partition on device: /dev/mapper/VGExaDb-LVDbSys1The following example shows imagehistory output after an update using Exadata Live Update. In this example, the output shows three image history records:
-
The first record contains information about the initial (fresh) image.
-
The second record contains information about the image after a full update using Exadata Live Update. At this stage, the update has been applied but there is outstanding work that is yet to be applied.
-
The last record show the image state after application of the outstanding work associated with the Exadata Live Update.
# imagehistory
Version : 25.2.2.0.0.250919
Exadata Live Update Version : n/a
Image activation date : 2025-10-06 18:05:41 -0700
Imaging mode : fresh
Imaging status : success
Version : 25.2.3.0.0.251020
Exadata Live Update Version : 25.2.3.0.0.251020 (full) (Live Update applied. Reboot at any time to finalize outstanding items.)
Image activation date : 2025-11-04 20:18:40 -0800
Imaging mode : patch
Imaging status : success
Version : 25.2.3.0.0.251020
Exadata Live Update Version : 25.2.3.0.0.251020 (full)
Image activation date : 2025-11-04 22:43:30 -0800
Imaging mode : patch
Imaging status : success