5 Troubleshooting Oracle Linux Upgrades
This chapter provides troubleshooting information and describes known issues that might affect the upgrade process.
Tools for Troubleshooting
Use the following options to generate more output when you are generating the preupgrade report or performing the actual upgrade:
-
--verbose
displays warnings, error messages, and other critical information. -
--debug
adds debug information in addition to the same output as the--verbose
option.
You can use the following resources and tools for obtaining troubleshooting information:
-
/var/log/leapp/leapp-report.txt
-
/var/log/leapp/leapp-upgrade.log
-
/var/log/leapp/dnf-debugdata/
: a directory for debug information. Note that this directory is created only if you use the--debug
option when issuing either the preupgrade or the upgrade command. -
journalctl command
Known Issues
The following are known issues that you might encounter when upgrading an Oracle Linux 7 system to Oracle Linux 8.
Upgrade Issues
-
Optional Resilient Storage group isn't supported with Leapp
Leapp doesn't support the optional Resilient Storage group. The presence in the system of certain packages from that group might prevent a leapp upgrade to complete.
For example, the Resilient Storage group includes the
lvm2-cluster
package. In an Oracle Linux 7 installation, this package can be optionally added as part of either the Infrastructure Server and File profile or the Print Server profile. However, the package is an inhibitor and would cause the Leapp upgrade to fail.BugID 33573562
-
Leapp might report missing packages that are marked for installation
The
/var/log/leapp/leapp-preupgrade.log
or/var/log/leapp/leapp-upgrade.log
files might report a warning similar to the following:Warning: Packages marked by Leapp for install not found in repositories metadata: rpcgen python3-pyxattr libnsl2-devel rpcsvc-proto-devel
These packages are in the
Oracle Linux8 Codeready Builder
repository, which is a developer repository and is disabled by default.If the system requires these packages, then during the preupgrade or the upgrade phase, add the
--enablerepo ol8_codeready_builder
option to the appropriate Leapp command, for example:sudo leapp upgrade --oraclelinux --enablerepo ol8_codeready_builder
Repositories that have been enabled during the Leapp upgrade remain enabled on the Oracle Linux 8 system after the upgrade completes.
Alternatively, after completing the upgrade, you can manually install the packages required for your installation by using the dnf command.
Bug ID 32827043
-
MySQL-related
*.el7
packages might remain after an upgradeIn an Oracle Cloud Infrastructure instance, existing MySQL packages might remain after the upgrade is completed. You can verify their existence with the following command:
rpm -qa | grep el7 | grep -v leapp | grep -v kernel
mysql-community-client-plugins-8.0.25-1.el7.x86_64
To ensure that these packages are correctly updated, enable the
ol8_MySQL80
repository also when you run the upgrade.sudo leapp upgrade --oci --enablerepo ol8_mysql80
-
Some
el7
packages might not be upgradedThe same rpm -qa command syntax in the previous item that detects MySQL-related
*.el7
packages might also list more*el7
packages on the system that weren't upgraded. Packages might not be upgraded if they were installed from repositories that aren't supported by Leapp, such as developer repositories. For such packages, do the following:-
Go to https://yum.oracle.com and check the Oracle Linux 8 repositories that would serve the packages you need.
-
After the upgrade is completed, manually install the packages from those Oracle Linux 8 repositories.
-
After all the necessary packages have been installed, remove the residual
el7
packages from the system.
Bug ID 32878386
-
-
(aarch64) Upgrade log might report errors related to the vmd module
After completing an upgrade on aarch64 systems, the Leapp upgrade log might report the following message:
dracut-install: Failed to find module 'vmd'
The VMD module doesn't apply to the Arm architecture and therefore, the error message can be safely ignored.
Bug ID 34172552
Security and Authentication Issues
-
TCP Wrappers not used in Oracle Linux 8
TCP Wrappers aren't available in Oracle Linux 8. If you're using TCP Wrappers to control network traffic in the Oracle Linux 7 system through
/etc/hosts.deny
and/etc/hosts.allow
files, then after the upgrade you must create applicable firewall rules to implement the same control in Oracle Linux 8.For more information about implementing firewall rules, see Oracle Linux 8: Configuring the Firewall.
-
Oracle Linux 7 authentication configuration not converted after upgrade
Leapp does not convert Oracle Linux 7 authentication configuration that uses the deprecated authconfig utility to the corresponding Oracle Linux 8 authentication that uses the authselect command. After the upgrade, you need to reconfigure authentication appropriately in Oracle Linux 8.
For more information about using the authselect utility, see Oracle Linux 8: Setting Up System Users and Authentication.
-
Upgrade blocked if system has configured
pam_krb5
andpam_pkcs11
PAM modulesThe deprecated
pam_krb5
andpam_pkcs11
PAM modules in Oracle Linux 7 are removed during the upgrade process. Before performing the upgrade, you must first reconfigure the system's PAM configuration to disable these modules. Otherwise, the system becomes locked.For instructions to disable these modules, see Oracle Linux 7: Setting Up System Accounts and Authentication. See also the discussion about these modules in Providing Information to the Leapp Answerfile.
System Management Issues
-
Ksplice Uptrack software displays error messages
During the upgrade, the Oracle Ksplice Uptrack software might report errors similar to the following:
[ 256.033527] upgrade[390]: Upgrading : uptrack-1.2.74-0.el8.noarch 577/1453 [ 256.037151] upgrade[390]: Running scriptlet: uptrack-1.2.74-0.el8.noarch 577/1453 ... [ 256.045914] upgrade[390]: Traceback (most recent call last): [ 256.049230] upgrade[390]: File "/usr/lib/uptrack/access-key-from-uln", line 9, in <module> [ 256.051376] upgrade[390]: from up2date_client import up2dateAuth, rpcServer [ 256.056490] upgrade[390]: File "/usr/share/rhn/up2date_client/up2dateAuth.py", line 74 [ 256.059251] upgrade[390]: os.chmod(path, 0600) [ 256.060997] upgrade[390]: [ 256.062842] upgrade[390]: SyntaxError: invalid token
The report is a known but harmless issue, which you can ignore. After the upgrade is completed, Ksplice continues to operate normally.
File Systems and Storage Issues
-
Systems with Btrfs in a RAID configuration can't be upgraded
A system that uses the Btrfs file system in a RAID configuration can't be upgraded. In the
/var/log/leapp/leapp-report.txt
that's generated by the preupgrade command, this configuration is flagged as an inhibitor and no remedy is provided. If you upgrade the system and that configuration is detected, the upgrade process halts. -
Upgrade blocked if
winbind
andwins
Samba modules are usedIf
winbind
andwins
Samba modules are used in the/etc/nsswitch.conf
, the upgrade is blocked. As a workaround, remove these modules from the file first, then perform the upgrade. After the upgrade is complete, restore these module entries to the file.For more information about configuring these modules, see Oracle Linux 8: Managing Shared File Systems.
-
Hosts with network mounted file systems can't be upgraded
Leapp doesn't support upgrading systems with mounted file systems on network storage, NFS, or iSCSI. As a workaround, unmount the file systems and comment out their entries from
/etc/fstab
. After the upgrade is completed, you can restore the entries and remount the file systems.
Networking Issues
-
Possible upgrade error if system has several NICs with the same prefix as NIC that's used by kernel
The in-place upgrade process might cause an error if the system to be upgraded has more than one NIC that shares the same prefix as the NIC that's used by the kernel, for example
eth
. After the upgrade, the system's network connectivity is lost.For more information, see About Network Interface Names in Oracle Linux 8: Setting Up Networking.
-
NetworkManager might not start after the upgrade completes
After the upgrade, the system's
NetworkManager
might not start because of the failure of its name resolution service. The failure can be verified by checking the status of the service.systemctl status systemd-resolved.service
● systemd-resolved.service - Network Name Resolution Loaded: loaded (/usr/lib/systemd/system/systemd-resolved.service; disabled; > Active: inactive (dead) Docs: man:systemd-resolved.service(8) https://www.freedesktop.org/wiki/Software/systemd/resolved
The
/var/log/messages
file also reports the following error:dbus-daemon[742]: [system] Activation via systemd failed for unit 'dbus-org.freedesktop.resolve1.service': Unit dbus-org.freedesktop.resolve1.service not found.
To resolve this issue, choose one of the following workarounds:
-
Configure
NetworkManager
to not usesystemd-resolved.service
.Add the following entries to the
/etc/NetworkManager/conf.d/no-systemd-resolved.conf
file:[main] systemd-resolved=false
-
Enable the
systemd-resolved.service
as follows:systemctl enable systemd-resolved.service
Created symlink /etc/systemd/system/dbus-org.freedesktop.resolve1.service → /usr/lib/systemd/system/systemd-resolved.service. Created symlink /etc/systemd/system/multi-user.target.wants/systemd-resolved.service → /usr/lib/systemd/system/systemd-resolved.service.
systemctl start systemd-resolved.service
You can also adopt other methods that are more consistent with the network name resolution model that you're using for the specific setup. For useful information, see About Network Interface Names in Oracle Linux 8: Setting Up Networking.
-
Virtualization and Containers Issues
-
Docker removed during upgrade process
Docker is removed as part of the upgrade process. If you intend to run containers on the upgraded system, be prepared to migrate the container infrastructure to Podman.
For more information, see Oracle Linux: Podman User's Guide.
-
KVM virtual machine snapshots might not be listed after an upgrade
After an upgrade, the
libvirtd
service might report snapshot-related error messages similar to the following:libvirtd[53328]: internal error: Failed to parse snapshot XML from file '/var/lib/libvirt/qemu/snapshot/path-to-previous-snapshot-file'
Furthermore, listing available snapshots from before the upgrade generates an empty list.
sudo virsh snapshot-list previous-snapshot-file
Name Creation Time State -------------------------------
As a workaround, reboot the system. At the end of the boot process, the snapshots are listed and available again.
-
libvirtd service might fail to restart in nested virtualization configurations
In nested virtualization setups, the
libvrtd
service might not restart in the nested KVM host after the upgrade.As a workaround, reboot the nested KVM host.
-
Some KVM virtual machines might not start after the upgrade
If the KVM host is running the RHCK kernel, virtual machines, or guests that existed before the upgrade might not start after the Leapp upgrade completes. When start these guests, an error message similar to the following might be displayed:
sudo virsh start domain-name
error: Failed to start domain domain-name error: the CPU is incompatible with host CPU: Host CPU does not provide required features: hle, rtm
This error is reported because the RHCK kernel disables Intel Transactional Synchronization Extensions (TSX) by default and can't provide the Hardware Lock Elision (HLE) and Restricted Transactional Memory (RTM) features that the guest is expecting.
As a workaround, perform the following steps:
-
Edit
/etc/default/grub
by addingtsx=on
to the GRUB_CMDLINE_LINUX directive, as shown in the following example in bold:... GRUB_CMDLINE_LINUX="LANG=en_US.UTF-8 … tsx=on"
-
Regenerate the
grub.cfg
file.sudo grub2-mkconfig -o /boot/grub2/grub.cfg
-
Reboot the KVM host.
-
Development Tools Issues
-
Python 2 replaced by Python 3
In Oracle Linux 8, the python command isn't available by default. Python 3 is the primary Python version and isn't backward compatible with Python 2. Also, Python 2 is available in Oracle Linux 8 but only with limited support. If you have Python packages or scripts on the system to be upgraded, migrate these to Python 3 as soon as possible.
For more information, see Oracle Linux 8: Installing and Managing Python.
Hardware Related Issues
-
Systems with unrecognized hardware can't be upgraded
Support for certain hardware, such as the
e1000
driver, has been removed from RHCK 8. The upgrade can't proceed on platforms that have such hardware installed. Even though UEK might continue to support the hardware, the upgrade procedure is still inhibited if the hardware is detected on the system.
Leapp Overlay Size Issues
-
Upgrading might require increased overlay size
Upgrading Oracle Linux 7 systems with a huge number of packages to Oracle Linux 8 might fail because of insufficient space in the Leapp overlay file systems that are used during the upgrade. You might see the following error message:
Error: Transaction test error: installing package package-name needs 4MB on the / filesystem
As a workaround, increase the
LEAPP_OVL_SIZE
variable. The default size is 4096. The actual size you would need might be larger depending on the specific setup. Use the following command:sudo export LEAPP_OVL_SIZE=new-size
Duplicate Preupgrade Inhibitor Text Issue
-
When running the preupgrade, dupliate inhibitor text might appear
When running the preupgrade command, sometimes more than one instance of the following inhibitor problem might appear:
=========================================== REPORT OVERVIEW =========================================== Upgrade has been inhibited due to the following problems: 1. Missing required answers in the answer file 2. Missing required answers in the answer file 3. Missing required answers in the answer file ... Before continuing consult the full report: A report has been generated at /var/log/leapp/leapp-report.json A report has been generated at /var/log/leapp/leapp-report.txt =========================================== END OF REPORT OVERVIEW ===========================================
These relate to different problems that require you to update the Answerfile. More information about each problem can be found in the
/var/log/leapp/leapp-report.txt
file. For more information about updating the Answerfile, see Providing Information to the Leapp Answerfile.