Mellanox ConnectX adapter not detected after boot
On systems using the Mellanox ConnectX adapters, the driver does not load the InfiniBand and RMDA modules at boot time, resulting in a failure to detect the adapter when using RDMA and InfiniBand related tools, such as ibstat. Errors typically display as follows:
ibpanic: [26013] main: stat of IB device 'mthca0' failed: No such file or directory
This issue results because although the
mlx4_core
and mlx5_core
drivers are included in the initramfs to facilitate PXE boot, the
InfiniBand and RDMA modules are not. If you need the driver for
PXE boot, you can reload the driver manually after boot to trigger
the RDMA hotplug sequence, for example:
# modprobe mlx5_core
If you do not require the mlx4_core
or
mlx5_core
driver for PXE boot, you can remove
these drivers from the initramfs and they are loaded after boot,
as required, and the RDMA hotplug sequence is triggered normally.
To remove the drivers from the initramfs, create
/etc/dracut.conf.d/10-mlx_dracut-denylist.conf
to include the line:
omit_drivers+=" mlx4_* mlx5_* mlxfw "
After you have done this, rebuild the initramfs by running:
# dracut -f
The changes take effect after reboot.
(Bug ID 31353413)