2.11.3 Recovering Exadata X10M Database Servers with Customized Partitions

This procedure describes how to recover an Oracle Exadata X10M database server with RoCE Network Fabric from a snapshot-based backup when using customized partitions.

  1. Prepare an NFS server to host the backup archive file (mybackup.tar.bz2).

    The NFS server must be accessible by IP address.

    For example, on an NFS server with the IP address nfs_ip, where the directory /export is exported as an NFS mount, put the backup file (mybackup.tar.bz2) in the /export directory.

  2. Restart the recovery target system using the diagnostics.iso file.
    See Booting a Server using the Diagnostic ISO File in Oracle Exadata System Software User's Guide.
  3. Log in to the diagnostics shell as the root user.
    When prompted, enter the diagnostics shell.

    For example:

    Choose from following by typing letter in '()':
(e)nter interactive diagnostics shell. Must use credentials 
from Oracle support to login (reboot or power cycle to exit
the shell),
(r)estore system from NFS backup archive, 
Type e to enter the diagnostics shell and log in as the root user.
    If prompted, log in to the system as the root user. If you are prompted for the root user password and do not have it, then contact Oracle Support Services.
  4. If it is mounted, unmount /mnt/cell
    # umount /mnt/cell
  5. Confirm the md devices on the server.

    Confirm that the server contains the devices listed in the following example. Do not proceed and contact Oracle Support if your server differs substantially.

    # ls -al /dev/md*
brw-rw---- 1 root disk   9, 24 Dec 19 07:57 /dev/md24
brw-rw---- 1 root disk 259,  3 Dec 19 07:57 /dev/md24p1
brw-rw---- 1 root disk 259,  4 Dec 19 07:57 /dev/md24p2
brw-rw---- 1 root disk   9, 25 Dec 19 07:57 /dev/md25

/dev/md:
total 0
drwxr-xr-x  2 root root  120 Dec 19 07:57 .
drwxr-xr-x 19 root root 3420 Dec 19 08:17 ..
lrwxrwxrwx  1 root root    7 Dec 19 07:57 24 -> ../md24
lrwxrwxrwx  1 root root    9 Dec 19 07:57 24p1 -> ../md24p1
lrwxrwxrwx  1 root root    9 Dec 19 07:57 24p2 -> ../md24p2
lrwxrwxrwx  1 root root    7 Dec 19 07:57 25 -> ../md25
  6. Create the boot partition.
    1. Start an interactive session using the partd command.
      # parted /dev/md24
    2. Assign a disk label.
      (parted) mklabel gpt
    3. Set the unit size as sector.
      (parted) unit s
    4. Check the partition table by displaying the existing partitions.
      (parted) print
    5. Remove the partitions listed in the previous step.
      (parted) rm part#
    6. Create a new first partition.
      (parted) mkpart primary 64s 15114206s
    7. Specify this is a bootable partition.
      (parted) set 1 boot on
  7. Create the second primary (boot) partition.
    1. Create a second primary partition as a UEFI boot partition with fat32.
      (parted) mkpart primary fat32 15114207s 15638494s 
(parted) set 2 boot on
    2. Write the information to disk, then quit.
      (parted) quit
  8. Create the physical volume and volume group.
    # lvm pvcreate /dev/md25
# lvm vgcreate VGExaDb /dev/md25

    If the physical volume or volume group already exists, then remove and then re-create them as follows:.

    # lvm vgremove VGExaDb
# lvm pvremove /dev/md25
# lvm pvcreate /dev/md25
# lvm vgcreate VGExaDb /dev/md25
  9. Re-create the customized LVM partitions, then create and mount the file systems.

    Note:

    Use the following information and examples as guidance for this step. You must make the necessary adjustments for customized LVM partitions and file systems. For example, you may need to adjust the names and sizes of various partitions to match your previous customizations, or you may need to create additional custom partitions.
    1. Create the logical volumes.

      For example, the following commands re-create the logical volumes that exist by default on Oracle Exadata X10M systems: 

      # lvm lvcreate -n LVDbSys1 -L15G VGExaDb
# lvm lvcreate -n LVDbSys2 -L15G VGExaDb
# lvm lvcreate -n LVDbOra1 -L200G VGExaDb
# lvm lvcreate -n LVDbHome -L4G VGExaDb
# lvm lvcreate -n LVDbVar1 -L2G VGExaDb
# lvm lvcreate -n LVDbVar2 -L2G VGExaDb
# lvm lvcreate -n LVDbVarLog -L18G VGExaDb
# lvm lvcreate -n LVDbVarLogAudit -L1G VGExaDb
# lvm lvcreate -n LVDbTmp -L3G VGExaDb
    2. Create the file systems.
      # mkfs.xfs -f /dev/VGExaDb/LVDbSys1
# mkfs.xfs -f /dev/VGExaDb/LVDbSys2
# mkfs.xfs -f /dev/VGExaDb/LVDbOra1
# mkfs.xfs -f /dev/VGExaDb/LVDbHome
# mkfs.xfs -f /dev/VGExaDb/LVDbVar1
# mkfs.xfs -f /dev/VGExaDb/LVDbVar2
# mkfs.xfs -f /dev/VGExaDb/LVDbVarLog
# mkfs.xfs -f /dev/VGExaDb/LVDbVarLogAudit
# mkfs.xfs -f /dev/VGExaDb/LVDbTmp
# mkfs.xfs -f /dev/md24p1
    3. Label the file systems.
      # xfs_admin -L DBSYS /dev/VGExaDb/LVDbSys1
# xfs_admin -L DBORA /dev/VGExaDb/LVDbOra1
# xfs_admin -L HOME /dev/VGExaDb/LVDbHome
# xfs_admin -L VAR /dev/VGExaDb/LVDbVar1
# xfs_admin -L DIAG /dev/VGExaDb/LVDbVarLog
# xfs_admin -L AUDIT /dev/VGExaDb/LVDbVarLogAudit
# xfs_admin -L TMP /dev/VGExaDb/LVDbTmp
# xfs_admin -L BOOT /dev/md24p1
    4. Create mount points for all the partitions to mirror the original system, and mount the respective partitions.

      For example, assuming that /mnt is used as the top level directory for the recovery operation, you could use the following commands to create the directories and mount the partitions: 

      # mount -t xfs /dev/VGExaDb/LVDbSys1 /mnt
# mkdir -p /mnt/u01
# mount -t xfs /dev/VGExaDb/LVDbOra1 /mnt/u01
# mkdir -p /mnt/home
# mount -t xfs /dev/VGExaDb/LVDbHome /mnt/home
# mkdir -p /mnt/var
# mount -t xfs /dev/VGExaDb/LVDbVar1 /mnt/var
# mkdir -p /mnt/var/log
# mount -t xfs /dev/VGExaDb/LVDbVarLog /mnt/var/log
# mkdir -p /mnt/var/log/audit
# mount -t xfs /dev/VGExaDb/LVDbVarLogAudit /mnt/var/log/audit
# mkdir -p /mnt/tmp
# mount -t xfs /dev/VGExaDb/LVDbTmp /mnt/tmp
# mkdir -p /mnt/boot
# mount -t xfs /dev/md24p1 /mnt/boot
  10. Create the system swap space.
    For Oracle Exadata X10M the default swap size is 16 GB.

    For example:

    # lvm lvcreate -n LVDbSwap1 -L16G VGExaDb
# mkswap -L SWAP /dev/VGExaDb/LVDbSwap1
  11. Create /mnt/boot/efi, label /dev/md24p2, and mount /dev/md24p2 on /mnt/boot/efi with type vfat.
    # mkdir /mnt/boot/efi
# dosfslabel /dev/md24p2 ESP
# mount /dev/md24p2 /mnt/boot/efi -t vfat
  12. Bring up the network.
    # ip address add ip_address_for_eth0/netmask_for_eth0 dev eth0
# ip link set up eth0
# ip route add default via gateway_address dev eth0
  13. Mount the NFS server where you have the backup.

    The following example assumes that the backup is located in the /export directory of the NFS server with IP address nfs_ip. 

    # mkdir -p /root/mnt
# mount -t nfs -o ro,intr,soft,proto=tcp,nolock nfs_ip:/export /root/mnt
  14. Restore from backup.
    # tar --acls --xattrs --xattrs-include=* --format=pax -pjxvf /root/mnt/mybackup.tar.bz2 -C /mnt
  15. Unmount the restored file systems.

    For example:

    # umount /mnt/tmp
# umount /mnt/var/log/audit
# umount /mnt/var/log
# umount /mnt/var
# umount /mnt/home
# umount /mnt/u01
# umount /mnt/boot/efi
# umount /mnt/boot
# umount /mnt
  16. Check the boot devices and set the boot order.
    1. Check the available boot devices, and identify the boot device that is associated with \EFI\REDHAT\SHIMX64.EFI.

      For example:

      # efibootmgr -v
BootCurrent: 000C
Timeout: 1 seconds
BootOrder: 000C,0000,0001,0008,0009,0004,0005,0006,0007,0002,0003,000D
Boot0000* RedHat Boot Manager HD(2,GPT,9dd64f77-490d-4ece-8aed-bc4d9e82d10f,0xe69fdf,0x80000)/File(\EFI\REDHAT\SHIMX64.EFI)
Boot0001* NET0:PXE IPv4 Intel(R) I210 Gigabit  Network Connection /Pci(0x1c,0x4)/Pci(0x0,0x0)/MAC(0010e0fc6e94,1)/IPv4(0.0.0.00.0.0.0,0,0)..BO
Boot0002* PCIE5:PXE IPv4 Mellanox Network Adapter - B8:CE:F6:22:38:0A /Pci(0x2,0x0)/Pci(0x0,0x0)/MAC(b8cef622380a,1)/IPv4(0.0.0.00.0.0.0,0,0)..BO
Boot0003* PCIE5:PXE IPv4 Mellanox Network Adapter - B8:CE:F6:22:38:0B /Pci(0x2,0x0)/Pci(0x0,0x1)/MAC(b8cef622380b,1)/IPv4(0.0.0.00.0.0.0,0,0)..BO
Boot0004* PCIE3:PXE IPv4 Oracle Quad Port 10GBase-T Adapter /Pci(0x2,0x0)/Pci(0x0,0x0)/MAC(3cfdfe915070,1)/IPv4(0.0.0.00.0.0.0,0,0)..BO
Boot0005* PCIE3:PXE IPv4 Oracle Quad Port 10GBase-T Adapter /Pci(0x2,0x0)/Pci(0x0,0x1)/MAC(3cfdfe915071,1)/IPv4(0.0.0.00.0.0.0,0,0)..BO
Boot0006* PCIE3:PXE IPv4 Oracle Quad Port 10GBase-T Adapter /Pci(0x2,0x0)/Pci(0x0,0x2)/MAC(3cfdfe915072,1)/IPv4(0.0.0.00.0.0.0,0,0)..BO
Boot0007* PCIE3:PXE IPv4 Oracle Quad Port 10GBase-T Adapter /Pci(0x2,0x0)/Pci(0x0,0x3)/MAC(3cfdfe915073,1)/IPv4(0.0.0.00.0.0.0,0,0)..BO
Boot0008* PCIE1:PXE IPv4 Mellanox Network Adapter - B8:CE:F6:44:51:9C /Pci(0x2,0x0)/Pci(0x0,0x0)/MAC(b8cef644519c,1)/IPv4(0.0.0.00.0.0.0,0,0)..BO
Boot0009* PCIE1:PXE IPv4 Mellanox Network Adapter - B8:CE:F6:44:51:9D /Pci(0x2,0x0)/Pci(0x0,0x1)/MAC(b8cef644519d,1)/IPv4(0.0.0.00.0.0.0,0,0)..BO
Boot000C* USB:SP:SUN Remote ISO CDROM1.01 /Pci(0x14,0x0)/USB(7,0)/USB(3,0)/CDROM(1,0x28,0x3100)..BO
Boot000D* Oracle Linux (grubx64.efi) HD(2,GPT,9dd64f77-490d-4ece-8aed-bc4d9e82d10f,0xe69fdf,0x80000)/File(\EFI\REDHAT\GRUBX64.EFI)..BO
MirroredPercentageAbove4G: 0.00
MirrorMemoryBelow4GB: false
    2. Configure the device that is associated with \EFI\REDHAT\SHIMX64.EFI to be first in the boot order.

      For example:

      # efibootmgr -o 0000
BootCurrent: 000C
Timeout: 1 seconds
BootOrder: 0000
Boot0000* RedHat Boot Manager
Boot0001* NET0:PXE IPv4 Intel(R) I210 Gigabit  Network Connection
Boot0002* PCIE5:PXE IPv4 Mellanox Network Adapter - B8:CE:F6:22:38:0A
Boot0003* PCIE5:PXE IPv4 Mellanox Network Adapter - B8:CE:F6:22:38:0B
Boot0004* PCIE3:PXE IPv4 Oracle Quad Port 10GBase-T Adapter
Boot0005* PCIE3:PXE IPv4 Oracle Quad Port 10GBase-T Adapter
Boot0006* PCIE3:PXE IPv4 Oracle Quad Port 10GBase-T Adapter
Boot0007* PCIE3:PXE IPv4 Oracle Quad Port 10GBase-T Adapter
Boot0008* PCIE1:PXE IPv4 Mellanox Network Adapter - B8:CE:F6:44:51:9C
Boot0009* PCIE1:PXE IPv4 Mellanox Network Adapter - B8:CE:F6:44:51:9D
Boot000C* USB:SP:SUN Remote ISO CDROM1.01
Boot000D* Oracle Linux (grubx64.efi)
MirroredPercentageAbove4G: 0.00
MirrorMemoryBelow4GB: false
  17. Disconnect the diagnostics.iso file.
    See Booting a Server using the Diagnostic ISO File in Oracle Exadata System Software User's Guide.
  18. Restart the system.
    # reboot
  19. Log back into the server as the root user.
  20. Recreate the boot device mirroring configuration.
    1. Back up the /etc/mdadm.conf file.

      For example:

      # cp /etc/mdadm.conf /etc/mdadm.conf.backup
    2. Edit /etc/mdadm.conf and remove the lines starting with ARRAY.

      After you edit the file, the remaining contents should be similar to the following example:

      # cat /etc/mdadm.conf
MAILADDR root 
AUTO +imsm +1.x -all
    3. Recreate the boot device mirroring configuration.
      # mdadm -Esv | grep ^ARRAY >> /etc/mdadm.conf
    4. Examine /etc/mdadm.conf and verify the addition of new lines starting with ARRAY.

      In particular, verify that the file contains entries for /dev/md/24 and /dev/md/25.

      For example:

      # cat /etc/mdadm.conf 
MAILADDR root
AUTO +imsm +1.x -all
ARRAY /dev/md/24 level=raid1 metadata=1.2 num-devices=2 UUID=2a92373f:572a5a3a:807ae329:b4135cf3 name=localhost:24
ARRAY /dev/md/25 level=raid1 metadata=1.2 num-devices=2 UUID=cc7b75df:25f3a281:b4b65c44:0b8a2de3 name=localhost:25
  21. Recreate the initramfs image files.
    1. Back up the /boot/initramfs*.img files.

      For example:

      # mkdir /boot/backup
# cp /boot/initramfs*.img /boot/backup
    2. Recreate the initramfs image files.
      # dracut -f
  22. Restart the system.
    # reboot
  23. Log back into the server as the root user.
  24. Run the imageinfo command and verify that the image status is success.

    For example:

    # imageinfo

Kernel version: 5.4.17-2136.320.7.el8uek.x86_64 #2 SMP Mon Jun 5 14:17:11 PDT 2023 x86_64
Image kernel version: 5.4.17-2136.320.7.el8uek
Image version: 23.1.0.0.0.230707
Image activated: 2023-07-07 17:12:37 -0700
Image status: success
Exadata software version: 23.1.0.0.0.230707
Node type: KVMHOST
System partition on device: /dev/mapper/VGExaDb-LVDbSys1
  25. If the recovery was on Oracle Exadata Database Machine Eighth Rack, then perform the procedure described in Configuring Oracle Exadata Database Machine Eighth Rack Oracle Linux Database Server After Recovery.

Parent topic: Recovering Oracle Linux Database Servers Using a Snapshot-Based Backup