1. Managing Storage Devices
  2. Working With Linux I-O Storage

6 Working With Linux I-O Storage

Oracle Linux uses the Linux-IO Target (LIO) to provide the block-storage SCSI target for FCoE, iSCSI, and Mellanox InfiniBand (iSER and SRP). You manage LIO by using the targetcli shell provided in the targetcli package. Note that Mellanox InfiniBand is only supported with UEK. You can install the targetcli package by running: 

sudo dnf install -y targetcli

Fibre Channel over Ethernet (FCoE) encapsulates Fibre Channel packets in Ethernet frames, which enables them to be sent over Ethernet networks. To configure FCoE storage, you need to install the fcoe-utils package that includes both the fcoemon service and the fcoeadm command. You can install the fcoe-utils package by running: 

sudo dnf install -y fcoe-utils

About iSCSI Devices

The Internet Small Computer System Interface (iSCSI) is an IP-based standard for connecting storage devices. iSCSI encapsulates SCSI commands in IP network packets to support data transfer over long distances and sharing of storage by client systems. iSCSI uses the existing IP infrastructure and doesn't require the purchase and installation of fiber-optic cabling and interface adapters that are needed to implement Fibre Channel (FC) storage area networks.

A client system (iSCSI initiator) accesses the storage server (iSCSI target) over an IP network. To an iSCSI initiator, the storage appears to be locally attached.

An iSCSI target is typically a dedicated, network-connected storage device but it can also be a general-purpose computer.

Figure 6-1 shows a simple network where several iSCSI initiators can access the shared storage that's attached to an iSCSI target.

Figure 6-1 iSCSI Initiators and an iSCSI Target Connected via an IP-based Network


The diagram shows a simple Ethernet network where several iSCSI initiators are able to access the shared storage that's attached to an iSCSI target.

A hardware-based iSCSI initiator uses a dedicated iSCSI HBA. Oracle Linux supports iSCSI initiator functionality in software. The kernel-resident device driver uses the existing network interface card (NIC) and network stack to emulate a hardware iSCSI initiator. The iSCSI initiator functionality isn't available at the level of the system BIOS. Thus, you can't boot an Oracle Linux system from iSCSI storage.

To improve performance, some network cards implement TCP/IP Offload Engines (TOE) that can create a TCP frame for the iSCSI packet in hardware. Oracle Linux doesn't support TOE, although suitable drivers might be available directly from some card vendors.

For more information about LIO, see http://linux-iscsi.org/wiki/Main_Page.

Configuring an iSCSI Target

The following procedure describes how to set up a basic iSCSI target on an Oracle Linux system by using block storage backends. Note that you can use other storage backend types to set up an iSCSI target.

In the example, the targetcli command saves the current configuration to /etc/target/saveconfig.json. See the targetcli(8) manual page for additional information.

  1. Run the targetcli interactive shell: 

    sudo targetcli
    targetcli shell version 2.1.53
Copyright 2011-2013 by Datera, Inc and others.
For help on commands, type 'help'.

  2. (Optional) Use the ls command to list the object hierarchy, which is initially empty: 

    ls
    o- / ..................................................................... [...]
  o- backstores .......................................................... [...]
  | o- block .............................................. [Storage Objects: 0]
  | o- fileio ............................................. [Storage Objects: 0]
  | o- pscsi .............................................. [Storage Objects: 0]
  | o- ramdisk ............................................ [Storage Objects: 0]
  o- iscsi ........................................................ [Targets: 0]
  o- loopback ..................................................... [Targets: 0]

  3. Change to the /backstores/block directory and create a block storage object for the disk partitions that you want to provide as LUNs, for example: 

    cd /backstores/block
/backstores/block> create name=LUN_0 dev=/dev/sdb
Created block storage object LUN_0 using /dev/sdb.
/backstores/block> create name=LUN_1 dev=/dev/sdc
Created block storage object LUN_1 using /dev/sdc.

    The names that you assign to the storage objects are arbitrary.

    Note:

    The device path varies based on the Oracle Linux instance's disk configuration.

  4. Change to the /iscsi directory and create an iSCSI target: 

    cd /iscsi
/iscsi> create
Created target iqn.2013-01.com.mydom.host01.x8664:sn.ef8e14f87344.
Created TPG 1.

  5. (Optional): List the target portal group (TPG) hierarchy, which is initially empty: 

    /iscsi> ls
    o- iscsi .......................................................... [Targets: 1]
  o- iqn.2013-01.com.mydom.host01.x8664:sn.ef8e14f87344 .............. [TPGs: 1]
    o- tpg1 ............................................. [no-gen-acls, no-auth]
      o- acls ........................................................ [ACLs: 0]
      o- luns ........................................................ [LUNs: 0]
      o- portals .................................................. [Portals: 0]

  6. Change to the luns subdirectory of the TPG directory hierarchy and add the LUNs to the target portal group: 

    /iscsi> cd iqn.2013-01.com.mydom.host01.x8664:sn.ef8e14f87344/tpg1/luns 
/iscsi/iqn.20...344/tpg1/luns> create /backstores/block/LUN_0
Created LUN 0.
/iscsi/iqn.20...344/tpg1/luns> create /backstores/block/LUN_1
Created LUN 1.

  7. Change to the portals subdirectory of the TPG directory hierarchy and specify the IP address and TCP port of the iSCSI endpoint: 

    /iscsi/iqn.20...344/tpg1/luns> cd ../portals
/iscsi/iqn.20.../tpg1/portals> create 10.150.30.72 3260
Using default IP port 3260
Created network portal 10.150.30.72:3260.

    The default TCP port number is 3260.

    Note:

    An existing default portal would cause the portal creation to fail and a message similar to the following is generated: 
    Could not create NetworkPortal in configFS

    To resolve the issue, delete the default portal, then create the new portal again, for example: 

    /iscsi/iqn.20.../tpg1/portals> delete 0.0.0.0 ip_port=3260

  8. Enable TCP port 3260 either by adding the port or adding the iSCSI target:

    • Add the port: 

      sudo firewall-cmd --permanent --add-port=3260/tcp

    • Add the target: 

      sudo firewall-cmd --permanent --add-service \
iqn.2013-01.com.mydom.host01.x8664:sn.ef8e14f87344

  9. List the object hierarchy, which now shows the configured block storage objects and TPG: 

    /iscsi/iqn.20.../tpg1/portals> ls /
    o- / ..................................................................... [...]
  o- backstores .......................................................... [...]
  | o- block .............................................. [Storage Objects: 1]
  | | o- LUN_0 ....................... [/dev/sdb (10.0GiB) write-thru activated]
  | | o- LUN_1 ....................... [/dev/sdc (10.0GiB) write-thru activated]
  | o- fileio ............................................. [Storage Objects: 0]
  | o- pscsi .............................................. [Storage Objects: 0]
  | o- ramdisk ............................................ [Storage Objects: 0]
  o- iscsi ........................................................ [Targets: 1]
  | o- iqn.2013-01.com.mydom.host01.x8664:sn.ef8e14f87344 ............ [TPGs: 1]
  |   o- tpg1 ........................................... [no-gen-acls, no-auth]
  |     o- acls ...................................................... [ACLs: 0]
  |     o- luns ...................................................... [LUNs: 1]
  |     | o- lun0 ..................................... [block/LUN_0 (/dev/sdb)]
  |     | o- lun1 ..................................... [block/LUN_1 (/dev/sdc)]
  |     o- portals ................................................ [Portals: 1]
  |       o- 10.150.30.72:3260 ............................................ [OK]
  o- loopback ..................................................... [Targets: 0]

  10. Configure the access rights for logins by initiators.

    For example, to configure a demonstration mode that does not require authentication, change to the TGP directory and set the attributes as shown in the following example: 

    /iscsi/iqn.20.../tpg1/portals> cd ..
/iscsi/iqn.20...14f87344/tpg1> set attribute authentication=0 demo_mode_write_protect=0
                        generate_node_acls=1 cache_dynamic_acls=1
Parameter authentication is now '0'.
Parameter demo_mode_write_protect is now '0'.
Parameter generate_node_acls is now '1'.
Parameter cache_dynamic_acls is now '1'.

    Caution:

    The demonstration mode is inherently insecure. For information about configuring secure authentication modes, see http://linux-iscsi.org/wiki/ISCSI#Define_access_rights.

  11. Change to the root (/) directory and save the configuration.

    This step ensures that the changes persist across system reboots. Omitting the step might result in an empty configuration. 

    /iscsi/iqn.20...14f87344/tpg1> cd /
/> saveconfig
Last 10 configs saved in /etc/target/backup.
Configuration saved to /etc/target/saveconfig.json

  12. Enable the target service. 

    sudo systemctl enable target.service

Restoring a Saved Configuration for an iSCSI target

To restore a saved configuration for an iSCSI target, start the targetcli interactive shell and then run the following command: 

sudo targetcli
targetcli shell version 2.1.fb46
Copyright 2011-2013 by Datera, Inc and others.
For help on commands, type 'help'.
/> restoreconfig /etc/target/saveconfig.json

The /etc/target/saveconfig.json file stores the most recently saved configuration.

As an alternative, run the following command to restore saved configurations from previous versions: 

/> restoreconfig /etc/target/backup/saveconfig-20180516-18:53:29.json

Configuring an iSCSI Initiator

  1. Install the iscsi-initiator-utils package: 

    sudo dnf install iscsi-initiator-utils

  2. Use a discovery method, such as SendTargets or the Internet Storage Name Service (iSNS), to discover the iSCSI targets at the specified IP address.

    For example, you would use SendTargets as follows: 

    sudo iscsiadm -m discovery -t sendtargets -p 10.150.30.72

    The following output is displayed: 

    10.150.30.72:3260,1 iqn.2013-01.com.mydom.host01.x8664:sn.ef8e14f87344

    This command also starts the iscsid service if it is not already running.

    Note:

    Before running the discovery process, ensure that the firewall is configured to accept communication with an iSCSI target and that ICMP traffic is allowed.

  3. Display information about the targets that are now stored in the discovery database. 

    sudo iscsiadm -m discoverydb -t st -p 10.150.30.72
    # BEGIN RECORD 6.2.0.873-14
discovery.startup = manual
discovery.type = sendtargets
discovery.sendtargets.address = 10.150.30.72
discovery.sendtargets.port = 3260
discovery.sendtargets.auth.authmethod = None
discovery.sendtargets.auth.username = <empty>
discovery.sendtargets.auth.password = <empty>
discovery.sendtargets.auth.username_in = <empty>
discovery.sendtargets.auth.password_in = <empty>
discovery.sendtargets.timeo.login_timeout = 15
discovery.sendtargets.use_discoveryd = No
discovery.sendtargets.discoveryd_poll_inval = 30
discovery.sendtargets.reopen_max = 5
discovery.sendtargets.timeo.auth_timeout = 45
discovery.sendtargets.timeo.active_timeout = 30
discovery.sendtargets.iscsi.MaxRecvDataSegmentLength = 32768
# END RECORD

  4. Establish a session and log in to a specific target: 

    sudo iscsiadm -m node -T iqn.2013-01.com.mydom.host01.x8664:sn.ef8e14f87344 \
-p 10.150.30.72:3260 -l

    Login to [iface: default, target: iqn.2003-01.org.linux-iscsi.localhost.x8664:
sn.ef8e14f87344, portal: 10.150.30.72,3260] successful.

  5. Verify that the session is active and display the available LUNs: 

    sudo iscsiadm -m session -P 3

    The following output is displayed: 

    iSCSI Transport Class version 2.0-870
version 6.2.0.873-14
Target: iqn.2003-01.com.mydom.host01.x8664:sn.ef8e14f87344 (non-flash)
	Current Portal: 10.0.0.2:3260,1
	Persistent Portal: 10.0.0.2:3260,1
		**********
		Interface:
		**********
		Iface Name: default
		Iface Transport: tcp
		Iface Initiatorname: iqn.1994-05.com.mydom:ed7021225d52
		Iface IPaddress: 10.0.0.2
		Iface HWaddress: <empty>
		Iface Netdev: <empty>
		SID: 5
		iSCSI Connection State: LOGGED IN
		iSCSI Session State: LOGGED_IN
		Internal iscsid Session State: NO CHANGE
.
.
.
		************************
		Attached SCSI devices:
		************************
		Host Number: 8	State: running
		scsi8 Channel 00 Id 0 Lun: 0
			Attached scsi disk sdb		State: running
		scsi8 Channel 00 Id 0 Lun: 1
			Attached scsi disk sdc		State: running

The LUNs are represented as SCSI block devices (sd*) in the local /dev directory, for example: 

sudo fdisk -l | grep /dev/sd[bc]
Disk /dev/sdb: 10.7 GB, 10737418240 bytes, 20971520 sectors
Disk /dev/sdc: 10.7 GB, 10737418240 bytes, 20971520 sectors

To distinguish between target LUNs, examine the paths under /dev/disk/by-path, which is displayed by using the following command: 

ls -l /dev/disk/by-path/
lrwxrwxrwx  1 root root  9 May 15 21:05
  ip-10.150.30.72:3260-iscsi-iqn.2013-01.com.mydom.host01.x8664:
  sn.ef8e14f87344-lun-0 -> ../../sdb
lrwxrwxrwx 1 root root  9 May 15 21:05
  ip-10.150.30.72:3260-iscsi-iqn.2013-01.com.mydom.host01.x8664:
  sn.ef8e14f87344-lun-1 -> ../../sdc

You can view the initialization messages for the LUNs in the /var/log/messages file, for example: 

grep sdb /var/log/messages
...
May 18 14:19:36 localhost kernel: [12079.963376] sd 8:0:0:0: [sdb] Attached SCSI disk
...

You configure and use a LUN in the same way that you would any other physical storage device, for example, as an LVM physical volume, a file system, a swap partition, an Automatic Storage Management (ASM) disk, or a raw device.

When creating mount entries for iSCSI LUNs in /etc/fstab, specify the _netdev option, for example: 

UUID=084591f8-6b8b-c857-f002-ecf8a3b387f3     /iscsi_mount_point     ext4     _netdev   0  0

This option indicates that the file system resides on a device that requires network access, and prevents the system from mounting the file system until the network has been enabled.

Note:

When adding iSCSI LUN entries to /etc/fstab, see the LUN by using UUID= UUID rather than the device path. A device path can change after reconnecting the storage or rebooting the system. To display the UUID of a block device, the blkid command.

Any discovered LUNs remain available across reboots provided that the target continues to serve those LUNs and you don't log the system off the target.

For more information, see the iscsiadm(8) and iscsid(8) manual pages.

Updating the Discovery Database

If the LUNs that are available on an iSCSI target change, use the iscsiadm command on an iSCSI initiator to update the entries in its discovery database. The following example assumes that the target supports the SendTargets discovery method

To add new records that aren't in the database: 

sudo iscsiadm --mode discoverydb -type st -p 10.150.30.72 -o new --discover

To update existing records in the database: 

sudo iscsiadm -m discoverydb -t st -p 10.150.30.72 -o update --discover

To delete records from the database that are no longer supported by the target: 

sudo iscsiadm -m discoverydb -t st -p 10.150.30.72 -o delete --discover

For more information, see the iscsiadm(8) manual page.