1. Managing Storage and Storage Devices
  2. Working With iSCSI Devices

5 Working With iSCSI Devices

WARNING:

Oracle Linux 7 is now in Extended Support. See Oracle Linux Extended Support and Oracle Open Source Support Policies for more information.

Migrate applications and data to Oracle Linux 8 or Oracle Linux 9 as soon as possible.

This chapter discusses using iSCSI devices for data storage.

About Linux-IO Storage Configuration

Oracle Linux 7 with both UEK and RHCK uses the Linux-IO Target (LIO) to provide the block-storage SCSI target for FCoE, iSCSI, and Mellanox InfiniBand (iSER and SRP). You can manage LIO by using the targetcli shell provided in the targetcli package. Note that Mellanox InfiniBand is only supported with UEK.

Fibre Channel over Ethernet (FCoE) encapsulates Fibre Channel packets in Ethernet frames, which allows the packets to be sent over Ethernet networks. To configure FCoE storage, you also need to install the fcoe-utils package, which provides the fcoemon service and the fcoeadm command.

The Internet Small Computer System Interface (iSCSI) is an IP-based standard for connecting storage devices. iSCSI encapsulates SCSI commands in IP network packets, which allows data transfer over long distances and sharing of storage by client systems. As iSCSI uses the existing IP infrastructure, it does not require the purchase and installation of fiber-optic cabling and interface adapters that are needed to implement Fibre Channel (FC) storage area networks.

A client system (iSCSI initiator) accesses the storage server (iSCSI target) over an IP network. To an iSCSI initiator, the storage appears to be locally attached.

An iSCSI target is typically a dedicated, network-connected storage device but it can also be a general-purpose computer.

Figure 5-1 shows a simple network where several iSCSI initiators are able to access the shared storage that is attached to an iSCSI target.

Figure 5-1 iSCSI Initiators and an iSCSI Target Connected via an IP-based Network


The diagram shows a simple Ethernet network where several iSCSI initiators are able to access the shared storage that is attached to an iSCSI target.

A hardware-based iSCSI initiator uses a dedicated iSCSI HBA. Oracle Linux supports iSCSI initiator functionality in software. The kernel-resident device driver uses the existing network interface card (NIC) and network stack to emulate a hardware iSCSI initiator. As the iSCSI initiator functionality is not available at the level of the system BIOS, you cannot boot an Oracle Linux system from iSCSI storage .

To improve performance, some network cards implement TCP/IP Offload Engines (TOE) that can create a TCP frame for the iSCSI packet in hardware. Oracle Linux does not support TOE, although suitable drivers may be available directly from some card vendors.

For more information about LIO, see http://linux-iscsi.org/wiki/Main_Page.

Configuring an iSCSI Target

The following procedure describes how to set up a basic iSCSI target on an Oracle Linux system by using block storage backends. Note that you can use other storage backend types to set up an iSCSI target.

When you use the targetcli command, it saves the current configuration to the JSON-format file /etc/target/saveconfig.json. See the targetcli(8) manual page for additional information.

  1. Run the targetcli interactive shell: 

    sudo targetcli
    targetcli shell version 2.1.fb31
Copyright 2011-2013 by Datera, Inc and others.
For help on commands, type 'help'. 

/>

    At the interactive shell prompt, list the object hierarchy, which is initially empty: 

    /> ls
    o- / ..................................................................... [...]
  o- backstores .......................................................... [...]
  | o- block .............................................. [Storage Objects: 0]
  | o- fileio ............................................. [Storage Objects: 0]
  | o- pscsi .............................................. [Storage Objects: 0]
  | o- ramdisk ............................................ [Storage Objects: 0]
  o- iscsi ........................................................ [Targets: 0]
  o- loopback ..................................................... [Targets: 0]

  2. Change to the /backstores/block directory and create a block storage object for the disk partitions that you want to provide as LUNs, for example: 

    /> cd /backstores/block
/backstores/block> create name=LUN_0 dev=/dev/sdb
Created block storage object LUN_0 using /dev/sdb.
/backstores/block> create name=LUN_1 dev=/dev/sdc
Created block storage object LUN_1 using /dev/sdc.

    The names that you assign to the storage objects are arbitrary.

    Note:

    The device path varies, based on the Oracle Linux instance's disk configuration.

  3. Change to the /iscsi directory and create an iSCSI target: 

    /> cd /iscsi
/iscsi> create
    Created target iqn.2013-01.com.mydom.host01.x8664:sn.ef8e14f87344.
Created TPG 1.

    List the target portal group (TPG) hierarchy, which is initially empty: 

    /iscsi> ls
    o- iscsi .......................................................... [Targets: 1]
  o- iqn.2013-01.com.mydom.host01.x8664:sn.ef8e14f87344 .............. [TPGs: 1]
    o- tpg1 ............................................. [no-gen-acls, no-auth]
      o- acls ........................................................ [ACLs: 0]
      o- luns ........................................................ [LUNs: 0]
      o- portals .................................................. [Portals: 0]

  4. Change to the luns subdirectory of the TPG directory hierarchy and add the LUNs to the target portal group: 

    /iscsi> cd iqn.2013-01.com.mydom.host01.x8664:sn.ef8e14f87344/tpg1/luns 
/iscsi/iqn.20...344/tpg1/luns> create /backstores/block/LUN_0
Created LUN 0.
/iscsi/iqn.20...344/tpg1/luns> create /backstores/block/LUN_1
Created LUN 1.

  5. Change to the portals subdirectory of the TPG directory hierarchy and specify the IP address and TCP port of the iSCSI endpoint: 

    /iscsi/iqn.20...344/tpg1/luns> cd ../portals
/iscsi/iqn.20.../tpg1/portals> create 10.150.30.72 3260
Using default IP port 3260
Created network portal 10.150.30.72:3260.

    If you omit the TCP port number, the default value is 3260.

    Note:

    If the portal creation fails, and a message similar to the following is displayed, it is most likely because a default portal was previously created: 
    Could not create NetworkPortal in configFS

    In this case, first delete the default portal, then repeat Step 5 to create the new portal. 

    /iscsi/iqn.20.../tpg1/portals> delete 0.0.0.0 ip_port=3260

  6. Enable TCP port 3260 by running either of the following commands: 

    sudo firewall-cmd --permanent --add-port=3260/tcp
    sudo firewall-cmd --permanent --add-service iscsi-target

  7. List the object hierarchy, which now shows the configured block storage objects and TPG: 

    /iscsi/iqn.20.../tpg1/portals> ls /
o- / ..................................................................... [...]
  o- backstores .......................................................... [...]
  | o- block .............................................. [Storage Objects: 1]
  | | o- LUN_0 ....................... [/dev/sdb (10.0GiB) write-thru activated]
  | | o- LUN_1 ....................... [/dev/sdc (10.0GiB) write-thru activated]
  | o- fileio ............................................. [Storage Objects: 0]
  | o- pscsi .............................................. [Storage Objects: 0]
  | o- ramdisk ............................................ [Storage Objects: 0]
  o- iscsi ........................................................ [Targets: 1]
  | o- iqn.2013-01.com.mydom.host01.x8664:sn.ef8e14f87344 ............ [TPGs: 1]
  |   o- tpg1 ........................................... [no-gen-acls, no-auth]
  |     o- acls ...................................................... [ACLs: 0]
  |     o- luns ...................................................... [LUNs: 1]
  |     | o- lun0 ..................................... [block/LUN_0 (/dev/sdb)]
  |     | o- lun1 ..................................... [block/LUN_1 (/dev/sdc)]
  |     o- portals ................................................ [Portals: 1]
  |       o- 10.150.30.72:3260 ............................................ [OK]
  o- loopback ..................................................... [Targets: 0]

  8. Configure the access rights for logins by initiators. For example, to configure demonstration mode that does not require authentication, change to the TGP directory and set the values of the authentication and demo_mode_write_protect attributes to 0 and generate_node_acls cache_dynamic_acls to 1: 

    /iscsi/iqn.20.../tpg1/portals> cd ..
/iscsi/iqn.20...14f87344/tpg1> set attribute authentication=0 demo_mode_write_protect=0
                        generate_node_acls=1 cache_dynamic_acls=1
Parameter authentication is now '0'.
Parameter demo_mode_write_protect is now '0'.
Parameter generate_node_acls is now '1'.
Parameter cache_dynamic_acls is now '1'.

    Caution:

    Demonstration mode is inherently insecure. For information about configuring secure authentication modes, see http://linux-iscsi.org/wiki/ISCSI#Define_access_rights.

  9. Change to the root directory and save the configuration. 

    /iscsi/iqn.20...14f87344/tpg1> cd /
/> saveconfig
Last 10 configs saved in /etc/target/backup.
Configuration saved to /etc/target/saveconfig.json

  10. Enable the target service. 

    # systemctl enable target.service

Note:

Saving the configuration and enabling the target service ensures that the changes persist across system reboots. Note also that failure to run these commands can result in an empty configuration.

Restoring a Saved Configuration for an iSCSI target

To restore a saved configuration for an iSCSI target, start the targetcli interactive shell and then run the following command: 

sudo targetcli
targetcli shell version 2.1.fb46
Copyright 2011-2013 by Datera, Inc and others.
For help on commands, type 'help'.

/>

At the interactive shell prompt, type the appropriate command, for example:

/> restoreconfig /etc/target/saveconfig.json

In the previous example, /etc/target/saveconfig.json is the most recently saved configuration.

Or, you can run the following command to restore saved configurations from previous versions: 

/> restoreconfig /etc/target/backup/saveconfig-20180516-18:53:29.json

Configuring an iSCSI Initiator

To configure an Oracle Linux system as an iSCSI initiator:

  1. Install the iscsi-initiator-utils package: 

    sudo yum install iscsi-initiator-utils

  2. Use a discovery method to discover the iSCSI targets at the specified IP address.

    You can use either the SendTargets or the Internet Storage Name Service (iSNS) discovery method.

    For example, you would use the SendTargets discovery method as follows: 

    # iscsiadm -m discovery -t sendtargets -p 10.150.30.72
    10.150.30.72:3260,1 iqn.2013-01.com.mydom.host01.x8664:sn.ef8e14f87344

    This command also starts the iscsid service if it is not already running.

    • If the network connection fails, adjust the firewall settings for your network to allow communication with the iSCSI target: 

      sudo iptables -D INPUT -j REJECT --reject-with icmp-host-prohibited
sudo iptables -D FORWARD -j REJECT --reject-with icmp-host-prohibited

    • Then, re-run the command using the SendTargets discovery method: 

      sudo iscsiadm -m discovery -t sendtargets -p 10.150.30.72
      10.150.30.72:3260,1 iqn.2013-01.com.mydom.host01.x8664:sn.ef8e14f87344

  3. Display information about the targets that are now stored in the discovery database by running the following command: 

    sudo iscsiadm -m discoverydb -t st -p 10.150.30.72
    # BEGIN RECORD 6.2.0.873-14
discovery.startup = manual
discovery.type = sendtargets
discovery.sendtargets.address = 10.150.30.72
discovery.sendtargets.port = 3260
discovery.sendtargets.auth.authmethod = None
discovery.sendtargets.auth.username = <empty>
discovery.sendtargets.auth.password = <empty>
discovery.sendtargets.auth.username_in = <empty>
discovery.sendtargets.auth.password_in = <empty>
discovery.sendtargets.timeo.login_timeout = 15
discovery.sendtargets.use_discoveryd = No
discovery.sendtargets.discoveryd_poll_inval = 30
discovery.sendtargets.reopen_max = 5
discovery.sendtargets.timeo.auth_timeout = 45
discovery.sendtargets.timeo.active_timeout = 30
discovery.sendtargets.iscsi.MaxRecvDataSegmentLength = 32768
# END RECORD

  4. Establish a session and log in to a specific target: 

    sudo iscsiadm -m node -T iqn.2013-01.com.mydom.host01.x8664:sn.ef8e14f87344 -p 10.150.30.72:3260 -l
    Login to [iface: default, target: iqn.2003-01.org.linux-iscsi.localhost.x8664:
sn.ef8e14f87344, portal: 10.150.30.72,3260] successful.

  5. Verify that the session is active and display the available LUNs: 

    sudo iscsiadm -m session -P 3
    iSCSI Transport Class version 2.0-870
version 6.2.0.873-14
Target: iqn.2003-01.com.mydom.host01.x8664:sn.ef8e14f87344 (non-flash)
	Current Portal: 10.0.0.2:3260,1
	Persistent Portal: 10.0.0.2:3260,1
		**********
		Interface:
		**********
		Iface Name: default
		Iface Transport: tcp
		Iface Initiatorname: iqn.1994-05.com.mydom:ed7021225d52
		Iface IPaddress: 10.0.0.2
		Iface HWaddress: <empty>
		Iface Netdev: <empty>
		SID: 5
		iSCSI Connection State: LOGGED IN
		iSCSI Session State: LOGGED_IN
		Internal iscsid Session State: NO CHANGE
.
.
.
		************************
		Attached SCSI devices:
		************************
		Host Number: 8	State: running
		scsi8 Channel 00 Id 0 Lun: 0
			Attached scsi disk sdb		State: running
		scsi8 Channel 00 Id 0 Lun: 1
			Attached scsi disk sdc		State: running

    The LUNs are represented as SCSI block devices (sd*) in the local /dev directory, for example: 

    sudo fdisk -l | grep /dev/sd[bc]
    Disk /dev/sdb: 10.7 GB, 10737418240 bytes, 20971520 sectors
Disk /dev/sdc: 10.7 GB, 10737418240 bytes, 20971520 sectors

    To distinguish between target LUNs, examine the paths under /dev/disk/by-path: 

    ls -l /dev/disk/by-path/
    lrwxrwxrwx  1 root root  9 May 15 21:05
  ip-10.150.30.72:3260-iscsi-iqn.2013-01.com.mydom.host01.x8664:
  sn.ef8e14f87344-lun-0 -> ../../sdb
lrwxrwxrwx 1 root root  9 May 15 21:05
  ip-10.150.30.72:3260-iscsi-iqn.2013-01.com.mydom.host01.x8664:
  sn.ef8e14f87344-lun-1 -> ../../sdc

    You can view the initialization messages for the LUNs in the /var/log/messages file as follows: 

    sudo grep sdb /var/log/messages
    ...
May 18 14:19:36 localhost kernel: [12079.963376] sd 8:0:0:0: [sdb] Attached SCSI disk
...

    You can configure and use a LUN in the same way that you would any other physical storage device, for example, as an LVM physical volume, a file system, a swap partition, an Automatic Storage Management (ASM) disk, or a raw device.

    Specify the _netdev option when creating mount entries for iSCSI LUNs in /etc/fstab, for example: 

    UUID=084591f8-6b8b-c857-f002-ecf8a3b387f3     /iscsi_mount_point     ext4     _netdev   0  0

    This option indicates the file system resides on a device that requires network access, and prevents the system from attempting to mount the file system until the network has been enabled.

    Note:

    Specify an iSCSI LUN in /etc/fstab by using UUID= UUID rather than the device path. A device path can change after re-connecting the storage or rebooting the system. You can use the blkid command to display the UUID of a block device.

    Any discovered LUNs remain available across reboots provided that the target continues to serve those LUNs and you do not log the system off the target.

For more information, see the iscsiadm(8) and iscsid(8) manual pages.

Updating the Discovery Database

If the LUNs that are available on an iSCSI target change, you can use the iscsiadm command on an iSCSI initiator to update the entries in its discovery database. The following example assume that the target supports the SendTargets discovery method

To add new records that are not currently in the database: 

sudo iscsiadm --mode discoverydb -type st -p 10.150.30.72 -o new --discover

To update existing records in the database: 

sudo iscsiadm -m discoverydb -t st -p 10.150.30.72 -o update --discover

To delete records from the database that are no longer supported by the target: 

sudo iscsiadm -m discoverydb -t st -p 10.150.30.72 -o delete --discover

For more information, see the iscsiadm(8) manual page.