Configure iSCSI Persistence Storage

Configuration guide for demo / lab environment purposes, not for production.

Author(s): Tamas Bures | Created: 29 May 2020 | Last modified: 29 May 2020

Tested on: Red Hat OpenShift Platform v 4.4.3

Table of contents

Configure iSCSI Persistence Storage
Prerequisites
Resources
Additional resources
Installation and configure iSCSI target
Terminology
Create server (target)
Configure iSCSI target
Configure OCP

Configure iSCSI Persistence Storage↑

In this guide, I will configure iSCSI Persistence Storage using Red Hat Enterprise Linux 7.5 for Red Hat OpenShift Container Platform 4.4.3 (OCP).

Prerequisites↑

A working ESXi environment: 10.109.10.101
Uploaded ISO files to ESXi datastore (see Resources)
Installed and configured OCP 4.4.3 platform

Resources↑

Download the following files and resources:

Red Hat Enterprise Linux x86_64 7.5 Server installer (*.iso)

Additional resources↑

Access to Red Hat additional repositories:
- Optional
- Extras
- EPEL

Installation and configure iSCSI target↑

Terminology↑

iSCSI has its own vocabulary that takes some getting used to:

Target – an iSCSI export. This is what the iSCSI server provides.
Initiator – The client.
IQN – iSCSI Qualified Name. A unique name that identifies iSCSI targets and initiators.
LUN – Logical Unit Number. This represents the actual disk. There can be multiple LUNs exported by a Target.
ACL – Access Control List. Usually a list of allowed initiator IQNs
Portal – an IP and port combination that the target is accessible from.
TPG – Target Portal Group. A grouping of LUNs, ACLs, and Portals that defines access permissions for the Target.

Create server (target)↑

In order to be able to configure OCP platform to leverage our freshly created iSCSI storage, we need to install an iSCSI target. To do so I will use a minimal RHEL 7.5. I assume the minimal RHEL 7.5 is installed and it can access to Red Hat repositories, the image booted for the first time.

Server details:

Hostname: storage.cp4s.sechu.ibm
IP: 10.109.200.10
2 disks:
- sda: 50 GB for system
- sdb: 4 TB for storage

Disable SELINUX:

 sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config

Update system and install required packages:

 yum -y update
 yum -y install telnet ftp net-tools mc mlocate deltarpm bind bind-utils targetcli iscsi-initiator-utils

Cleaning up:

 rm -vf /root/install*log
 rm -vf /root/anaconda-ks.cfg

Disable unneccessary services:

 for i in abrt-ccpp abrtd atd auditd blk-availability certmonger cpuspeed cups \
     mcelogd mdmonitor netconsole numad oddjobd portreserve rhnsd rhsmcertd smartd \
     winbind postfix iptables ip6tables firewalld kdump; \
     do \
         systemctl disable $i; \
 done

Copy the hostname to the /etc/hosts file in case of network loss:

 cp /etc/hosts /etc/hosts.backup
 echo `ifconfig | sed -En \ 
 's/127.0.0.1//;s/.*inet (addr:)?(([0-9]*\.){3}[0-9]*).*/\2/p'` `hostname` >> /etc/hosts

Reboot

Configure iSCSI target↑

Verify, that sdb exists:

 lsblk

Example output:

 [root@storage ~]# lsblk
 NAME          MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
 sda             8:0    0   50G  0 disk
 |-sda1          8:1    0    1G  0 part /boot
 `-sda2          8:2    0   49G  0 part
   |-rhel-root 253:0    0 45.1G  0 lvm  /
   `-rhel-swap 253:1    0  3.9G  0 lvm  [SWAP]
 sdb             8:16   0    4T  0 disk         <------- THIS ONE
 sr0            11:0    1 1024M  0 rom

Create physical volume on the entire disk:

 pvcreate /dev/sdb

Example output:

 Physical volume "/dev/sdb" successfully created.

Create volume group:

 vgcreate cp4s_storage /dev/sdb

Example output:

 Volume group "cp4s_storage" successfully created

Check volume groups:

vgs

Example output:

 VG           #PV #LV #SN Attr   VSize   VFree
cp4s_storage   1   0   0 wz--n-  <4.00t <4.00t
rhel           1   2   0 wz--n- <49.00g  4.00m

Create logical volume:

 lvcreate --name cp4s_lv -l 100%FREE cp4s_storage

Example output:

 Logical volume "cp4s_lv" created.

Check LVM:

 lsblk

Example output:

 NAME                   MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
 sda                      8:0    0   50G  0 disk
 |-sda1                   8:1    0    1G  0 part /boot
 `-sda2                   8:2    0   49G  0 part
   |-rhel-root          253:0    0 45.1G  0 lvm  /
   `-rhel-swap          253:1    0  3.9G  0 lvm  [SWAP]
 sdb                      8:16   0    4T  0 disk
 `-cp4s_storage-cp4s_lv 253:2    0    4T  0 lvm        <-------- THIS ONE
 sr0                     11:0    1 1024M  0 rom

Start iSCSI configuration tool:

 targetcli

and issue the ls command:

 Warning: Could not load preferences file /root/.targetcli/prefs.bin.
 targetcli shell version 2.1.fb46
 Copyright 2011-2013 by Datera, Inc and others.
 For help on commands, type 'help'.

 /> ls
 o- / .................................................................. [...]
   o- backstores ....................................................... [...]
   | o- block ........................................... [Storage Objects: 0]
   | o- fileio .......................................... [Storage Objects: 0]
   | o- pscsi ........................................... [Storage Objects: 0]
   | o- ramdisk ......................................... [Storage Objects: 0]
   o- iscsi ..................................................... [Targets: 0]
   o- loopback .................................................. [Targets: 0]

Create backstore

 /> cd backstores/block
 /backstores/block> create dev=/dev/mapper/cp4s_storage-cp4s_lv name=cp4s_iscsi_pv

Example output:

 Created block storage object cp4s_iscsi_pv using /dev/mapper/cp4s_storage-cp4s_lv.

List again with ls. Example output:

 /backstores/block> ls
 o- block ............................................... [Storage Objects: 1]
   o- cp4s_iscsi_pv  [/dev/mapper/cp4s_storage-cp4s_lv (4.0TiB) write-thru deactivated]
     o- alua ................................................ [ALUA Groups: 1]
       o- default_tg_pt_gp .................... [ALUA state: Active/optimized]

Create target:

 cd /iscsi
 create

Example output:

 Created target iqn.2003-01.org.linux-iscsi.storage.x8664:sn.27a43c7db8d8.
 Created TPG 1.
 Global pref auto_add_default_portal=true
 Created default portal listening on all IPs (0.0.0.0), port 3260.

We now create our iSCSI target with IQN iqn.2003-01.org.linux-iscsi.storage.x8664:sn.27a43c7db8d8, listening on all network addresses configured to the OS on port 3260.

List the current configuration again:

 cd /
 ls

Example output:

 o- / .................................................................. [...]
   o- backstores ....................................................... [...]
   | o- block ........................................... [Storage Objects: 1]
   | | o- cp4s_iscsi_pv  [/dev/mapper/cp4s_storage-cp4s_lv (4.0TiB) write-thru deactivated]
   | |   o- alua ............................................ [ALUA Groups: 1]
   | |     o- default_tg_pt_gp ................ [ALUA state: Active/optimized]
   | o- fileio .......................................... [Storage Objects: 0]
   | o- pscsi ........................................... [Storage Objects: 0]
   | o- ramdisk ......................................... [Storage Objects: 0]
   o- iscsi ..................................................... [Targets: 1]
   | o- iqn.2003-01.org.linux-iscsi.storage.x8664:sn.27a43c7db8d8 .. [TPGs: 1]
   |   o- tpg1 ........................................ [no-gen-acls, no-auth]
   |     o- acls ................................................... [ACLs: 0]
   |     o- luns ................................................... [LUNs: 0]
   |     o- portals ............................................. [Portals: 1]
   |       o- 0.0.0.0:3260 .............................................. [OK]
   o- loopback .................................................. [Targets: 0]

Create LUN:

 cd /iscsi/iqn.2003-01.org.linux-iscsi.storage.x8664:sn.27a43c7db8d8/tpg1/luns
 create /backstores/block/cp4s_iscsi_pv

Example output:

 Created LUN 0.

List again for verification:

 cd /
 ls

Example output:

 o- / .................................................................. [...]
   o- backstores ....................................................... [...]
   | o- block ........................................... [Storage Objects: 1]
   | | o- cp4s_iscsi_pv  [/dev/mapper/cp4s_storage-cp4s_lv (4.0TiB) write-thru activated]
   | |   o- alua ............................................ [ALUA Groups: 1]
   | |     o- default_tg_pt_gp ................ [ALUA state: Active/optimized]
   | o- fileio .......................................... [Storage Objects: 0]
   | o- pscsi ........................................... [Storage Objects: 0]
   | o- ramdisk ......................................... [Storage Objects: 0]
   o- iscsi ..................................................... [Targets: 1]
   | o- iqn.2003-01.org.linux-iscsi.storage.x8664:sn.27a43c7db8d8 .. [TPGs: 1]
   |   o- tpg1 ........................................ [no-gen-acls, no-auth]
   |     o- acls ................................................... [ACLs: 0]
   |     o- luns ................................................... [LUNs: 1]
   |     | o- lun0  [block/cp4s_iscsi_pv (/dev/mapper/cp4s_storage-cp4s_lv) (default_tg_pt_gp)]
   |     o- portals ............................................. [Portals: 1]
   |       o- 0.0.0.0:3260 .............................................. [OK]
   o- loopback .................................................. [Targets: 0]

Create an IQN for our OCP cluster, can be anything but must be unique. I'll use the following IQN pattern:
```
 iqn.<current year>-<current-month>.<cluster domain>:ocp-cluster.<ocp cluster id>
```
My final IQN:
- Cluster: iqn.2020-05.cp4s.sechu.ibm:ocp.cluster.ef902b98-e4f0-4c81-9f24-d800d75b5dca
Connect to all of your nodes (masters and workers) using SSH and gather the IQN addresses by executing the following command:
```
 cat /etc/iscsi/initiatorname.iscsi
```
My nodes:
- Master 1: iqn.1994-05.com.redhat:3693ca3df126
- Master 2: iqn.1994-05.com.redhat:363217bb5538
- Master 3: iqn.1994-05.com.redhat:766a4711d4b6
- Worker 1: iqn.1994-05.com.redhat:c8112cd26213
- Worker 2: iqn.1994-05.com.redhat:413af38cf8b0
- Worker 3: iqn.1994-05.com.redhat:7c3fc3a53d4

Create ACL for all the initiator IQN names on the target :

 cd /iscsi/iqn.2003-01.org.linux-iscsi.storage.x8664:sn.27a43c7db8d8/tpg1/acls

 create wwn=iqn.2020-05.cp4s.sechu.ibm:ocp.cluster.ef902b98-e4f0-4c81-9f24-d800d75b5dca
 create wwn=iqn.1994-05.com.redhat:3693ca3df126
 create wwn=iqn.1994-05.com.redhat:363217bb5538
 create wwn=iqn.1994-05.com.redhat:766a4711d4b6
 create wwn=iqn.1994-05.com.redhat:c8112cd26213
 create wwn=iqn.1994-05.com.redhat:413af38cf8b0
 create wwn=iqn.1994-05.com.redhat:7c3fc3a53d4

Example output:

 Created Node ACL for iqn.2020-05.cp4s.sechu.ibm:ocp.cluster.ef902b98-e4f0-4c81-9f24-d800d75b5dca
 Created mapped LUN 0.

Check final configuration:

 cd / 
 ls

Example output:

 o- / ................................................................................... [...]
   o- backstores ........................................................................ [...]
   | o- block ............................................................ [Storage Objects: 1]
   | | o- cp4s_iscsi_pv ...... [/dev/mapper/cp4s_storage-cp4s_lv (4.0TiB) write-thru activated]
   | |   o- alua ............................................................. [ALUA Groups: 1]
   | |     o- default_tg_pt_gp ................................. [ALUA state: Active/optimized]
   | o- fileio ........................................................... [Storage Objects: 0]
   | o- pscsi ............................................................ [Storage Objects: 0]
   | o- ramdisk .......................................................... [Storage Objects: 0]
   o- iscsi ...................................................................... [Targets: 1]
   | o- iqn.2003-01.org.linux-iscsi.storage.x8664:sn.27a43c7db8d8 ................... [TPGs: 1]
   |   o- tpg1 ......................................................... [no-gen-acls, no-auth]
   |     o- acls .................................................................... [ACLs: 7]
   |     | o- iqn.1994-05.com.redhat:363217bb5538 ............................ [Mapped LUNs: 1]
   |     | | o- mapped_lun0 ................................... [lun0 block/cp4s_iscsi_pv (rw)]
   |     | o- iqn.1994-05.com.redhat:3693ca3df126 ............................ [Mapped LUNs: 1]
   |     | | o- mapped_lun0 ................................... [lun0 block/cp4s_iscsi_pv (rw)]
   |     | o- iqn.1994-05.com.redhat:413af38cf8b0 ............................ [Mapped LUNs: 1]
   |     | | o- mapped_lun0 ................................... [lun0 block/cp4s_iscsi_pv (rw)]
   |     | o- iqn.1994-05.com.redhat:766a4711d4b6 ............................ [Mapped LUNs: 1]
   |     | | o- mapped_lun0 ................................... [lun0 block/cp4s_iscsi_pv (rw)]
   |     | o- iqn.1994-05.com.redhat:7c3fc3a53d4 ............................. [Mapped LUNs: 1]
   |     | | o- mapped_lun0 ................................... [lun0 block/cp4s_iscsi_pv (rw)]
   |     | o- iqn.1994-05.com.redhat:c8112cd26213 ............................ [Mapped LUNs: 1]
   |     | | o- mapped_lun0 ................................... [lun0 block/cp4s_iscsi_pv (rw)]
   |     | o- iqn.2020-05.cp4s.sechu.ibm:ocp.cluster.ef902b98-e4f0-4c81-9f24-d800d75b5dca  [Mapped 
   LUNs: 1]
   |     |   o- mapped_lun0 ................................... [lun0 block/cp4s_iscsi_pv (rw)]
   |     o- luns .................................................................... [LUNs: 1]
   |     | o- lun0  [block/cp4s_iscsi_pv (/dev/mapper/cp4s_storage-cp4s_lv) (default_tg_pt_gp)]
   |     o- portals .............................................................. [Portals: 1]
   |       o- 0.0.0.0:3260 ............................................................... [OK]
   o- loopback ................................................................... [Targets: 0]

Save configuration and exit:
```
 saveconfig
 exit
```

Configure OCP↑

Create the iSCSI configuration yaml file on the machine you will access the OCP console:

 apiVersion: v1
 kind: PersistentVolume
 metadata:
   name: iscsi-pv
 spec:
   capacity:
     storage: 1Ti
   accessModes:
     - ReadWriteOnce
   persistentVolumeReclaimPolicy: Retain
   iscsi:
     iqn: 'iqn.2003-01.org.linux-iscsi.cp4s-storage.x8664:sn.a9eaa1043169'
     lun: 0
     targetPortal: '10.109.200.10:3260'
     fsType: ext4
     readOnly: false
     initiatorName: 'iqn.2020-05.cp4s.sechu.ibm:ocp.cluster.ef902b98-e4f0-4c81-9f24-d800d75b5dca`

Where:

iqn: is the target server's iQN
initiatorName: is the customized cluster iQN

Sign in to the OCP console with admin privileges.
Navigate to Storage - Persistence Volumes.
Click the Create Persistent Volume button.
Drag and drop the config file to the editor.
Save the configuration.