I’m not a big fan of iSCSI – probably a hangover from years of fibre channel experience before ethernet based storage networks became commonplace – and the issue I ran into today didn’t do anything to increase my comfort level. My experience to date is that fibre channel storage systems seem to “just work” whereas iSCSI can be finnicky and temperamental.
A customer had purchased a new HP P2000 G3 iSCSI storage array to provide a shared storage repository for XenServer HA for the blades hosting their XenDesktop farm, in addition to providing a storage location for templates and a handful of test VMs. I configured two ports on each P2000 controller and assigned two NICs from each XenServer blade for iSCSI traffic as per the XenServer multipathing best practice guide which you can find at CTX136354. I enabled multipathing on all hosts in the farm and proceeded to create the storage repository.
After creating the SR it was visible on one host only, and on the rest it showed with a status of “Unplugged” in XenCenter. Watching the SMLog file while trying to “repair” the SR from the GUI, or running “xe pbd-plug uuid=…” from the command line, generated errors similar to the below:
Aug 21, 2013 4:27:14 AM Error: Repairing SR P2000_HA - Internal error: Failure("Storage_access failed with: SR_BACKEND_FAILURE: [ non-zero exit; ; Traceback (most recent call last):\n File \"/opt/xensource/sm/LVMoISCSISR\", line 549, in ?\n SRCommand.run(LVHDoISCSISR, DRIVER_INFO)\n File \"/opt/xensource/sm/SRCommand.py\", line 250, in run\n sr = driver(cmd, cmd.sr_uuid)\n File \"/opt/xensource/sm/SR.py\", line 136, in __init__\n self.load(sr_uuid)\n File \"/opt/xensource/sm/LVMoISCSISR\", line 150, in load\n self.iscsi = self.iscsiSRs\nIndexError: list index out of range\n ]")
Needless to say this didn’t make a lot of sense. Eventually after running out of ideas, some semi-random googling uncovered that the /etc/iscsi/initiatorname.iscsi file was missing (on 10 of 11 servers in the pool!!) and was not recreated by changing the IQN in the XenCenter console. To fix this, I ran the following commands (note the initiator name must be the same as what is set in the XenCenter console)
[root@XenHome ~]# echo InitiatorName=iqn.2011-07.com.xenserver01:10f967a6 > /etc/iscsi/initiatorname.iscsi [root@XenHome ~]# echo InitiatorAlias=XenServer01 >> /etc/iscsi/initiatorname.iscsi [root@XenHome ~]# /etc/init.d/open-iscsi stop [root@XenHome ~]# /etc/init.d/open-iscsi start
To test iSCSI was now operational, I ran the following command (replace the IP address with the address of your iSCSI SAN):
[root@XenHome ~]# iscsiadm -m discovery -t sendtargets -p 192.168.1.10
A list of target LUNs was returned, I was able to successfully “repair” the SR and get on with my day.