FAQ / Creating Cluster-Wide Target Groups for ALUA

NMV is currently unable to create the cluster-wide target groups necessary for allowing client-side multipathing software to take full advantage of COMSTAR Asymmetric Logical Unit Access support provided by the new HA Cluster STMF Proxy service.

To add this functionality to NexentaStor at present, the target groups once created must be fully populated with the remaining target mode ports as per the instructions contained within this guide.

This guide assumes two NexentaStor appliances, nextest3 & nextest4, each with two Fibre Channel HBA ports configured in target mode for COMSTAR Asymmetric Logical Unit Access support.

The target group will need to be created initially on one NexentaStor appliance using NMV, this effectively registers the target group name with NMS on both the local and remote appliance, allowing the remaining target ports be added manually.

When creating the target group, select only a single target port on the local appliance even if multiple target ports exist.

On nextest3, create the target group tg01 containing a single target port as follows:

 

Verify the target group tg01 has also been created on nextest4 containing one of its own target ports as follows:

 

Once the target group has been created and registered with NMS, the remaining target ports need to be manually added to the target group tg01 on both appliances, this procedure requires accessing the underlying Bash shell from within NMC as follows:

nmc@nextest3:/$ option expert_mode=1
nmc@nextest3:/$ !bash

You are about to enter the Unix ("raw") shell and execute low-level Unix command(s). Warning: using low-level Unix
commands is not recommended! Execute? Yes

root@nextest3:/volumes#

With the HA Cluster STMF Proxy service running, the stmfadm list-target –v command will return all Fibre Channel ports in target mode on both the local and remote appliance. The Fibre Channel ports in target mode on the remote appliance are easily identified by the Provider Name of pppt.

The output from the command on nextest3 is as follows:

root@nextest3:/volumes# stmfadm list-target -v

Target: wwn.500110A00018700E
Operational Status: Online
Provider Name : pppt remote target port 1 on nextest4
Alias : -
Protocol : Fibre Channel
Sessions : 0
Target: wwn.500110A00018700C
Operational Status: Online
Provider Name : pppt remote target port 0 on nextest4
Alias : -
Protocol : Fibre Channel
Sessions : 0
Target: wwn.2100001B320F034D
Operational Status: Online
Provider Name : qlt local target port 0 on nextest3
Alias : qlt0,0
Protocol : Fibre Channel
Sessions : 0
Target: wwn.2101001B322F034D
Operational Status: Online
Provider Name : qlt local target port 1 on nextest3
Alias : qlt1,0
Protocol : Fibre Channel
Sessions : 0

To view the current member of the target group tg01 on nextest3 run the following command:

root@nextest3:/volumes# stmfadm list-tg -v
Target Group: tg01
Member: wwn.2100001B320F034D

To allow the remaining target ports to be added to the target group, the target ports currently outside the target group first need to be offlined to allow them to be added to the target group.

On nextest3, the commands are as follows:

stmfadm offline-target wwn.2101001B322F034D
stmfadm offline-target wwn.500110A00018700C
stmfadm offline-target wwn.500110A00018700E

The remaining target ports can then be added into the target group with the following commands:

stmfadm add-tg-member -g tg01 wwn.2101001B322F034D
stmfadm add-tg-member -g tg01 wwn.500110A00018700C
stmfadm add-tg-member -g tg01 wwn.500110A00018700E

Once added, the target ports need to be onlined again to allow them to receive SCSI I/O commands for the Logical Units they serve as follows:

stmfadm online-target wwn.2101001B322F034D
stmfadm online-target wwn.500110A00018700C
stmfadm online-target wwn.500110A00018700E

Access the underlying Bash shell from within NMC on nextest4 as follows:

nmc@nextest4:/$ option expert_mode=1
nmc@nextest4:/$ !bash
You are about to enter the Unix ("raw") shell and execute low-level Unix command(s). Warning: using low-level Unix
commands is not recommended! Execute? Yes
root@nextest4:/volumes#

The corresponding output from the stmfadm list-target –v command on nextest4 is as follows:

root@nextest4:/volumes# stmfadm list-target -v
Target: wwn.2101001B322F034D
Operational Status: Online
Provider Name : pppt remote target port 1 on nextest3
Alias : -
Protocol : Fibre Channel
Sessions : 0
Target: wwn.2100001B320F034D
Operational Status: Online
Provider Name : pppt remote target port 0 on nextest3
Alias : -
Protocol : Fibre Channel
Sessions : 0
Target: wwn.500110A00018700C
Operational Status: Online
Provider Name : qlt local target port 0 on nextest4
Alias : qlt0,0
Protocol : Fibre Channel
Sessions : 0
Target: wwn.500110A00018700E
Operational Status: Online
Provider Name : qlt local target port 1 on nextest4
Alias : qlt1,0
Protocol : Fibre Channel
Sessions : 0

To view the current member of the target group tg01 on nextest4 run the following command:

root@nextest4:/volumes# stmfadm list-tg -v
Target Group: tg01
Member: wwn.500110A00018700C

To allow the remaining target ports to be added to the target group, the target ports currently outside the target group first need to be offlined to allow them to be added to the target group.

On nextest4, the commands are as follows:

stmfadm offline-target wwn.500110A00018700E
stmfadm offline-target wwn.2100001B322F034D
stmfadm offline-target wwn.2101001B322F034D

The remaining target ports can then be added into the target group with the following commands:

stmfadm add-tg-member -g tg01 wwn.500110A00018700E
stmfadm add-tg-member -g tg01 wwn.2100001B322F034D
stmfadm add-tg-member -g tg01 wwn.2101001B322F034D

Once added, the target ports need to be onlined again to allow them to receive SCSI I/O commands for the Logical Units they serve as follows:

stmfadm online-target wwn.500110A00018700E
stmfadm online-target wwn.2100001B322F034D
stmfadm online-target wwn.2101001B322F034D

Once completed, the target group tg01 on both nextest3 & nextest4 will contain all four target ports on both the local and remote appliance as follows:

root@nextest3:/volumes# stmfadm list-tg -v
Target Group: tg01
Member: wwn.2100001B320F034D
Member: wwn.2101001B322F034D
Member: wwn.500110A00018700C
Member: wwn.500110A00018700E

It is now possible within NMV to create mappings to Logical Units using the cluster-wide target group tg01.

Posted in: NexentaStor