ID #1036

COMSTAR Fibre Channel and Client Multipathing

Introduction

This guide describes how to configure COMSTAR fibre channel targets for failover, eliminating single points of failure from the client to the target using redundant fabrics and native multi-pathing functionality. The following diagram depicts the hardware configuration for this setup. Note there is no interconnect between the two fibre fabrics; redundant paths through each to the underlying ZVOL are provided by seperate HBA's which in turn are used to present a single unified volume to the client utilizing multi-pathing technology.

Sample configuration

 

Configure Clustered ZVOL's

Create zvols on the shared storage and then configure them for clustering as described in FAQ 34. How do I configure COMSTAR Fibre Channel targets for fail over?

Configure Multi-pathing on the client

Once the clustered zvols are created and exported via COMSTAR, it is necessary to configure the client side multi-pathing to provide redundancy. The following sub-sections describe multi-pathing configuration for a number of operating systems.

Windows Server 2008 - QLogic HBA's

Install the latest QLogic STOR Miniport driver and SANsurfer FC HBA Manager as appropriate (available from the downloads section at www.qlogic.com).

Prior to enabling Multipath I/O on Windows Server 2008, any COMSTAR Fibre Channel target accessible through the SAN fabric by more than one physical path is displayed as multiple disks in the Storage section of Computer Management. Each disk represents a different physical path to the same Fibre Channel target; therefore the total number of disks represents the total number of physical paths through the SAN fabric to the Fibre Channel target. In the example below there are two physical paths to the same disc shown:

Display of multiple=

 

To install MPIO on Windows Server 2008, start Server Manager and select Multipath I/O from the Add Features wizard. Once Multipath I/O is installed, the MPIO applet will then be added to the Control Panel.

Selecting Windows Server 2008 MPIO features

To add support for COMSTAR Fibre Channel targets on Windows Server 2008, start the MPIO applet from the Control Panel:

Features icon

The Discover Multi-Paths tab should list NEXENTA COMSTAR as an SPC-3 compliant device, click Add and then reboot the server if prompted, the Devices list under the MPIO-ed Devices tab will now show NEXENTA COMSTAR as a supported device.

MPIO Properties

If COMSTAR does not appear in the above list then:

  • On the Nexenta heads ensure all cards are set to target mode.
  • Check all cabling to ensure there are physical paths to the COMSTAR fibre channel targets.
  • If cables are not keyed (i.e. a cable can be plugged into either of the A/B ports on the GBIC) try swapping over the A/B ports.
  • Ensure any fibre switches are not blocking the path due to zoning.

After enabling Multipath I/O on Windows Server 2008, any COMSTAR Fibre Channel target accessible through the SAN fabric by more than one physical path is now displayed as a single disk in the Storage section of Computer Management, irrespective of the total number of physical paths through the SAN fabric to the Fibre Channel target:

Single path 

Finally configure MPIO load balancing policy from the disc properties (in this example we choose fail over only, however, the other available policys will also work):

Load balancing property

Linux - QLogic HBA's

Multipathing in Linux is provided by the device-mapper-multipath package (also known as dm-multipath) - use the distribution specific package manager to check/install the package.

By default, all devices are blacklisted for multipathing; to enable multipathing, comment out the blacklist section in /etc/multipath.conf as follows:

#blacklist {
#        devnode "*"
#}

On failover the default action of the Linux multipather is to fault the drive (because all paths are temporarily lost which is normal behaviour in a non clustered environment). This is avoided by updating the options in /etc/multipath.conf to include the no_path_retry and polling_interval options:

defaults {
polling_interval 5
no_path_retry 12
}

Setting no_path_retry to 12 means that on failure of all of the paths the multipath daemon should retry the path connections 12 times before failing the device. The polling interval defines the number of seconds to wait between each successive retry (so in the above example it will retry for a total of 60 seconds). This configuration provides the following characteristics:

  • On failover all paths are temporarily lost whilst the associated ha volume is brought up on the node it is failing over to.
  • On complete controller failure (rather than node failure) the timeout of 60 seconds will be reached and the device will return an I/O error.

If the failover takes longer that 60 seconds (which can be the case when importing a zpool with a very large number of discs) the no path retry parameter should be increased accordingly.

Having reconfigured the linux client in this way you need to start the multipath daemon by running the following commands:

modprobe dm-multipath
service multipathd start

 

To ensure the multipath daemon starts at bootup, run the following command:

chkconfig multipathd on 

 

Finally, to list multipathed devices, run the command:

multipath -ll

 

The output should be similar to that below, showing all the paths available and that one of them is enabled:

mpath0 (3600144f0ee48850000004c765d440001) dm-2
NEXENTA,COMSTAR
[size=10G][features=0][hwhandler=0][rw]
\_ round-robin 0 [prio=1][enabled]
 \_ 1:0:0:0 sda 8:0   [active][ready]
\_ round-robin 0 [prio=1][enabled]
 \_ 0:0:0:0 sdb 8:16 
[active][ready]
\_ round-robin 0 [prio=1][enabled]
 \_ 1:0:3:0 sdc 8:32 
[active][ready]
\_ round-robin 0 [prio=1][enabled]
 \_ 0:0:3:0 sdd 8:48 
[active][ready]

The device can now be accessed (in the above example) using the path /dev/mapper/mpath0

Tags: -

Related entries:

Last update: 2012-07-17 14:50
Author: Paul Griffiths
Revision: 1.5

{writeDiggMsgTag} {writeFacebookMsgTag} {writePrintMsgTag} {writeSend2FriendMsgTag} {writePDFTag}
{translationForm}
Please rate this FAQ:

Average rating: 4.5 (2 Votes)

completely useless 1 2 3 4 5 most valuable

You cannot comment on this entry