How-to: Configure an ESX Resource to be Passthrough

Last updated
Save as PDF

How to configure a resource to be passthrough in ESX and why does it matter?

Configuring a resource as a passthrough device in ESX allows you to bypass the virtualization layer in ESX, allowing a VM direct access to the hardware. This matters when you want to obtain true characteristics of that hardware and not have it modified by the ESX hypervisor (virtualized). Results from using hardware resources configured in passthrough mode will be the same as that of using a bare metal system.

Configuring hardware on an ESX system assumes that you have enough of other resources such as CPU to run your workload, such as in a performance test. However, best use of configuring passthrough devices would be for compliance testing especially in case such as Support's lab where resources are always shared. One example is to allow dedicated use of a HBA with VMs in order to demonstrate how various multipathing software works with Pure. This can assist in reproducing various conditions and scenarios Pure luns behave with various multipathing software. Further, having a VM use HBAs directly saves a lot of time from configuring a bare metal system to do the same with multibooting since everything can be set within ESX.

In this following example, an Emulex HBA has been configured in passthrough mode on "marioesx.dev.purestorage.com". On the Pure arrays side, 3 volumes have been made and connected to the HBA's WWPN. The HBA will then appear as "PCI device" on ESX and can be attached to any one VM at a time. The VMs can be made of various OSes that run within ESX. This then allows one to switch between OSes with the same HBA and luns to observe each multipather behavior.

On problemchild I have configured the following

root@problemchild-ct0:/home/os76# purehost list
Name WWN IQN Host Group
emulex-fc-hba 10:00:00:90:FA:07:20:BC - -
              10:00:00:90:FA:07:20:BD -

root@problemchild-ct0:/home/os76# purevol list --connect
Name Size LUN Host Group Host
emulex-fc-vol1 1T 1 - emulex-fc-hba
emulex-fc-vol2 1T 2 - emulex-fc-hba
emulex-fc-vol3 1T 3 - emulex-fc-hba 

root@problemchild-ct0:/home/os76# purevol list
Name Size Source Created Serial
emulex-fc-vol1 1T - 2013-11-11 12:22:40 PST BF28DA2EE4CF586D00010002
emulex-fc-vol2 1T - 2013-11-11 12:22:44 PST BF28DA2EE4CF586D00010003
emulex-fc-vol3 1T - 2013-11-11 12:22:47 PST BF28DA2EE4CF586D00010004

HOW TO CONFIGURE THE HBA IN PASSTHROUGH MODE

Steps

* Open vSphere client and connect to the ESX server you want to configure, either directly or through vCenter.

* Click on the ESX server you want to configure then click on the "Configuration" tab.

* Click on the Advanced Settings link and you should see the following screen

(Note - this ESX server already has the Emulex HBA configured in passthrough mode so its showing here)

* Click on the Edit link just above the window on the right.

* Locate the physical hardware you want to remove virtualization on. Put a checkmark in all the boxes that you want to devirtualize. In this example, the Emulex HBA has checks in all boxes including the parent, which means take the whole adapter.

After this is done, the system will request for a reboot after clicking on OK. Make sure you have prepared the ESX box to be rebooted by pausing all VMs and alerting others who use it.

How to configure the HBA and assign it to a VM.

Since the HBA is now configured in passthrough, it is now dedicated to be used by a VM. This is the difference between a passthrough and virtualized device. The ESX server makes the device a shared device in virtualization mode between VMs.

* Select a VM and right click on it to Edit settings.

* Click on Add to add a resource to the VM.

* Select PCI Device. You'll need to repeat the Add step if you want to add more than one port. In this demo, we want to add both HBA ports

because we want to get MPIO to work correctly with two paths. Repeat this step to add the other port from the screenshot above (03:00.0 and 03:00.1).

The repeat is needed because ESX only allows us to add one port at a time.

* Click on Next until you come to Finish to complete the configuration of the VM. Be sure to repeat the steps to add the second HBA port.

The following screen should be the result after adding both HBA ports to the VM

Click on OK to save the VM's configuration. In the previous screenshots, I moved the PCI devices from one VM to another. This is how the HBA ports can be reassigned to another VM.

TESTING MPIO IN LINUX TO SEE THE PATHS TO PURE VOLS

On my RHEL64 VM, I can now see multipathd show multiple paths for each lun.

[root@rhel64 ~]# multipath -l
3624a9370bf28da2ee4cf586d00010004 dm-2 PURE,FlashArray
size=1.0T features='0' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=0 status=active
 |- 3:0:1:3 sdg 8:96 active undef running
 |- 3:0:0:3 sdd 8:48 active undef running
 |- 3:0:2:3 sdj 8:144 active undef running
 |- 3:0:3:3 sdm 8:192 active undef running
 |- 3:0:4:3 sdp 8:240 active undef running
 |- 3:0:5:3 sds 65:32 active undef running
 |- 3:0:6:3 sdv 65:80 active undef running
 |- 3:0:7:3 sdy 65:128 active undef running
 |- 4:0:0:3 sdab 65:176 active undef running
 |- 4:0:1:3 sdae 65:224 active undef running
 |- 4:0:2:3 sdah 66:16 active undef running
 |- 4:0:3:3 sdak 66:64 active undef running
 |- 4:0:4:3 sdan 66:112 active undef running
 |- 4:0:5:3 sdaq 66:160 active undef running
 |- 4:0:6:3 sdat 66:208 active undef running
 `- 4:0:7:3 sdaw 67:0 active undef running
3624a9370bf28da2ee4cf586d00010003 dm-1 PURE,FlashArray
size=1.0T features='0' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=0 status=active
 |- 3:0:0:2 sdc 8:32 active undef running
 |- 3:0:2:2 sdi 8:128 active undef running
 |- 3:0:3:2 sdl 8:176 active undef running
 |- 3:0:1:2 sdf 8:80 active undef running
 |- 3:0:4:2 sdo 8:224 active undef running
 |- 3:0:5:2 sdr 65:16 active undef running
 |- 3:0:6:2 sdu 65:64 active undef running
 |- 3:0:7:2 sdx 65:112 active undef running
 |- 4:0:0:2 sdaa 65:160 active undef running
 |- 4:0:1:2 sdad 65:208 active undef running
 |- 4:0:2:2 sdag 66:0 active undef running
 |- 4:0:3:2 sdaj 66:48 active undef running
 |- 4:0:4:2 sdam 66:96 active undef running
 |- 4:0:5:2 sdap 66:144 active undef running
 |- 4:0:6:2 sdas 66:192 active undef running
 `- 4:0:7:2 sdav 66:240 active undef running
3624a9370bf28da2ee4cf586d00010002 dm-0 PURE,FlashArray
size=1.0T features='0' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=0 status=active
 |- 3:0:1:1 sde 8:64 active undef running
 |- 3:0:0:1 sdb 8:16 active undef running
 |- 3:0:3:1 sdk 8:160 active undef running
 |- 3:0:2:1 sdh 8:112 active undef running
 |- 3:0:4:1 sdn 8:208 active undef running
 |- 3:0:5:1 sdq 65:0 active undef running
 |- 3:0:6:1 sdt 65:48 active undef running
 |- 3:0:7:1 sdw 65:96 active undef running
 |- 4:0:0:1 sdz 65:144 active undef running
 |- 4:0:1:1 sdac 65:192 active undef running
 |- 4:0:2:1 sdaf 65:240 active undef running
 |- 4:0:3:1 sdai 66:32 active undef running
 |- 4:0:4:1 sdal 66:80 active undef running
 |- 4:0:5:1 sdao 66:128 active undef running
 |- 4:0:6:1 sdar 66:176 active undef running
 `- 4:0:7:1 sdau 66:224 active undef running

HOW DO I KNOW THESE ARE THE CORRECT LUNS ASSIGNED TO MY HBA ADAPTER?

The purevol list output has serial numbers matching that of the multipath -l output (in bold) above.

WHY DO I SEE SO MANY PATHS? I THOUGHT THERES ONLY A FEW.

In this lab, the Brocade 300 switch has an open default zone configured. This isn't the best practice however it makes administration simpler on a small FC switch by not having to change the zoning config every time.