Ok so its been a while,
Ive been very busy with work mainly getting our new HP bladesystem networked correctly with VMware VSphere aka ESX4/ESXi4.
We brought the kit with two HP Flex-10 modules using the CX4 10 gig ethernet ports on each switch. These go back to the Cisco catalyst 6509 core switch.
The problem i came across is that the two Flex-10 switches do NOT support link aggregation/etherchannel if your using the CX-4 10gig ports. This is because the cross connect on the blade system does not support it yet. The HP virtual connect software may do in the future but not at the moment. VSphere4 does not support virtual connect active/active configuration when using “distributed switching”, there are all kinds of gotchyas with this new hardware and software.
So, enter many many weeks of messing about trying to get the correct configuration. Basically the only way to have network failover is to do it in an active / standby configuration, this means that only one of the 10gig links is utlized at any one time. This is because the HP Virtual Connect doesn’t support aggregation over two Flex-10 switches using CX4. Heres what i came up with:
Configure the Cisco core end for simple trunking on both CX4 10 gig ports. Make sure trunkfast is enabled so if one of the links fail the port will transition through spanning-tree quickly.
description CX-4-10G-HP-Virtual-Connect-#1
switchport
switchport trunk encapsulation dot1q
switchport mode trunk
spanning-tree portfast trunk
Ok so now that you’ve got all your VLAN’s trunking down to the virtual connect. You need to configure the HP virtual connect with an active / standby configuration:Create shared uplink sets for your vlans for both links. One links connect mode needs to be “Auto” and the other needs to be configured as “Standby”.

Ok so now you have the virtual connect shared uplink the same tagged vlans on each side and one has auto mode the other has standby. Notice how “Smart Link” is ticked for all the tagged vlan’s – make sure you do this. HP Smart Link tells the down stream switch that an uplink connection has been lost. This is essential for link status detection in ESXi4.
Its time to configure the distributed switch. Once youve setup your distributed switch and done the correct adapter configuration in your port groups. The teaming and failover settings need to be configured as follows:
Route based on original port ID.
Link Status only
Notify Switches YES
Failover YES
As shown in the following graphic:

Make sure you match up your virutal adapters correctly – in the above MGMT port group dvuplink1 one correspnds to a virtual nic connected to HP shared uplink #1. Yep dvuplink2 corresponds to the HP shared uplink #2 which is the standby shared uplink set. So you can see this MGMT port group now has failover.
I know its kind of a waste to have two 10gig Ethernet links and only use one of them. But we are forced to do this until HP virtual connect supports the aggregation between two flex-10 switches / cross connect. Also the features of VMware VSphere distributed switching is just too good to pass up so I’ve configured the best i can to get the most benefit. Distributed switching means we don’t have to configure the v switches again and again with every host we add to the ESX/blade cluster.