• We’re currently investigating an issue related to the forum theme and styling that is impacting page layout and visual formatting. The problem has been identified, and we are actively working on a resolution. There is no impact to user data or functionality, this is strictly a front-end display issue. We’ll post an update once the fix has been deployed. Thanks for your patience while we get this sorted.

Need help with NIC/Switch teaming/trunking

D

dclive

Elite Member
I'm a bit new to ESXi, and very new to trunking / teaming in networking.

http://blog.scottlowe.org/2008...king-with-hp-procurve/ based on this, it seems like I can buy an HP Procurve 1800 8G and set it up (with a few Intel gige NICs in an ESXi box) so that I can team the NICs together for both better incoming and better outgoing speeds.

This is of interest for HA scenarios and also for iSCSI speed, as I may be tied to two different iSCSI targets.

Can anyone lend a hand as far as how one would go about setting up the ESXi side and how one would go about setting up the HP Procurve switch side? It seems like the instructions say turn on trunking on the Procurve 1800, but the minute I do that I lose all connectivity with the rest of the networking (when I enable trunking on the 2 ports that attach the ESXi box to the Procurve switch). How does this work?

2 Intel GigE NICs have a CAT6 each going to the HP Procurve 1800-8G
The 1800-8G has various other devices attached to it - an iSCSI 'SAN', an uplink to the rest of the network....

Again, the moment I trunk ports 7&8, the two coming in from the ESXi box, all VMs on the ESXi box fall of the network. What else is required?

The moment I remove the trunks and put those two ports back to "Not a Trunk Member", all VMs show up on the network and I can manage the ESXi host (and ping it) again.
 
Force the switch ports to 802.1q trunk, same on the ESX side. Worry about getting a single trunk link working for now, we can do the teaming later.

Trunking introduces the concept of a native vlan. This is the vlan that is sent and received on the trunk with NO vlan information in the trunk header - there isn't a trunk header, just a normal frame. So both sides have to be set to the same native vlan. For now make sure this vlan is the IP network of your server, for now. You can add vlans to the trunk later on the switch configuration and ESX, but you have to get that working before you can move on.

For example say your server is in vlan 10 and without trunking this is working just fine. When you do the switch and ESX configuration on the link to make it a trunk set the native vlan to 10 and you should maintain reachability. The concept of what the native vlan is is very important, get that going first. A common problem is a native vlan mismatch where one side of the link the native vlan is 50 and the other side is 60. So on the 50 native device it will send it without a trunk header, the other side native 60 picks it up and puts it in vlan 60 - when it really needs to be in vlan 50. In this scenario both vlans 50 and 60 are broken across the trunk.
 
You're going to have to start at a _much_ more basic level than that. Sorry; I've no networking background.

Let's say I've just gotten the Procurve 1800-8G. I turned on jumbo frames. I have 2 NICs (an ESX box) plugged into it, plus an openfiler iSCSI san, plus an uplink to the rest of the network's switches.

Now what? Make a trunk for the 2 ESX NICs? But if I do that I immediately lose connectivity to the ESX box....??
 
It sounds like you need some one to set this up for you then. Esp if this is a business. Anyway

For ESX(i) you should have 6 (or more) nic ports for HA (since you said iSCSI). 2 for HA control / service 2 for HA iSCSI and 2 HA clients. iSCSI should *never* be on the user lans due to the typically custom configuration of the switches to handle jumbo frames + flow control. For high security you should not have control / vmotion / service on the machines LANs either. Also iSCSI does not use encryption for performance reasons and passwords are often plaintext if enabled. If you put it on the client lans, All Joe script kiddie in the office needs to do is download the initiator off MS's site and connect to your LUNs.

You start by assigning 1 lan port to service.
Attach 2 of the physical ports on the nics to the vSwitch.
Those 2 ports you assign set up for 802.1q and follow what Spidey said.
configure 2 ports on the switch to do 802.1q. I don't do procurve yet so I can't help you there.

Depending on whose rules you follow, you can assign the trunks native vlan all to the same vlan or to their own. Cisco recommends that each trunk have a different vlan for each trunk for higher security but take that for what you will. I tend to use different ones.

I will leave teaming to spidey for the same reason (no procurve knowledge, I can do it on Cisco though.)

Please note that true HA is config intensive and also involves a lot of gear. Esp with iSCSI in the mix. ESX(i) is an all your eggs in one basket situation since you have one machine handling several. A failed network link can down all of the machines at once. So you tend to span gear and have special configs for each type of data. iSCSI you tend to configure the switch for flow control and jumboframes, for clients you are not going to see the flow control as often etc. iSCSI you will also tend to span switches, so your 2 "iSCSI" ports on the ESX server will connect to 1 port on each switch, same with the SAN devices. Most SAN devices with their own controllers will have at least dual controllers each with dual lan connections but you mentioned openfiler so I assume this is home brew unit which is not this advanced.

Good Luck.

 
I need instruction so I can learn. It's not a business.

I have 2 NICs on ESXi. That's it. Now, given that, and given it will all go on one interface, and given that the Procurve uses trunking, can you help with next steps? Did you take a glance at Scott's article? I'm not sure exactly what Spidey said so I need a bit of clarification. As I've said, when I enable trunking for the two ports in question, I lose all network connectivity.

Can you assist?
 
I was editing my post while you posted.

In that case, I would highly recommend that you pop a cheap (supported) nic in the machine to use as a service nic. It will save you tons of frustration.

Ok 802.1q is a trunking standard

http://en.wikipedia.org/wiki/IEEE_802.1Q

Native VLAN

Clause 9 of the standard defines the encapsulation protocol used to multiplex VLANs over a single link, and introduces the concept of a native VLAN. Frames belonging to the native VLAN are not modified when sent over the trunk. Conversely, if an untagged frame is received on a trunk port, the frame is placed into the VLAN that is native to this port. This concept has been introduced to ensure interoperability with older and low-cost devices that do not understand 802.1Q.

For example, if an 802.1Q port has VLANs 2, 3 and 4 assigned to it with VLAN 2 being the Native VLAN, frames on VLAN 2 that egress (exit) the aforementioned port are not given an 802.1Q header (ie., they are plain Ethernet frames). Frames which ingress (enter) this port and have no 802.1Q header are put into VLAN 2. Behaviour of traffic relating to VLANs 3 & 4 is as to be expected.

What that says is for each 802.1q trunk there is always 1 vlan that is the native lan. Both sides of the trunk must have this set to the same number. So if you have port g0/24 trunking to g1/24 you must:

Set the switchports to trunk mode
Set the encapsulation to 802.1q (not required on all devices but some devices support other trunking standards.)
Set the native vlan (configed based in the switchport config)
Set the allowed vlans (good practice as the defaults can vary from device to device)

on both ports before they will connect. Cisco has an IOS command to show vlan inconsistencies that will display if the trunk came up or is failing and typically will show you why. I would assume that the procurve does also.

--edit--

This may help: ... you beat me to it dclive.

http://blog.scottlowe.org/2006...ing-and-vlan-trunking/
 
Is there anyone familiar with the HP Procurve side of this? The original post has Procurve-specific ESXi directions, but I need a little bit of interpretation assistance, because all this mention of 802.1q/etc. doesn't relate to anything I see in the Procurve setup, yet this Procurve can do trunking and can do link aggregation, which is all that I am looking for.

The basic issue is that when I do step one in your list (set switchports to trunk mode) I lose all connectivity.

2 is the max NICs I can put in the machine. 🙂
 
I am flipping through the manual for the 1800-8g and I think I can understand why you are confused. In the manual they call a trunk what I would call an aggregate group. Your switch does 802.1q so don't worry about the name.

This is from looking at the manual so I could be wrong. Sorry.

It looks like you have to create a vlan first. Then click modify and assign ports to the vlan. The PVID would be the "default" vlan.

As for teaming it looks like you go in to the "trunks" and assign ports to the trunk group and then use the "T1 / T2 etc as the "ports" that are assigned to vlans.

You need to match the settings on both sides of the links will not come up.


-----
The basic issue is that when I do step one in your list (set switchports to trunk mode) I lose all connectivity.
-----

Yes this happens because both sides are not talking the same language. I would try to set up a trunk on just one port at first. Assign your service connection on the ESXi server to the other nic and get the trunk functioning on 1 nic port first. Then move to aggergate links.
 
You must log in or register to reply here.

TRENDING THREADS

Back
Top