ESX server build - fastest 1U possible

[SnOop005]

I will get switches that support LAG, not a problem, but I still have an issue with this concept:
the MD3000i is housing 5 VMs, one of them is a file server, a person drags a file from the file server to his local machine (1gb file).
At the same time a person sends a 40 page full color print job to a color printer on another VM that's a print server. That's 1.3gb worth of data pouring in to the printer.
and at the same time 45 people are working in their Terminal Server accounts and are working w/ office apps....

what happens?! There's a crunch at that point, right? or is iSCSI so good, that it's still almost as fast as internal storage?

this is what I don't get. all of the above will going over 2 1gig ethernet links?!?!?!?!?

If this is your everyday environment you'll be fine with 2 1GBe link.

 

[jordanl17]

sorry, to be a pain...

I know we are a very small company when it comes to severs, but are you saying in the senario above I'd experinece virtually no lag/slowdown?

I need to get a book on VMware/San

Thanks

 

[imagoon]

There's a couple of issues that you need to be aware of. 1. In order to bond the gigabit ethernet on the MD3000i tofully use the 2gb you will need a switch that supports LAG (802.3AD). 2. With 2GBe your maximum sustained transfer rate is about 250mb/sec which I think 15x 15k SAS drive will easily surpass which brings up another question, will any of your servers be doing intensive I/O that requires 15k drives? You're giving up usable space for speed, at the same time you need to consider scalability. You have total of 9tb of storage, if you decide to go with a RAID10 setup you will end up with half of that amount. 4.5 TB of usable space may be enough for today but also consider the possbility that one day you're network will outgrow 4.5TB the cost and down time involved in expanding or replacing storage.

Uh. No.

I have an MD3000i and none of the ports are "LAG" the MD3000i doesn't even support the protocol. I also have all 4 cables running and can push the disks past a single 2 gig connection. There was another thread where I even showed HDtach screens showing that when configured properly the unit will exceed a single link on a single LUN. Last I checked 165MB/s is > 125MB 1 gig speed limit. No LAG groups anywhere in the setup.

 

[jordanl17]

So, basically, your saying I should go with PS6000 (equalogic) w/ 4gbe connections?
I just called Dell and I'm going to download a tool that will monitor disk IOPs and usage, should I trust that tool?

Uh. No.

I have an MD3000i and none of the ports are "LAG" the MD3000i doesn't even support the protocol. I also have all 4 cables running and can push the disks past a single 2 gig connection. There was another thread where I even showed HDtach screens showing that when configured properly the unit will exceed a single link on a single LUN. Last I checked 165MB/s is > 125MB 1 gig speed limit. No LAG groups anywhere in the setup.

 

[imagoon]

Imagoon,

What are your thoughts on a used/refurb MD3000i, w/
Hard Drives: 15 x Dell 600GB 15K SAS Hot-Plug Hard Drives
Controllers: 2 x Dual-Port Gigabit Ethernet Controllers

It can be had for about 12,000 that is doable for me, as I'm already about to pay 6,000 for das. one big question for me. I'm having issues believing that my VMs will operate as fast over 2gb worth of ethernet. What kind of performance hit will I take.. especially if I end up with 5 VMs on the iscsi box.... I just don't think it will run as fast as direct storage.
am I wrong?

I must be wrong, everyone is doing the iscsi thing. is 2gb enough, does the MD3000i use all 4 ports, and fall back to 2 ports if a controller fails?

The key thing is to configure it properly. (Dell will help.) #1 You *need* 2 switches with at least flow control. Jumbo frames supported to 9218 is a bonus. You will create 2 lans (2 ip ranges) place port 1 of controllers 1 & 2 on one switch, port 2 on CTL 1 and 2 on the other. You will then at least run 2 cables to each ESXi box.

IE (ips are random, but would work)
CTL1:
IP P1: 192.168.171.101
IP P2: 192.168.172.101

CTL2:
IP P1: 192.168.171.102
IP P2: 192.168.172.102

ESX machine (1):

IP P1: 192.168.171.200
IP P2: 192.168.172.200

ESX machine (2):

IP P1: 192.168.171.201
IP P2: 192.168.172.201

You have 2 separate switches, one for each subnet (171 and 172). You do not interconnect them.

Configure the iSCSI initiator on the 2 ESXi boxes and point them at the 3000i. It will see them try to connect and add them to the list of "quick pick" targets to make the configuration easier.

In MD3000i
Create the hosts.
Create the hostgroup.
Add the hosts to the host group.
Create a LUN, add the LUN to the hostgroup.

The 2 ESXi machines will see the LUN after you click refresh.
Create the VMFS however you want.

Now the key thing -> you should see 4 connections appear for each LUN you add. Configure the group for round robin. From the point on that lun will round robin the connections, allowing you to exceed the 1 link limit (easily I may add).

Don't take this the step by step guide. I did leave a few out. But should give you the idea.

 

[imagoon]

So, basically, your saying I should go with PS6000 (equalogic) w/ 4gbe connections?
I just called Dell and I'm going to download a tool that will monitor disk IOPs and usage, should I trust that tool?

Odds are the MD3000i you quoted would easily handle the load. The key things are 1) Flow control 2) "iSCSI support" which just means enough memory in the switch to be able to actually buffer the load. 3) Jumbo frames (9218 MTU). Might be 9216... need to check. It is easy to see as it is a common number.

Those are pretty much in order of importance. 1 and 2 are almost equal though. No flow control will wreck performance. Not enough buffer will wreck performance.

#3 just lowers the cpu load for the interfaces.

The ones I have handle 165meg a second on to 5 10k disks with no issue. I also have multiple disk groups in it. My CAD plotters are hosted on a printer server in the same cluster that hosts the office file servers. Most of the time the ports rarely go above 60% sustained all day. The backups are harsher than "the day life." The unit is "Active:active" and both controllers will handle data for each LUN.

It is not as "strong" as a 4gig fiber unit but it does fine small to mid business.

 

[jordanl17]

Odds are the MD3000i you quoted would easily handle the load. The key things are 1) Flow control 2) "iSCSI support" which just means enough memory in the switch to be able to actually buffer the load. 3) Jumbo frames (9218 MTU). Might be 9216... need to check. It is easy to see as it is a common number.

Those are pretty much in order of importance. 1 and 2 are almost equal though. No flow control will wreck performance. Not enough buffer will wreck performance.

#3 just lowers the cpu load for the interfaces.

The ones I have handle 165meg a second on to 5 10k disks with no issue. I also have multiple disk groups in it. My CAD plotters are hosted on a printer server in the same cluster that hosts the office file servers. Most of the time the ports rarely go above 60% sustained all day. The backups are harsher than "the day life." The unit is "Active:active" and both controllers will handle data for each LUN.

It is not as "strong" as a 4gig fiber unit but it does fine small to mid business.

Ok, I've come a long way on this project so far.

Can you recommend a good switch (I'd get 2 of course). I already have lots of Cisco switches, but I've heard that they aren't that good for iSCSI.

Thoughts?

 

[imagoon]

For the cheap side, Dell 6224 switches seem to work fine. I have a coworker that uses 5224's just fine. You have the advantage that if you call Dell support, they will be more helpful. When it comes the Cisco and iSCSI I would need to defer to someone like spidey. I use them for the LAN and WAN environments but I never researched iSCSI on cisco.