• We’re currently investigating an issue related to the forum theme and styling that is impacting page layout and visual formatting. The problem has been identified, and we are actively working on a resolution. There is no impact to user data or functionality, this is strictly a front-end display issue. We’ll post an update once the fix has been deployed. Thanks for your patience while we get this sorted.

ESXI Server

Chess

Golden Member
All:

Trying to determine the best way to go ahead and build a "lab" an for home use my own server farm....

Trying to determine if its easier to buy a complete server from newegg etc or just build something ?



At work, I have plenty of hardware IBM Blades with EMC SAN etc... obviously I cant afford that for my home server farm 🙂


Just curious what route people go...

Thanks in advance !
 
How much performance do you need?

FreeNAS is a popular solution for home SANs, since it's free and happily supports both iSCSI and NFS, which both work well with ESX.

Basically comes down to:

1) get a box to run FreeNAS
2) get another box or boxes to work as hypervisors
3) get a switch that supports VLANs
4) get an old Cisco router

Some people build their own, some people buy used servers for cheap on eBay and build a rack in their basements, whatever. That stuff doesn't matter as much, IMO - it's more important that the software environment match.

(And yes, there may be some benefit to working with all the "enterprise" hardware, but I'm still not convinced it's anything a proper geek couldn't google and figure in a few minutes. Tab A goes into Slot B regardless.)

What's your budget, what are your intended uses, etc.?
 
dave:

Hey, i already have storage... that isnt a problem.. just trying to decide what type of box to build for the hypervisor and creating virtual machines....

I know i could buy a used Dell PE 1950/2950 but i prefer a tower case and put all the drives in there etc....

Havent thought about the budget.... of course the answer the least if possible LOL......

Not worried about vlans, and that stuff either.... just want to setup a nice server farm, i have 4 TB of storage, I can add more if needed that isnt an issue, as drives are dirt cheap.....
 
Having some sort of BMC is a must for an ESXi server in my opinion. You can definitely roll your own with a SuperMicro server mobo, but there are also cheap PowerEdge tower servers on Ebay.

What's your budget like?
 
Having some sort of BMC is a must for an ESXi server in my opinion. You can definitely roll your own with a SuperMicro server mobo, but there are also cheap PowerEdge tower servers on Ebay.

What's your budget like?

I agree with mfenn on the BMC (or an IP KVM). You can setup your lab with enterprise grade equipment on the cheap as long as you don't "need" the latest equipment. I don't have anything against building your own, I'm running 2 Supermicro systems myself right now. But HP Proliant G5's, Poweredge 1950/2650's, and IBM xServer M1's are all very affordable on eBay/Craigslist. Far cheaper than building your own. Shameless plug (sorry if that's not allowed), I've got an IBM x3650 M1 for sale right now with a pair of X5355 Quad Cores and 20Gb of RAM for $300 + shipping.

Here's my thoughts on building home labs with enterprise level gear on the cheap (meaning sub-$1,000 in my book). You're pretty much looking at Xeon 5000 series which "generation" depends on how cheap you want to go. Xeon 50xx - 54xx use DDR2, max 4 Cores, No HT. Xeon 55xx - 56xx use DDR3, max 6 cores, and are available with HT. They are also more efficient meaning they use less power and generate less heat. Performance wise, realistically a pair of 5300/5400 quads is probably more than enough for a home lab. Memory prices are pretty even between them right now, but the DDR2 FB-DIMM's run VERY hot for memory. Even with enterprise level gear you can run into cooling issues.

I recommend sticking with HP when looking at this generation of equipment. For the "first" generation Xeon 5000's, the Poweredge and xSeries don't support the 5400 series Xeon's. As far as it working even though it's not supported, I don't have any experience with those Poweredges, but I can tell you on the xSeries only 1 revision of board supports the 5400 series Xeon's and the system part number is the same so the only way you'll know is by booting into the BIOS and checking the board revision. You're typically going to be looking at $200-$300 plus shipping for this generation.

For the second generation, it's a really close match between HP and Dell (IBM's carry too much of a premium). Typically you'd be looking at either a DL360/380 G5 or Poweredge R610/710. You'll be looking closer to $500-$600 plus shipping for this generation.

HP Pro's: They include their BMC with a dedicated NIC port, it's optional on the Poweredge (shares a NIC port by default). The HP's have their power supplies on the left when looking at the back, like every other piece of hardware I own, whereas the Dell's are on the right. Just an annoyance when doing cable management. HP's internal setup is a bit more refined (IE fans with integrated plugs rather than cables).

Dell Pro's: Slightly cheaper, slightly quieter. I really do like the little LCD on the front, I set it to show power usage on my home server.

If you're set on going with a tower, that limits the options significantly. As somebody who's got tower model Poweredge's and xSeries, I'd stick with rackmount, even if you just stack them on the floor. The towers are too deep to fit under a desk anyways, so why bother.

I've actually been meaning to make a budget server lab thread for a while. mfenn, would you be opposed to me doing that and stickying it? We get people asking about cheap server builds pretty regularly.
 
Last edited:
Sorry guys have been swamped with work and travel etc......


I just really want to setup a lab enviroment, but also run my HTPC, fileshare/store, etc from it.....

I dont mine having a tower, and to be honest I dont really want to buy a powerdge, that would take up room... something small would be ideal... but if that cant happen then so be it 😉
 
You should be able to find a compatible Intel/Supermicro board relatively easily and then whatever case, etc.

Define "small" though haha. Some of the smaller things may not run ESXi - just depends on the hardware.

The main thing is to have a compatible NIC as the installer isn't going to work without one. You can fudge some of the other things a bit.

Having easy access to the console of the actual server can be important - especially if you are doing a lab to mess around with things.
 
I just really want to setup a lab enviroment, but also run my HTPC,

This doesn't make too much sense. ESXi isn't meant to offer rich graphical output from the VMs directly to a display device. You could make something work if you got VT-d capable hardware and a licensed version of vSphere, but that's a lot of money.
 
This doesn't make too much sense. ESXi isn't meant to offer rich graphical output from the VMs directly to a display device. You could make something work if you got VT-d capable hardware and a licensed version of vSphere, but that's a lot of money.

lack of sleep and watching the redskins get bombed last night...

I meant to say a htpc server really is what I want to have.... i have over 4 tb of music and videos and apps i want to be able to file store and share and stream if needed
 
lack of sleep and watching the redskins get bombed last night...

I meant to say a htpc server really is what I want to have.... i have over 4 tb of music and videos and apps i want to be able to file store and share and stream if needed

So you want a server and have a separate HTPC doing video playback?

You could do all of the above in a single box - just set up a server, install desktop Ubuntu with your video playback apps, and then install a hypervisor like VirtualBox.
 
I want to build something similar (perhaps a bit bigger) like this, with 2 ESXi 6 RC servers (so I can learn/test vMotion), an 8 port gigabit switch, and a cheap/homemade SAN (For learning/testing DRS). And I want to spend less than $1,500 for it 🙂

I know that I'm asking a lot there, since you really need at least 8 GB on each server to really test those more advanced features correctly.

Is buying some refurbished Dell PowerEdge or IBM xSeries servers still considered to be the best way to go about this? If so, any recommendations?
 
Last edited:
Yeah some refurb Dell/HP/IBM boxes would be the ideal way if you can find what you want, especially at your price point.

Realize that compatibility for the next release isn't out yet. I'd stay away from some of the older boxes - try for something a little bit newer.

BTW there isn't much to learn or test with vMotion. You don't have to configure much for it to work - just a vmkernel port with vMotion enabled for the most part.

You will need a copy of the vCenter software though - eval mode will have all features unlocked so you can play with it for free for a little while.

If you do get servers, make sure you get the same generation CPU and it will make it easier so you don't have to setup an EVC cluster.

If you want to test FT the CPU's need to be the same generation and ideally same clock speed if possible, Hardware Virtualization support in the BIOS among other things.
 
I want to build something similar (perhaps a bit bigger) like this, with 2 ESXi 6 RC servers (so I can learn/test vMotion), an 8 port gigabit switch, and a cheap/homemade SAN (For learning/testing DRS). And I want to spend less than $1,500 for it 🙂

I know that I'm asking a lot there, since you really need at least 8 GB on each server to really test those more advanced features correctly.

Is buying some refurbished Dell PowerEdge or IBM xSeries servers still considered to be the best way to go about this? If so, any recommendations?

Yeah, make your own thread! :awe:

mfenn
General Hardware Moderator
 
So you want a server and have a separate HTPC doing video playback?

You could do all of the above in a single box - just set up a server, install desktop Ubuntu with your video playback apps, and then install a hypervisor like VirtualBox.

That's certainly true, but if part of the goal is to learn VMware, then installing the server inside of a VM isn't the worst idea.

One thing I haven't seen from the OP is how much he's willing to spend.
 
Watch prices for some of the devices out there. While I am all for buying refurbs, you may end up better off buying a new Dell T110 for $400 than a used 2950/2900 server based on power usage alone.

I saw a 2650 mentioned up there. Just straight up pass on those. They don't run anything current in the HyperV/vSphere world.
 
All:

Trying to determine the best way to go ahead and build a "lab" an for home use my own server farm....

Trying to determine if its easier to buy a complete server from newegg etc or just build something ?



At work, I have plenty of hardware IBM Blades with EMC SAN etc... obviously I cant afford that for my home server farm 🙂


Just curious what route people go...

Thanks in advance !


Ah, well this all depends on a couple things.

1st. How much are you willing to spend? $500? $1000? As cheap as possible?

2nd. Besides building a server farm are you trying to learn anything in particular? Anything about ESXi you're trying to learn? If so going with some used server hardware may be a route you could pursue. Still not particularly cheap.

3rd. What exactly are your goals? You answered some of this with your recent post. Having a smallish box or single tower to host file server, HTPC, and shares.

Addressing the 1st. I didn't go the ESXi route since we use them at work and I've had a bit of hands on experience with them. Plus I get to setup tests before we deploy. (Lucky for me 😀)

For my home lab I bought some consumer level equipment and setup a Hyper-V environment. This allowed me to virtualize between 6-8 systems for about $600. I had some spare hardware I re-used that allowed me to save a few hundred. If I had bought new, would have been about $200 more. I used Hyper-V core and ran Windows 2008 R2, Windows 2012 and Windows 7 machines on top. Each had 1-2 cores depending on use and 4-8gb of memory.

For this I built an FX-8320 with 32gb of memory and used 4 500gb 2.5" 7200rpm drives in a RAID 10 and two 256gb SSD's standalone (wasn't worried about these going out, but wanted the speed benefits from the RAID 10). I also added an additional Intel gigabit network card to the system to separate traffic. The FX-8320 gave me 8 cores and since I didn't need a ton of raw speed per core was a good price. Lastly to save I just used the built in RAID controller on the motherboard. Not ideal by any means and I would only ever use this in a test environment that is not critical, but it works on the cheap and for playing around with.

2nd. ESXi is a bit more picky when it comes to the hardware you use. Consumer level hardware can be used, but do your homework so you aren't left with something that isn't compatible. You can get some server grade hardware for a decent price though. I haven't spec'd out market prices for building recently, but from poking around it'd be around $1000 give or take a bit.

3rd. What kind of HTPC are you running? Software, use, etc? I run a Plex server from a virtual machine that is always running. I use it with a Chromecast to stream movies and shows to my TV. You can do very similar setups with many other pieces of software. The biggest thing is to look at what the needs of the virtual machine are. For me it like cores to convert the video to the needed format. I've settled on 2 for my needs. Streaming to more than one device will obviously increase requirements.


Think about the actual and peak usage of your systems. Couple that with your budget and it'll help you decide where to focus your budget. Generally cores and memory are the top two things people look at closely followed by storage. You don't need the fastest memory or processor speed, but will want a responsive storage system. This can be achieved with a RAID setup (10 is what I prefer) or going with SSD's. SSD's aren't cheap though compared to mechanical when you need space. I find RAID 10 for 7200rpm drives to be very good.
 
Watch prices for some of the devices out there. While I am all for buying refurbs, you may end up better off buying a new Dell T110 for $400 than a used 2950/2900 server based on power usage alone.

I saw a 2650 mentioned up there. Just straight up pass on those. They don't run anything current in the HyperV/vSphere world.

Honestly I would steer clear from any of the 2xxx series servers We just phased out a PowerEdge 2950 at work that had been in production for over 5 years. They are workhorses, but are older and will have very limited support for newer software. A better option would be a T4xx series or something similar. They are towers and not near the workhorses but will be cheaper and have more relevant hardware.
 
Yeah, make your own thread! :awe:

mfenn
General Hardware Moderator

I know... but we already have two open threads for ESXi servers and I didn't want to open a third when these requests seem similar.

So we'll have 3 ESXi server threads then; there's nothing wrong with that. Nobody worries about having six different gaming PC threads on the front page, so you shouldn't worry about having the third ESXi thread on the front page. This thread will get confusing quickly if you start mixing your own, distinct, requirements into this thread and people respond to them. When that happens, signal to noise ratio drops for everybody, and we're all unhappy.

mfenn
General Hardware Moderator
 
Last edited by a moderator:
You needs to tell people how many VM's or Cores/RAM you plan on using before you can get any realistic advice.
 
Back
Top