build help - virtualization lab (likely Hyper-V)

[simas]

I would like to refresh my knowledge of Microsoft RDBMS (SQL Server 2016 in particular) and thinking of setting up a virtualized environment that would have both real (licensed/owned) systems I need, as well as temporary guests evaluating software from Microsoft. key SQL Server features now are only testable in multi instance setups (availability groups ,etc.) thus consideration to just build a lab. I am most familiar with Windows Server OS (running domain at home) and considering Hyper-V server as barebone virtualizer, clustering it if possible. Answers to template questions are below, please let me know if anything else I can add.

Key goals
- resiliency where possible
- low power consumption where possible.
- guests , one win 2012 domain controller (always on), Win 7 workstation, 2 Win 2016 evaluation systems powered from time to time.

no gaming, no high demand workloads.

1. What PC would be used for - run virtualizer software that could host guest OS systems. Guests would be primarily Windows Server machines for multiple SQL Server 2016 installations.
2. Budget - $600 for system without monitor , $1000 if I can make it a cluster.
3. Living and buying in USA, close to physical Microcenter store
5. No Brand preferences.
6. I have existing hardware i7-3770k, 32 Gb RAM that I can free up from it is doing right now. I know it does not have VT-D so not sure if it is worth having for this node.

I do not overclock.

thank you for your help

 

[Burpo]

If you need VT-D you could drop a Xeon E3-1240 v2 in your existing machine (add video), and sell the I7-3770K.

http://ark.intel.com/products/65730/Intel-Xeon-Processor-E3-1240-v2-8M-Cache-3_40-GHz

 

[XavierMace]

How much performance do you want/need? If you're running multiple SQL instances, that can chew through a quad core really fast.

 

[cytg111]

http://forums.anandtech.com/showthread.php?t=2479850

Best deal by far :

Intel S2600CP2J dual-2011 Motherboard - $175
2x Intel Xeon E5-2670 (8c/16t - 2.6GHz) - $130
8x 8GB ECC RAM - $85

http://www.natex.us/product-p/intels2600cp2j-custom.htm
http://www.natex.us/product-p/64gb-am472d3ld4p13c9ec.htm

16c/32t, 64GB ECC + mobo for $390, ie. less than a 6700K + mobo combo at MicroCenter

 

[simas]

Thank you for suggestions!
for the Xeon drop in - yes, I believe that is an option. It would add hardware virtualization support without addressing resiliency too much.

for hardware requirements, I would need at least the following
- Win 2012 Essentials (domain controller), 2 threads, 4 Gb RAM - https://msdn.microsoft.com/en-us/library/ms143506.aspx
- Win 7 workstation , 1 thread, 3 GB ram (based on existing image)
- 2 Win Server machines 2 threads each , 6 GB ram, each running SQL Server 2016
https://msdn.microsoft.com/en-us/library/ms143506.aspx

so at minimum 20 GB RAM, 7 threads without any growth space. the goal for the lab is not performance (I have done enough performance optimizations over decades including installations with TBs RAM sizes), the goal is to have enough to test out some of the new features, expansion of the availability groups, HA/DR (which desires multiple instances with their own IPs), R server within SQL .

as I am thinking this through, would I be better served buying at least two of identical machines , running hypervisor on each, and have them in logical cluster that could in turn host collection of VMs.

is there something in between NUCs and full on workstations that could work? I like the idea of NUCs but am not sure there is enough storage flexibility in them to make it work..
Thank you

 

[simas]

cytg111 - thank you, our messages crossed in cyberspace.

for your suggestion , do you mind telling me more (as I am reading your thread).
My understanding is that solution is roughly as follows
++ insane amount of compute for dollar.
however
- basically SB technology circa 2011 in terms of energy and performance efficiency
- server pulls, could have been in production for 4 years
- no warranty beyond 30 days provided by refurbisher

still needs
video
CPU cooler
case

is this roughly correct?

also, what is your power draw on the idle for this system (if you already installed it)?
Thank you!

 

[XavierMace]

Being in production for 4 years is nothing for a server. There's a huge difference between servers and desktop computers when it comes to longevity. Given this is for a lab, you're hard pressed to beat surplus equipment for performance/value.

Since you're running a hypervisor, you're typically going to be accessing your VM's remotely so the server doesn't need anything more than basic video which pretty much all server boards have.

No, it's not going to be as efficient as a Skylake system. But you're going to get a buttload more performance for the money. I've got a Westmere EP system (dual L5640's, 64Gb of RAM), it's pulling 150w with half a dozen VM's running on it. You can get complete running servers of that age for a few hundred dollars these days.

 

[simas]

Thank you XavierMace

Funny , how in 'other life' (real job), I get rid of anything older than 3 (at most 4) years as part of normal hardware currency and in non-work considering picking up such systems..


If Anandtech exports would mind more assistance - assuming I will look at this
http://www.natex.us/Chenbro-Intel-S2600CP2J-p/rm13704package-128gb.htm
described as

"

Description





*Intel S2600CP2J Motherboard w/ Dual LGA 2011 Sockets, 16 Dimm Slots
*2 x E5-2670 SR0KX 2.6GHz (3.3GHz Turbo) 20MB L3 Cache LGA2011 115W Eight-Core
*128GB PC3-12800R DDR3 1600 ECC Registered RAM
*1 x 550W Power Supply
*1 x Intel Hologram X520-DA1 Single port 10 Gigabit Network Adapter
*Does not include any drive cages
"
what else do I need for it to power on ? drive cages to host HDDs, cables for the storage, what else? do I need GPU? what about network, pictures show two GibE network slots already.

Thank you

 

[XavierMace]

Drive trays, cables, potentially a RAID controller depending on what type of back plane the case has. As mentioned previously, server has basic VGA onboard (Matrox G200). Price looks good on that unit although I prefer redundant/hotswap power supplies. Keep in mind since it's a 1U, it's going to be rather loud. It's a pair of onboard GbE ports (Intel I350) the X520 10GbE card they include. I'm assuming it's got heatsinks/fans although it doesn't explicitly say that.

Personally, I also have a preference for ESXi over Hyper-V. I'd install that onto an internal USB thumb drive.

 

[simas]

Thank you again. Few more questions

- just how loud are these things? as DBA for large company they keep us away from hardware and it is hard to draw a comparison

- heatsinks you have to buy for a few extra $ if I understood the site correctly

- for hypervisor, I am reading up on what limitations free/home use version of ESXi offers vs Hyper-V. if there are no limitations that bother me (i.e. limiting to only 2 CPUs is not a concern given the server I want)

- agree for redundancy on PSU and everything else. will look what else is available. the item is that I do not want to go much above $1000 for essentially recycled hardware as in that case I should just pony up and buy a real thing from dell/hp with warranty and support. there is a point of diminishing returns here.

 

[redzo]

Thank you for suggestions!
for the Xeon drop in - yes, I believe that is an option. It would add hardware virtualization support without addressing resiliency too much.

for hardware requirements, I would need at least the following
- Win 2012 Essentials (domain controller), 2 threads, 4 Gb RAM - https://msdn.microsoft.com/en-us/library/ms143506.aspx
- Win 7 workstation , 1 thread, 3 GB ram (based on existing image)
- 2 Win Server machines 2 threads each , 6 GB ram, each running SQL Server 2016
https://msdn.microsoft.com/en-us/library/ms143506.aspx

so at minimum 20 GB RAM, 7 threads without any growth space. the goal for the lab is not performance (I have done enough performance optimizations over decades including installations with TBs RAM sizes), the goal is to have enough to test out some of the new features, expansion of the availability groups, HA/DR (which desires multiple instances with their own IPs), R server within SQL .

as I am thinking this through, would I be better served buying at least two of identical machines , running hypervisor on each, and have them in logical cluster that could in turn host collection of VMs.

is there something in between NUCs and full on workstations that could work? I like the idea of NUCs but am not sure there is enough storage flexibility in them to make it work..
Thank you

Hyper v is included in Windows 8.1 and 10. You don't have to install a windows server version.

Do you need vt-d? vt-d is for passing through host devices to the guests(nic's, hdd's). It doesn't look like you need it.

Do you even need a server? It looks like your 3770k system will do just fine. How about just try it out on your current system. It doesn't cost you anything. If it works, the only thing that I can think of, the thing that will keep your sanity is: drop an ssd on that machine and run all your vms from on the the ssd, but not before testing it out if it works on one of your mechanical disks.

 

[cytg111]

cytg111 - thank you, our messages crossed in cyberspace.

for your suggestion , do you mind telling me more (as I am reading your thread).
My understanding is that solution is roughly as follows
++ insane amount of compute for dollar.
however
- basically SB technology circa 2011 in terms of energy and performance efficiency
- server pulls, could have been in production for 4 years
- no warranty beyond 30 days provided by refurbisher

still needs
video
CPU cooler
case

is this roughly correct?

also, what is your power draw on the idle for this system (if you already installed it)?
Thank you!

Np. Yes your assesment seems correct to me, that is the cost/benefit equation you have to balance out if its worth it to you or not. 16 cores/32 threads for "pocket change", downsides is what you listed though I would add that since sandy we have only seen small incremental advances ipc/ghz ivy-haswell-broadwell-skylake.
It will be a long time before a 16 core sandy beast will be too slow for my needs.
Oh and I dont have the system yet, the chips are in the mail, still shopping for ram and motherboard (I figure ill just mount it on the wall, bypass the casing ).

 

[XavierMace]

Thank you again. Few more questions

- just how loud are these things? as DBA for large company they keep us away from hardware and it is hard to draw a comparison

- heatsinks you have to buy for a few extra $ if I understood the site correctly

- for hypervisor, I am reading up on what limitations free/home use version of ESXi offers vs Hyper-V. if there are no limitations that bother me (i.e. limiting to only 2 CPUs is not a concern given the server I want)

- agree for redundancy on PSU and everything else. will look what else is available. the item is that I do not want to go much above $1000 for essentially recycled hardware as in that case I should just pony up and buy a real thing from dell/hp with warranty and support. there is a point of diminishing returns here.

If you're talking about running this on your desk, it's going to annoy you and everybody in probably a 20 foot radius around you. You'll be able to hear it over headphones unless you've got bad hearing or REALLY good headphone.

Heatsinks will be about $30/ea. Redundant PSU's are going to eat of the rest of the budget quickly. The only way to get decent ones cheap are to get a cheap system that already had them unfortunately.

While I'm never one to turn somebody away from buying new, as somebody who sold HP's for 6+ years, you're looking at least $6k for a comparably equipped new system with support. Threw together a quick quote on Dell's website:

PowerEdge R430 Rack Server

Starting Price $10,516.00
Instant Savings $3,290.51

Subtotal $7,225.49

That's configured to match your used system as close as possible. If the company you work for is a Dell shop, you might be able to order through them for quite a bit cheaper, but I'd still expect to pay $6k. That's a lot for a lab system. HP is generally a bit more than Dell (although I prefer HP).

 

[simas]

"Hyper v is included in Windows 8.1 and 10. You don't have to install a windows server version"

Thank you, SQL Server 2016 would not install on non server OS so I need Windows Server regardless, testing out scripted installs/configs against both Win 2012 R2 server and Win 2016 Server (once available).

"need for VT-D"

if I can find a way to make this system/cluster resilient enough I would retire existing server by virtualizing it. original intent was for existing 3770k machine to be
- domain controller
- native backup for all clients (built into Windows Essentials line)
- media/file server

Windows 2012 Essentials did all of that well, however I have since bought low level NAS for file serving and moved media to Nvidia Shield TV console with backup on NAS. I still want domain controller functionality and would want the images of my client machines to be taken for ease of recovery. My thought was that isolating and passing HDD to this VM explicitly for image backup were to help with performance, am I somewhat correct?


"Do you even need a server? It looks like your 3770k system "

I am considering it. existing 3770k is used as defined above, however I do not want to disrupt it without reason - in theory I can put virtualbox on my gaming machine (2600k, 16 GB RAM), install Win 2012 Essentials from CD, migrate the domain controller and retire old server. I will do this is I decide to reuse old equipment for the lab. my concern in that case would be whether 3770k is enough to run 3 instances of VMs each with SQL 2016. RAM wise I think I am ok, 4/8 CPU , not so sure about


"
If you're talking about running this on your desk, it's going to annoy you and everybody in probably a 20 foot radius around you. You'll be able to hear it over headphones unless you've got bad hearing or REALLY good headphone.
"

it will go to the basement where I am planning to put equipment "rack" (whether of the shelf or self made is TDB). still, noise is a concern. for the servers overall, if I buy from Dell/HP it would be for lower configuration that what is available in refurb space. 7k spend on a server would not currently pass family counsel approval , thank you testing the quote - i did a lot of installs and work on both Dell and HP systems (as well as Cisco and IBMs) and was tired repeating that as RDBMS "we do not care", at all.

so where I am now
1. still looking at natex.us to see if anything makes sense in combination of performance/noise/price
2. seeing what alternatives other to natex.us
3. looking at free ESXi and its limitations
4. understanding what needs to be present in order for server-server to power up and work

I am also buying a house and will post a different set of questions elsewhere on to see if there are good guides/recommendations/ways to find experts who can help with/do wiring (for pay obviously) vs going with random electricians.

 

[XavierMace]

Check eBay as well. Oodles of surplus servers on there. Just stick with a seller with feedback and read the description carefully to see what is included.

Since you're running a single host, the only restriction of free ESXi that would apply would be no access to the API so you can't run software like Veeam with it. You'd have to do file level backups on the VM's if you need backups.

As a general rule, 2U servers are quieter than 1U. HP is usually louder than Dell. Supermicro depends on the configuration and especially the power supplies. You usually get a few extra onboard GbE ports as well going with 2U.

I'd have to look through Dell's SKU's, but if buying surplus try to look for a server with the lower wattage platinum rated PSU's. It may not be a huge electricity savings, but typically the sellers don't adjust price based on which model of PSU's are installed, just the quantity so might as well get the more expensive efficient ones. In Supermicro's case they are substantially quieter as well.

For labs, I'm willing to sacrifice some CPU grunt for lower power SKU's on the CPU's. If price is close, try to get a server with the desired memory capacity using as few sticks as possible. It pains me to see 16+ slots filled with 2Gb DIMM's. If going Dell, make sure it's got the iDRAC Enterprise (with license) otherwise you lose most of the remote management features.

 

[simas]

First -THANK YOU!! For the help, advice, comments, and simply taking your time. I think a lot of people take for granted how valuable is to have community that can respond and I wanted to say thanks for what I have recieved

Second, I wanted to update in case anyone else was looking for something similar - after thinking about it I decided to go with multiple weaker devices vs net new powerful, loud, refurbished server.

Here is what I purchased and am doing
1. Got new NAS device to be "primary NAS" in synology 216j to complement existing synology 115j I have. Synology due to comfort with their ecosystem and fairly low touch operations in terms of on going care. All data now lives in two NAS systems that cross sync with each other, with additional backup of primary NAS to usb3 HDD. $170
2. Since data left the server, went with Qotom 310g4 device containing 3215u Celeron dual core and 4 gigabit LAN ports. $170 (device is 140 plus shipping , taxes, custom duties). Add to it 8GB RAM ($36) and 250 GB mSATA from Crucial ($88) for approximately 300 dollars I get new server that is modern tech, not refurb, with LAN aggregation and storage and memory expansion capabilities
3. New server is promoted to run domain, old server is demoted
4. Old server will be used for virtualization host only and be powered down unless needed. I have not decided on whether to seek relevant Xeon to swap for 3770K. Similarly have not finalized choice of hyper visor