New VM server!

[Red Squirrel]

Been over a month since I've been waiting but all the parts finally came in. Was waiting on the CPU, kinda important.

Specs:
Intel Xeon E3 1270 V2 Quad Core Processor 3.5GHZ 8MB LGA1155 69W Retail Box *IR-$27*
Kingston KVR16E11K4/32 32GB 4X8GB Kits DDR3-1600 CL11 DIMM ECC Quad Channel Memory
Kingston SSDNow V300 120GB 2.5in SATA3 LSI SandForce Solid State Disk Flash Drive SSD
Supermicro 822T-400LPB 2U 6 3.5in Hot Swap 7LP EATX 400W Chassis
Supermicro X9SCL+-F mATX LGA1155 DDR3 ECC 2GBE IPMI 3PCIE 6SATA2 9USB2.0 Motherboard

Built and booted up first try! Now to assign an IPMI IP and stuff, and do a memtest and all the usual stuff. IPMI does not seem to work out of the box in Linux though. :/ It wants Java, but when I click to install it, it does not work. There are millions of versions of Java and apps tend to be very version dependant so I doubt installing just the latest will work, but I'll mess with that later. Worse case scenario I need to use a local Windows VM to manage it.
Anyway, enough yapping, here are a few pics:



Installing the CPU. I always feel like a sci fi villain when I install a CPU. It feels like it's the final step in completing some AI robot that will destroy the world.


Downside of using a SSD for system drive... there is never an easy way to mount it without buying some special bracket. It's just sitting there for now, but think I will try to see if I can find an adapter that will make it work in the drive caddy, that way I can load it from the front. While hot swap is obviously out of the question for the OS drive, at least it will make replacement easier if it fails. Reason I use a SSD is I figure there is probably is lower chance of failure and it saves me from having to use hardware raid 1 with two spindle drives.


First power on!


Fans, temps, n stuff

 

[thecoolnessrune]

I take it that the Supermicro shroud doesn't fit that board properly?

 

[ch33zw1z]

Cool man :thumbsup:

 

[Red Squirrel]

I take it that the Supermicro shroud doesn't fit that board properly?

Did not come with one, though my last build which was my file server using a 24 bay one did come with a shroud but I could not get it to fit with all the cables and such. I'm sure there's a way, but I did not really try that hard.

The cable management I did is not the best, but for inside a server I don't usually put that much effort into it, I'll bundle them up to maximize air flow but other than that I don't tie them down too much.

Nice thing with rackmount cases though is they are so much easier to work in than standard PC cases.

 

[imagoon]

Needs more RAM! 384GB is a good start.

 

[Red Squirrel]

Yeah 32GB is the max it would take without upgrading to the socket 2011 platform which pretty much doubled the price of everything. I was toying with the idea, but really if I'm getting by with 8GB of ram right now I think 32 will be plenty. I can always build more servers in the future too and take advantage of HA.

I need to upgrade my power first though, my inverter is only 750w and I get 4 hours of run time so I also need more batteries if I will add more equipment. I'd like to move to a several kw 48v dual conversion system at some point and add a couple more strings of batteries. Couple redundant rectifiers and one or two inverters. We don't get that many outages here but it's nice to be prepared and have plenty of run time.

 

[thecoolnessrune]

I went through 32GB on each of my lab servers in a heartbeat. After 2 vCenter servers, 2 DNS servers, a TV hosting server, and a backup server, I was tapped out with just 16GB going to my SAN. Had to bump them up to be able to get the more useful VM's running.

 

[Red Squirrel]

Most of my stuff is Linux, so a typical VM has like 256MB of ram. The Windows/Minecraft server will easily eat 4GB on it's own though. It also runs my Unifi management and few other Windows things.

though I do plan to get into more projects so as I need to make more VMs I could potentially find myself needing more ram but I'll just build another server at that point. The 2011 stuff might be cheaper by then too. Considering how much trouble I had finding a E3 that was in stock I don't think they'll be making that platform for very long, if it has not stopped already.

 

[Red Squirrel]

Well this freaking sucks... Ram is bad. I knew this was going too smooth, there's ALWAYS something that has to go wrong. ARGGG

 

[thecoolnessrune]

Lol! Well at least you find out now instead of later! When I built my setup I had 1 stick out of 8 be bad on arrival as well. Thankfully replacement was painless through Newegg.

 

[drebo]

Honestly, it'd probably have been cheaper to buy an old PowerEdge R610 or Proliant DL380 G7 on ebay.

 

[Red Squirrel]

Honestly, it'd probably have been cheaper to buy an old PowerEdge R610 or Proliant DL380 G7 on ebay.

Not with the shipping. Most people want like $300+ to ship to Canada, it's pretty ridiculous. At least I have warranty (which I'll need to RMA half the parts till I figure out what's wrong).

I think I also picked the wrong motherboard... it looks like this one has lot of issues. Damnit. That's always my luck with this stuff.

http://www.lucidpixels.com/blog/supermicrox9scm-fissues

It's also saying my ram is not ECC, when it actually is. It's also detecting the cpu as a core i7 and not a Xeon. It sounds like I may have a bigger issue than bad ram. What a pain. I wish I could build one single computer in my life time where there arn't tons of issues to deal with. For some time, I thought this was going to be it, because everything was going so smooth. But I judged too soon considering I had not tried any tests yet.

 

[drebo]

Well, the solution to that's easy. Don't live in Canada.

 

[Red Squirrel]

Using memtest 5.01 and so far so good. Here's hoping my issue is with that version of memtest and not the system.

 

[yinan]

You can replace the fans with quieter fans without any problems so that Supermicro case isn't nearly as annoying.

 

[ultimatebob]

Man... I really need to do my own VMWare ESXi hosting setup at home. The PowerEdge T110's with only 8 GB of memory that I've been using aren't cutting it with 5.5, especially if you want to play with the new vSphere web client.

 

[seepy83]

especially if you want to play with the new vSphere web client.

The web client sucks so bad. I tried my hardest to use it 100% of the time, especially because VMware says it will be the only option in the future (next release, I think?). But, after 3 weeks, I went back to using the traditional client for all tasks except when I absolutely must use the web client (which is basically only when I need to edit VMs that are at hardware version 10).

There are some parts of the web client that I like (the "Work In Progress" area is one of them), but overall I find that it isn't as responsive and needing to refresh regularly to get a current/real-time view is annoying.

 

[yinan]

They will never be able to fully get rid of the thick client. You run into a chicken and egg scenario. How do you deploy vCenter without a way to connect to the host and deploy the VM.

 

[Red Squirrel]

You can replace the fans with quieter fans without any problems so that Supermicro case isn't nearly as annoying.

Actually I'm surprised at how QUIET this case is. Wont be needing to change anything. I imagine the fans rev up based on temperature. If I pull one fan out then they all go to 100%. This will be in my server room anyway and it already sounds like a data centre in there. Got a couple 1U boxes too.

So the memtest with the latest memtest86 passed, so the issues I had may have been a false alarm. Though it still kinda bothers me that it crapped out like that, the older version worked fine before on other systems so not sure why it would suddenly not work.

 

[imagoon]

They will never be able to fully get rid of the thick client. You run into a chicken and egg scenario. How do you deploy vCenter without a way to connect to the host and deploy the VM.

Simple, they plan to include the web client in the base ESXi package. At least a basic version suited towards the initial deployments and free editions.

 

[Red Squirrel]

I run Linux so ESXi is probably going to be a pain in the ass to use if I do go with that. I do recall using the web interface and it was ridiculously flaky. Had to always hit refresh to reload it since it would just lock up and stuff.

I'll be trying Proxmox first, though that is web based too so I really hope it works in Linux and does not use Java or anything like that. Java is suppose to work in Linux but I find when it comes to web based stuff it's very hit and miss.

Though worse case scenario I can use VNC for the console as KVM/Qemu usually has a VNC option.

 

[imagoon]

I run Linux so ESXi is probably going to be a pain in the ass to use if I do go with that. I do recall using the web interface and it was ridiculously flaky. Had to always hit refresh to reload it since it would just lock up and stuff.

I'll be trying Proxmox first, though that is web based too so I really hope it works in Linux and does not use Java or anything like that. Java is suppose to work in Linux but I find when it comes to web based stuff it's very hit and miss.

Though worse case scenario I can use VNC for the console as KVM/Qemu usually has a VNC option.

ESXi is very Linux friendly. No idea why it would be "a pain in the ass." The web interface, specifically the 5.5 one is actually pretty good. Just doesn't have the speed of the thick client. Most cases I fine the open source VM systems full of promise, lack of delivery.

 

[Red Squirrel]

ESXi is very Linux friendly. No idea why it would be "a pain in the ass." The web interface, specifically the 5.5 one is actually pretty good. Just doesn't have the speed of the thick client. Most cases I fine the open source VM systems full of promise, lack of delivery.

Hmm it may have changed since last time I used it then. Though I did not check the HCL before choosing parts so it may not even work on this server. Is it full of limitations though? Like how much ram and cpu cores it can use? This is not a lab, this is actually production, so I need something that will deliver (without costing anything).

If Proxmox proves to be not that great I might just use KVM on it's own. ESXi is the easiest as far as using but it comes with a price tag if you want all the features. Eventually I do want to do HA and stuff. Either way, I'll experiment and may end up going ESXi if nothing else delivers. I did not have any way to experiment so that's why I bought the server before making a decision.

 

[Red Squirrel]

Racked it today. It's now on UPS power and fully connected and labeled.







Starting to run out of ports on my switch! I have a 10/100 switch I scored cheap off ebay a while back, I might actually put it into service. A lot of stuff can be moved to 10/100 such as my HTPC, IPMI, Wifi APs etc...

 

[imagoon]

Hmm it may have changed since last time I used it then. Though I did not check the HCL before choosing parts so it may not even work on this server. Is it full of limitations though? Like how much ram and cpu cores it can use? This is not a lab, this is actually production, so I need something that will deliver (without costing anything).

If Proxmox proves to be not that great I might just use KVM on it's own. ESXi is the easiest as far as using but it comes with a price tag if you want all the features. Eventually I do want to do HA and stuff. Either way, I'll experiment and may end up going ESXi if nothing else delivers. I did not have any way to experiment so that's why I bought the server before making a decision.

Free is limited to 4TB of RAM, 62TB Virtual disks and 2 sockets, unlimited cores / socket. The last versions limitation was the same as your current hardware also so the hardware you selected would have not been limited by the free license.