Interested in Virtualization but lost on what to do

[frowertr]

I have a Poweredge server that mostly unused. Used in my business for basic stuff like webhosting, email, fileserver, etc... I'd like to setup virtualization on it so that a friend of mine could host some services on it, but I want to keep it totally separate from what I do. That way, if he screws his VM up, he can simply re-install and start over again without affecting my services.

I guess what I am describing is basic virtualization. However, I don't even know where to begin. I am a hardware/network guy and VMs are kinda new to me. What exact software would I need in order to do this? I know that VMware is out there but their webpage seems a bit "overwhelming" as I don't know what exact piece of software I am looking for. Also, are there any free options?

OS on my system is Ubuntu Server 13.10 by the way.

 

[imagoon]

Start by looking at the various VMWare, hyper-V, (and Xen if you don't mind stabbing yourself while you read) to get a basic idea. For the most part this isnt even a "Linux" question because only Xen runs on Linux at the moment. The rest use something completely homebrewed (VMWare ESXi) or a hypervisored version of their OS (Hyper-V / Windows.) Xen / Libvirt are off in their own little niche world.

I would start with ESXi Free since that is the least Windows like of the group (since you posted in linux / use Ubuntu) it also has better support for Linux OS. Plug the CD in, reboot the machine to it and install. Of course it will wipe the machine so backup important stuff... Then http to the IP address you give the box and get the management tools and go from there.

 

[Red Squirrel]

There's also a solution called Proxmox which is Linux based.

Be careful with the commercial ones as they seem to keep changing their licensing schemes, something that is free now may not be free later, for example. Though from what I've read Xen is the most friendly, it even does free multi host environments if I recall. KVM/libvirt/Qemu also supports all this stuff. (I always get confused at all these names but I believe they're basically all the same app)

 

[imagoon]

There's also a solution called Proxmox which is Linux based.

Be careful with the commercial ones as they seem to keep changing their licensing schemes, something that is free now may not be free later, for example. Though from what I've read Xen is the most friendly, it even does free multi host environments if I recall. KVM/libvirt/Qemu also supports all this stuff. (I always get confused at all these names but I believe they're basically all the same app)

So far the big vendors keep making more of their apps free but I digress. If this is personal learning any of them is good idea. If you want to use it to enhance your career, Hyper-V or VMWare is the way to go. Way far in third is Xen (and normally Citrix's version of it) and way down in the sub 1% are libvirt apps. I see KVM mentioned a lot also but its market share has been tiny and as some placed put it "eroding VMWares empire" by some sub 1% to some other sub 1% market share.

 

[frowertr]

Great! Thanks for the suggestions guys. Yeah, I wasn't sure where to post this but I figured that most of the guys hanging out in this forum would know the correct direction on where to point me.

I didn't know the hypervisors would wipe the machine, but I guess it makes sense now that I think about it. Not a problem.

 

[frowertr]

Reading through the install guide of Vsphere 5.1 installation and setup. Seems fairly straightforward for most things. I have basically all requirements met for the install but do have a question about installing the boot image. The manual says on pg. 32:

Place all data that your virtual machines use on physical disks allocated specifically to virtual machines. Performance is better when you do not place your virtual machines on the disk containing the ESXi boot image. Use physical disks that are large enough to hold disk images that all the virtual machines use.

I am using three Near-line 6Gbps SAS drives (each 1TB) in RAID 5 via a Perc710. Are they suggesting to use one whole drive just to install the ESXi boot image??

 

[manly]

KVM is in the mainline Linux kernel and used in production by many companies. Most distros will have suitable packages that will install the hypervisor along with some GUI tools. I've used it with Ubuntu mostly, so honestly I'd go that route if I were you. Just make sure you have plenty of RAM.

As to your last question, running multiple VMs can be extremely disk I/O intensive. Keeping your VMs separate from the OS can alleviate the situation a bit. Like you said, it's a good suggestion, but not necessarily a requirement.

 

[Red Squirrel]

Raid 10 instead of raid 5 is also highly recommended for VMs. That said, I have ran 7ish production VMs on raid 5 and it was ok. I am not that familiar when it comes to measuring disk IO stuff though so I could not tell you if the performance was great or not, but the VMs did their job fine.

 

[frowertr]

I'll look into KVM as well. Man, I got lots of research to do for basically a "favor" to a buddy. Oh well, I like getting new toys anyway...

So if ESXi is recommended to be installed on its own drive, would it be best to purchase a new SSD just for the boot image? That way, I could keep the image separate from my RAID 5 array where the actually VM will be running.

If I did that I guess I would also need redundancy for the SSD as well. RAID 1 for the two SSDs? Can the PERC even do a RAID 1 array and a separate RAID 5 array?

 

[Demo24]

Just an fyi, vSphere 5.5 is the latest version, always best to try and run the latest vSphere version.

It's always a better idea to separate the ESXi OS from the VM HD's. One problem would be if you have to reinstall esxi you'd wipe the drives again. Remember, you could just install it all on a USB drive. If the usb drive gets corrupted, reinstall esxi on the drive and then import your VM's from your storage HD's.

Also, you do not need SSD's for the vSphere base. It will run just fine on 7200 rpm drives, and the reason is that it doesn't really need to use those much after it has booted. SSD's would be wasted as boot drives, and the only reason I would put them in is as a SSD cache (this may require licensing). Also, unless you get an enterprise drives you can be playing with fire running consumer SSD's in Raid. I'd recommend 2 small mechanical drives in Raid 1 at the very least, or usb as mentioned above.

I run vmware all day at work. My configurations have servers with 3x146gb (10 or 15k just for fast boot) in Raid 10 for the ESXi base. All my VM's run on dedicated storage drives, which are at minimum 10k on one, and the rest connect to my SAN with 15k drives. I'm working on adding some SSD's to improve caching. The OS install may be a bit overkill, but I need redundancy and reliability so cost isn't factored.

All my storage drives are in Raid5 so I disagree that it's not suitable for VM. My EMC SAN came pre-configured with Raid5, and that thing can push some serious IOP's! Before the SAN I had a server with 8x10k drives, also in Raid5, no performance problems there either.


I've also looked at KVM, but I just didn't want to deal with it. Vmware offered some of the best performance, a very large knowledge base, ease of use, and the reliability I needed. Pretty easy choice.

Lastly, you'll need to back this stuff up. I use Veeam, there is a free one that will work fine for you stuff, as it backs up the whole VM file at once.

 

[frowertr]

Just an fyi, vSphere 5.5 is the latest version, always best to try and run the latest vSphere version.

It's always a better idea to separate the ESXi OS from the VM HD's. One problem would be if you have to reinstall esxi you'd wipe the drives again. Remember, you could just install it all on a USB drive. If the usb drive gets corrupted, reinstall esxi on the drive and then import your VM's from your storage HD's.

Also, you do not need SSD's for the vSphere base. It will run just fine on 7200 rpm drives, and the reason is that it doesn't really need to use those much after it has booted. SSD's would be wasted as boot drives, and the only reason I would put them in is as a SSD cache (this may require licensing). Also, unless you get an enterprise drives you can be playing with fire running consumer SSD's in Raid. I'd recommend 2 small mechanical drives in Raid 1 at the very least, or usb as mentioned above.

I run vmware all day at work. My configurations have servers with 3x146gb (10 or 15k just for fast boot) in Raid 10 for the ESXi base. All my VM's run on dedicated storage drives, which are at minimum 10k on one, and the rest connect to my SAN with 15k drives. I'm working on adding some SSD's to improve caching. The OS install may be a bit overkill, but I need redundancy and reliability so cost isn't factored.

All my storage drives are in Raid5 so I disagree that it's not suitable for VM. My EMC SAN came pre-configured with Raid5, and that thing can push some serious IOP's! Before the SAN I had a server with 8x10k drives, also in Raid5, no performance problems there either.

I've also looked at KVM, but I just didn't want to deal with it. Vmware offered some of the best performance, a very large knowledge base, ease of use, and the reliability I needed. Pretty easy choice.

Lastly, you'll need to back this stuff up. I use Veeam, there is a free one that will work fine for you stuff, as it backs up the whole VM file at once.

Wow! Thanks for all that great information! Ok, I guess I'll purchase two small mechanical drives from Dell (probably the same near-line I currently have) and use that as the base. I'd rather not use a USB stick.

I'll also check out Veeam for backups.

Man, I'm kinda excited to set this up. Will give my server something to do

 

[imagoon]

You can also install ESXi to a 1GB USB stick and use that boot. Since those are about a $1 it the local computer store it generally is the recommended method. I used a stack of left over HP 2GB for actual environment. The actual ESXi OS/hypervisor really doesn't need much in the way of disk / performance. Don't install it on an SSD for example.

 

[frowertr]

You can also install ESXi to a 1GB USB stick and use that boot. Since those are about a $1 it the local computer store it generally is the recommended method. I used a stack of left over HP 2GB for actual environment. The actual ESXi OS/hypervisor really doesn't need much in the way of disk / performance. Don't install it on an SSD for example.

Well it would definitely be cheaper. I was just worried about the usb stick failing. Two new 300gb SAS 2.5inc drives from Dell are $200 each. Not sure if I can use non dell drives or not.

In production environments they are really using a USB thumb drive to boot ESXi?

Where are the .iso images kept of your various OS's? I assumed the virgin .iso(s) were kept on the ESXi base image disk.

Edit: After doing some more reading, it appears that running ESXi from USB media seems to be a very common thing. I had no idea. Seems it doesn't give you any fault tolerance should the USB key fail. One would have to be at the physical site in order to put in a cloned USB key in order to bring the server back on-line.

 

[Demo24]

Well it would definitely be cheaper. I was just worried about the usb stick failing. Two new 300gb SAS 2.5inc drives from Dell are $200 each. Not sure if I can use non dell drives or not.

In production environments they are really using a USB thumb drive to boot ESXi?

Where are the .iso images kept of your various OS's? I assumed the virgin .iso(s) were kept on the ESXi base image disk.

Edit: After doing some more reading, it appears that running ESXi from USB media seems to be a very common thing. I had no idea. Seems it doesn't give you any fault tolerance should the USB key fail. One would have to be at the physical site in order to put in a cloned USB key in order to bring the server back on-line.

In large production environments they do PXE boot, so it's all network based and that means fewer HD's. Too much trouble for your environment, and not something I care to deal with now either.

I've never done the USB thing either as it kinda scares me as to what may happen if it failed. I'd actually forgotten about it, but I'm kinda excited as that will be a great way for me to do some testing.

I'm little confused about the ISO question? You asking about the ISO's you use to install whatever guest OS you plan to use on your VM? You can do a few things: a)store them on your management PC and mount them over the network when installing b) put them on a datastore, this could either be the HD's where your ESXi is installed or the one you use for your VM's, up to you. c) on any external storage, connect it however, usb, network, etc.

 

[imagoon]

Well it would definitely be cheaper. I was just worried about the usb stick failing. Two new 300gb SAS 2.5inc drives from Dell are $200 each. Not sure if I can use non dell drives or not.

In production environments they are really using a USB thumb drive to boot ESXi?

Where are the .iso images kept of your various OS's? I assumed the virgin .iso(s) were kept on the ESXi base image disk.

Edit: After doing some more reading, it appears that running ESXi from USB media seems to be a very common thing. I had no idea. Seems it doesn't give you any fault tolerance should the USB key fail. One would have to be at the physical site in order to put in a cloned USB key in order to bring the server back on-line.

Once booted you can remove the stick. Honestly quite a few people that use the dells use the internal USB port. The stick is used to boot and nothing else. Everything else is stored on a ram drive or to a datastore you designate. I have a couple of older 1 gig sticks hitting the 5 year mark as a boot device.

Many of the Dell servers have an option for dual redundant memory cards.

You store the .iso files whereever you want. You can use the datastore or just mount them via the VM interface.

 

[frowertr]

I have learned more from this thread than searching the internet for the past couple of days. I appreciate it guys.

I have a networking question as it pertains to ESXi. I have a dual NIC Ethernet controller in the server. According to what I understand, it is best to use one nic for management and the other for dedicated VM access.

My plan is to have the ESXi management NIC on my local LAN. I then plan to create another subnet just for the VM(s) and use the other NIC for this subnet. I will just run another Ethernet wire back to my router to setup this "DMZ".

If one had multiple VMs in the DMZ but only had one static public IP, how would you traffic get routed (from the internet) properly to the correct VM? Does each VM needs it own public IP address??

For example, I need internet traffic routed to my VM and I would need internet traffic routed to my buddies VM but I only have the one static IP applied to my routers WAN interface.

 

[Demo24]

That depends on how complex this is going to get. Multiple web servers, or just something to be accessible from the outside? For something basic you could do port forwarding on the router. Ex: externalIP:50000 ---> internalIP of server

This only works if you are willing to use non standard ports. This may create unique challenges for you though.

Alternatively look for info on things like a reverse proxy: http://superuser.com/questions/596115/multiple-web-servers-behind-a-single-firewall

 

[frowertr]

That depends on how complex this is going to get. Multiple web servers, or just something to be accessible from the outside? For something basic you could do port forwarding on the router. Ex: externalIP:50000 ---> internalIP of server

This only works if you are willing to use non standard ports. This may create unique challenges for you though.

Alternatively look for info on things like a reverse proxy: http://superuser.com/questions/596115/multiple-web-servers-behind-a-single-firewall

Ok, I am familiar with Port forwarding so that isn't an issue. I kinda figured that is how it would have to work. And, no, he won't be doing any webhosting so that shouldn't create any problems either. I'll take a look at that link.

Edit: That reverse proxy is interesting. If I ever needed to host multiple websites I could install a VM just to handle the proxy requests. Man, I should sell my free server time and become a small hosting service. ; )

 

[Scarpozzi]

Ok, I am familiar with Port forwarding so that isn't an issue. I kinda figured that is how it would have to work. And, no, he won't be doing any webhosting so that shouldn't create any problems either. I'll take a look at that link.

Edit: That reverse proxy is interesting. If I ever needed to host multiple websites I could install a VM just to handle the proxy requests. Man, I should sell my free server time and become a small hosting service. ; )

If you have the bandwidth and a DNS server, you could do it. Reverse proxies are very useful because you can load balance and do rewrites to make many webs seamlessly appear as one.

So Xen and KVM are both Linux virtualization hypervisors. You really need to see which system you can get free and go with that one. ESX used to be based on RedHat, but I'm not sure what's under the hood these days since another guy in our shop manages most of our ESX hypervisors.

You have the server....you need to establish how you want to configure storage for the VMs (get as many large hard drives as you can fit in the box if you're doing local storage).... Then see how many VLANs you can get your server on. Figure out what you can do to design the Storage and Network Infrastructures up front...then as you setup your VMs, you can easily choose from the resources you define. I typically will setup a VLAN for web traffic and open the firewall for 80/443 to the VLAN...then create all my web instances and put them on that VLAN. I'll still configure local firewalls, but it's similar to having a DMZ setup.

 

[sourceninja]

Just wanted to throw ovirt.org into the fray as another virtualization platform worth trying. It sits on KVM and works great.

My lab server is a old Dell DL380 G6 with dual X5660 (6 core 2.8ghz), 96G of ram, quad nic, and 8X300GB 10K sata. I run Vsphere enterprise 5.5 (I have a free license for 'lab' usage) on a usb stick (I have a spare usb stick just in case) and run the vcenter appliance. I picked vmware because I use this machine to lab up test environments and I sometimes need to virtualize hypervisors. VMWare works great with virtualizing hyper-v, KVM, or even VMware. This lets me build setups to test things like replicaton, high availability, and even complex things like vmware site recovery manager (running 2 Virtualized EMC VNX san and a few virtualized pfsense routers).

At my last job we would run each web application as it's own minimal linux VM. Each with just enough OS to run the required app. We then had another reverse proxy vm running NGINX to route everything to the DMZ. The NGINX host also delivered our down for maintenance webpages and all static content that was common to our sites. It worked great and I highly recommend it.

 

[frowertr]

If you have the bandwidth and a DNS server, you could do it. Reverse proxies are very useful because you can load balance and do rewrites to make many webs seamlessly appear as one.

So Xen and KVM are both Linux virtualization hypervisors. You really need to see which system you can get free and go with that one. ESX used to be based on RedHat, but I'm not sure what's under the hood these days since another guy in our shop manages most of our ESX hypervisors.

You have the server....you need to establish how you want to configure storage for the VMs (get as many large hard drives as you can fit in the box if you're doing local storage).... Then see how many VLANs you can get your server on. Figure out what you can do to design the Storage and Network Infrastructures up front...then as you setup your VMs, you can easily choose from the resources you define. I typically will setup a VLAN for web traffic and open the firewall for 80/443 to the VLAN...then create all my web instances and put them on that VLAN. I'll still configure local firewalls, but it's similar to having a DMZ setup.

Thanks for the information. I will definitely keep reverse proxys in mind if I decide to host any websites for him.

LAN infrastructure is going to be kept pretty simple for now. I only have a 2-port NIC in my server. My plan was to assign one port the management port (which will be on my local LAN) and then assign the VM to the second port and that will be a DMZ. I am in the process of running the dedicated ethernet line for the VM/DMZ back to one of the available ports on my router now for this. I guess he will need to let me know what ports he will use so that I can open those up on the firewall.

Storage won't be a problem. I have available 8 external hot plug drive trays in my server all connected to a Perc710. Currently running 3 drives in RAID 5. I don't think storage will be a concern.

 

[frowertr]

Just wanted to throw ovirt.org into the fray as another virtualization platform worth trying. It sits on KVM and works great.

My lab server is a old Dell DL380 G6 with dual X5660 (6 core 2.8ghz), 96G of ram, quad nic, and 8X300GB 10K sata. I run Vsphere enterprise 5.5 (I have a free license for 'lab' usage) on a usb stick (I have a spare usb stick just in case) and run the vcenter appliance. I picked vmware because I use this machine to lab up test environments and I sometimes need to virtualize hypervisors. VMWare works great with virtualizing hyper-v, KVM, or even VMware. This lets me build setups to test things like replicaton, high availability, and even complex things like vmware site recovery manager (running 2 Virtualized EMC VNX san and a few virtualized pfsense routers).

At my last job we would run each web application as it's own minimal linux VM. Each with just enough OS to run the required app. We then had another reverse proxy vm running NGINX to route everything to the DMZ. The NGINX host also delivered our down for maintenance webpages and all static content that was common to our sites. It worked great and I highly recommend it.

Great! I'll check into that one as well. Free license for "lab usage". Yeah, I need that too! 🙂