Linux software raid for VMware server

[Red Squirrel]

I'll be setting up a linux software raid for my VMware server. I'm very new to this but been reading up.

I always preallocate my VM space and usually choose to split in 2GB files (or is it best to leave it as 1 file?). Basically, I'll be dealing with big files most of the time. (the small text config files will be on the same volume as well, so still some small files).

So what kind of chunk size should I be looking at for this? I'm guessing bigger is better, but to what extent?

Also, what is better hardware wise: (RAID5) (disk4 is a standalone non raid drive)

IDE0: CDROM, disk4
IDE1: disk1
IDE3: disk2, disk3

OR

IDE0: disk4,CDROM
IDE1: disk1
IDE2: disk2
IDE3: disk3


I'm guessing the 2nd choice. Though IDE 2 and 3 are through a PCI board vs motherboard IDE so not sure if performance is lower on those cards, but either way I'll have to use it anyway. Disk4 is the OS, I wont bother raiding that part, I'll just take an image and store it somewhere.


Also, if a drive fails, how easy and safe is it to recover? And how do I know which drive actually failed without unplugging them all and trying to listen for weird noises?



Oh and another thing, how well does a software raid setup cope with bad shut down (ex: power failure). I don't plan to put this box on a UPS as I don't have enough capacity to add another server. This wont be production, but still don't want to loose everything to a disk corruption if power goes out.

 

[Nothinman]

I'd probably put the OS disk on the same channel as the CDROM since that's the most likely culprit for performance problems.

And I would probably preallocate one big file, I'm not aware of any reason to split them into 2G chunks and I usually use XFS which is good with big files.

 

[Brazen]

Originally posted by: Nothinman
I'd probably put the OS disk on the same channel as the CDROM since that's the most likely culprit for performance problems.

And I would probably preallocate one big file, I'm not aware of any reason to split them into 2G chunks and I usually use XFS which is good with big files.

The only reason vmware supports splitting into chunks is because some filesystems have problems with files over 2GB. The filesystem may not support files over 2G (such as FAT, IIRC) at all or may take a performance hit. This is not the case with XFS, which is what I also use to store my vms (and also use the noatime option).

Another reason would be if you may have a need for copying the files, having them in chunks can be helpful in case a copy fails then you just have to restart that chunk and any completed chunks will not be affected, as opposed to say if you have one big file and get 90 percent through a copy and it fails (for some reason, maybe a network connection drops) then you would have to start all over from the beginning. I just use one big chunk for my vm harddrive files.

And definately preallocate. I have been burned by dynamic growing harddisk files. I think it's better to be frugal with your harddrive size and keep your data on a separate hard drive file. For instance, if it is a mysql server, then I put the /var/lib/mysql partition on a separate harddrive and if I need to expand the harddrive than I just expand the file with vmextend (I think that is right, and I think the vm has to be powered off) and then expand the filesystem.

 

[Red Squirrel]

Oh thats one thing I overlooked is the file system, was just going to use ext3, but xfs is better I assume, for large files? How do I create a partition with that file system, do I need any kind of special driver?

 

[Brazen]

Originally posted by: RedSquirrel
Oh thats one thing I overlooked is the file system, was just going to use ext3, but xfs is better I assume, for large files? How do I create a partition with that file system, do I need any kind of special driver?

What distro are you using?

 

[Red Squirrel]

I'll be using fedora core 8, most likely. I was reading up and ext3 with large file flag may be my best choice though, as XFS is more prone to corruption in the event of a bad shut down. This would mean an entire VM, basically. Or is there a way to make XFS less prone? We don't get too many power outages, but still want to consider what would happen if power goes out. I probably wont have a UPS on it due to physical space restrictions.

 

[Nothinman]

ext3 only journals metadata as well so no matter what filesystem you use there's a chance of dataloss on hard shutdown. If it's really that important find a way to use a UPS.

 

[Brazen]

You are using Fedora 8 for your server? Do you really want to grit your teeth through a major version upgrade every 6 months? I do not know about using xfs on Fedora 8.

I have a CentOS 4 server at home that has been running for a year and a half. Getting xfs on CentOS requires adding some repos and installing a couple extra pieces of software. It does not have a UPS (I actually own one, but just like you due to space constraints...) and power goes out here several times a year. Usually it's just a blip, but that's enough to cause the server to reset (and those blips tend to be more distruptive than a complete power outage anyway). This houses 6 of my virtual machines on an XFS partition and I've never had any data corruption.

 

[Brazen]

Originally posted by: Nothinman
ext3 only journals metadata

uh, are you sure?

hmmm, wikipedia (http://en.wikipedia.org/wiki/Ext3) says journaling file data is a feature, but by default *most* linux distros use metadata-only journaling. Here all along I thought data journalling was being done on Ext3, not that I ever cared about it.

 

[child of wonder]

This is what I did with my VMWare server.

The host OS is Debian 4.0r3. Debian is rock solid and has killer package management. Easy choice there.

I'm running two 250GB SATA drives in RAID 1 using mdadm. Debian is installed there along with the VM files. Yes, I should have the VMs on their own RAID array but since I'll only be running 3-4 VMs at a time, this isn't a big issue.

The best way to set up the disks would be having two small drives in RAID 1 for the host OS then mount a separate RAID 1 or 5 array solely for the VMs. My chunk size is 64KB.

I'm also using XFS.

Also, to monitor the array you can set up mdadm to send you an email if a drive fails. Just make sure you have some sort of email server running on the host OS so it can send the message. I used Postfix and set it to only allow messages from 127.0.0.1 and all the messages are relayed through my ISP.

To the OP, you'll want the second choice hands down. In fact, I'd sell those IDE drives and invest in SATA if possible. Managing all those IDE cables will be very cumbersome and you could easily get a SATA card with 4+ ports.

 

[Nothinman]

You are using Fedora 8 for your server? Do you really want to grit your teeth through a major version upgrade every 6 months? I do not know about using xfs on Fedora 8.

You might have to fight with anaconda to get it to let you create the root filesystem as anything besides ext[23] but creating a data filesystem as something else won't be any more difficult than use mkfs.whatever after the installation. I don't think they officially support it but then again we're talking about Fedora here and not RHEL.

uh, are you sure?

hmmm, wikipedia (http://en.wikipedia.org/wiki/Ext3) says journaling file data is a feature, but by default *most* linux distros use metadata-only journaling. Here all along I thought data journalling was being done on Ext3, not that I ever cared about it.

It's possible to journal data but no distros enable it by default because it's extremely slow and a lot less tested. Essentially every write happens twice now because it's written once to the log and then again to the filesystem.

 

[Red Squirrel]

I'm not forced to using fedora 8 mind you, just figured I'd go with latest as it will be a new install. Would CentOS be better?

As for IDE, its just a cheap choice, as I already have 2 500GB IDE drives and the IDE card so I ordered another drive. Though if I was doing this from scratch I would go sata for sure.

 

[child of wonder]

I'd go with CentOS vs Fedora Core. It'll be a bit more stable.

Then, if you can, set up two small IDE drives in RAID 1 to host the OS and use your 500GB drives to host the virtual machines.

 

[Nothinman]

I'd go with CentOS over Fedora easily and Debian over both if it's an option, the longer support is a must.

 

[Brazen]

Originally posted by: Nothinman
I'd go with CentOS over Fedora easily and Debian over both if it's an option, the longer support is a must.

QFT


I would go with CentOS over Fedora easily (obviously, that's what I did), but I would go with Ubuntu LTS Server over Debian or the others. I would go with Ubuntu over Debian only because Ubuntu LTS (6.06) is listed as a "supported" operating system by VMWare Server and Debian is not.

CentOS is sorta supported because RHEL is supported and CentOS is a nearly identical RHEL clone (only the graphics are changed, basically). Way back when I set up my server was when VMWare Server was at version 1.0.0 and Ubuntu was not supported, otherwise I would have used Ubuntu LTS Server for mine.

 

[Red Squirrel]

Cool think I'll go CentOS then. I played with that a bit at work and liked it. I won't be putting a GUI so graphics wont matter. VMware Server does work on FC8 but you got to go through some hoops to get it going. I just can't wait for my hardware (fans/mounting brackets/3rd hdd) to come in so I can get started.

 

[Brazen]

Originally posted by: RedSquirrel
Cool think I'll go CentOS then. I played with that a bit at work and liked it. I won't be putting a GUI so graphics wont matter. VMware Server does work on FC8 but you got to go through some hoops to get it going. I just can't wait for my hardware (fans/mounting brackets/3rd hdd) to come in so I can get started.

You should check the VMWare Server documentation if you want to use CentOS 5. I think VMWare Server 1.0.4 only supports RedHat 4 (hence, CentOS 4).

Doing a gui-less install on CentOS 4 is easy. At the end, when you customize your packages, just go all the way to the bottom and choose "minimal install" which deselects everything else. On CentOS 5 it's a little more tedious because you have actually go through and uncheck everything. Basically, on CentOS 5 you can uncheck every subcategory except the Base category. I even uncheck a few things in there, but you should probably leave everything in the Base subcategory that is checked by default until you really know linux and understand the programs in there.

If this doesn't make any sense, don't worry, it will when you go through the install process

 

[Red Squirrel]

Yeah I've done GUI less installs before so no issue there.

 

[Nothinman]

Doing a gui-less install on CentOS 4 is easy. At the end, when you customize your packages, just go all the way to the bottom and choose "minimal install" which deselects everything else. On CentOS 5 it's a little more tedious because you have actually go through and uncheck everything.

Yea, the fact that anaconda includes a completely separate tool for selecting the base set of package is retarded but it's workable.