SATA SSDs vs PCI-E SSDs (i.e. OCZ Z-Drive)

[taltamir]

Unless you are building a business server of some sort, a home system with all bells and whistles should be:
1a. Single non RAID SSD with TRIM, as small as you can use and still fit in all your programs and games (no need to fit documents)
1b. A RAID1 of two spindle drives, this will contain an auto backup of your SSD (if setup correctly there should be very little to backup, my backup of my SSD is a mere 3.5GB) as well as containing your documents, etc.

or alternatively.

2a. same as 1a
2b. a single spindle drive containing most your stuff.
2c. A fileserver running RAID1 or RAID10, ideally with ZFS, contains backups of your SSD and single Spindle disk

I am currently running a variation of #2... I had previously set up my fileserver with RAID6 instead of RAID10 to my regret. (well, raidz2 = raid6 for ZFS)

As said, RAID =! backup... RAID0 actually makes reliability WORSE (by a lot). RAID1 is the most reliable (followed by RAID10 or RAID01), only those should be used.
Reliability from best to worse: RAID1 + Backup > RAID1 but no backup = Backup by no RAID1
Depending on the exact backup scheme RAID1 only is either better or worse then backup only... but either one leaves vulnerabilities which are taken care of by combining the two.

 

[sub.mesa]

If ZFS is the choice, then booting from a mirror or RAID-Z might be better than buying an SSD for the system disk. Then you can use the SSD for something cooler like L2ARC cache device or SLOG or some ZVOLs containing system disks (think iSCSI images).

Also using RAM to store the OS (~500MB) is possible, to remove the need of a separate system disk that needs to be active.

If you install and boot from mirror or RAID-Z, your system disk also shares the redundancy level of your pool. Note that booting from RAID-Z only works on FreeBSD and using my ZFS web-interface (v0.1.6).

 

[taltamir]

If ZFS is the choice, then booting from a mirror or RAID-Z might be better than buying an SSD for the system disk. Then you can use the SSD for something cooler like L2ARC cache device or SLOG or some ZVOLs containing system disks (think iSCSI images).

I was assuming that the main machine is running windows. ZFS is only for the fileserver

 

[sub.mesa]

Oh alright. Yeah that makes more sense.

 

[sol326]

Now this is more like what I was looking for. EXCELLENT. So how about if this IS a business server?? How would you suggest setting up the disks then?

 

[taltamir]

Now this is more like what I was looking for. EXCELLENT. So how about if this IS a business server?? How would you suggest setting up the disks then?

this entirely depends on what kind of business server it is. what exactly are the parameters of its operation?
(if you can specify what it does, disk access patterns, volume of data, etc would help)

 

[sol326]

Sorry took me so long to get back, out on errands. OK I'm looking for scalability of course. Growth is always a factor.

I do several things. I am using VMWare on some 8GB SSD CFast or CF cards then using a storage server (SAN) to store all the datastores. Looking into building SAN right now. Already built VMWare box and trying to spread loads between boxes due to multiple operations under one roof. Although the majority of work is Accounting intensive through various quickbooks and different databases depending on the clients needs are used by at least 5 people continuously.

I guess my biggest question is what is best setup for a SAN using VMWare, would you use VMWare on that box and have ssds or PCI-es for the main storage. Keeping in mind how VMWare works and only the 8GB CF or CFast drives are the bootups. The main storage would be on the ssds with some spindle drives as backups. OR the PCIe with some spindle backups? I also run a few different servers for web servers to test ideas that may not justify a whole hosting platform until it's able to pay for itself.

I am leaning towards the PCI-e to save space since it's just as much as all the SSD drives you need in a raid and have to idle them the same to assure efficiency as well.

 

[LokutusofBorg]

So an actual commercial (pre-built) SAN is out of your budget? I don't see the advantage of trying to build a home-grown SAN instead of just building a stand-alone "file server" and having your VMs access it via network share. Are your VMs the actual file server already? I guess that makes sense in a QuickBooks scenario, but then I don't see how virtualizing the file server would gain you any flexibility or performance increases. It's all network and IO traffic, and you're not gaining anything for either of those things through virtualization.

These PCIe cards *can* be raided together, but I imagine only if you run Windows on the bare metal. Several reviews of these cards have shown Intel's ICH10 RAID to be more than sufficient (it actually outperforms some other solutions).

FusionIO has a new "workstation" setup that's probably worth looking at in your scenario. It may not be enough space, but if you look at a more traditional divided setup then it absolutely makes more sense for your most performance-intensive applications.

SSDs on servers are all about targeted improvements. Even pre-SSD you didn't use high performance storage (SAS) for your whole (front-line) storage solution. Some stuff needs high performance storage, most stuff doesn't. If *all* your stuff needs high performance storage then, well, that's what it needs. But the only thing you've mentioned is QB.

 

[sol326]

OK OK My newbieness is shining bright. I obviously don't know the difference between a 'SAN' and a 'file server', thought they were interchangeable from some reason. So yes essentially what you described is what I'm looking to do. Is there a reason you would use Server 2008 R2 over OpenFiler?

So how would you set up the 'file server' in this instance? Yes QB and such DBs like AME 2.0 and I rent some space and let those clients use server space, thus some use various adobe and specialized programs for their fields. Along with some website software that will be accessin it as well. But mostly accounting intensive since it is an accounting firm, but like all self employed individuals, dabbles in a little of everything.

 

[LokutusofBorg]

I don't have enough info about your setup to offer any specific advice. If it were me I would only commit to virtualization for very compelling reasons, as it changes almost everything else you have to do. If you're renting out pure file servers then I see virtualization as a drawback in that scenario, but I certainly don't have a complete picture. Being able to rent out an isolated server is certainly a good reason to virtualize, but if you're renting out isolated servers just to accomplish a file server for a QB database, then it would seem to me worth looking at revisiting that decision. That may be worthless advice, given that you've already thought through things enough to have built your VMWare box already.

If you can get away with everything on one very large box hosting your VMs then you don't need external storage. If you can get away with two boxes hosting your VMs and you don't need the ability to move VMs instantaneously (move a given VM to box 2 overnight, for instance) then you don't need external storage. If you do decide you need external storage, then you should probably look into iSCSI. You mention HBAs in a previous post, but you don't need to go there unless you know you need to go there.

EDIT: Forgot to mention that I've never heard of OpenFiler. If I was running my own business and faced the prospect of paying the license costs of Windows Server myself, then yeah I would probably be looking at alternatives. However, buying a pre-built Dell server isn't a ton of money, and I am a big fan of Windows.

Oh, and unless you have some performance-intensive apps actually running on your VMs then going with SSDs is questionable. File servers are not going to see much gain from SSDs if you're anything close to being limited by network bandwidth. Now an actual database server, running SQL? Yes, definite gains from SSD. I am, in fact, trying to move things at work towards hosting our databases on SSDs and keeping our DB servers out of the virtualization cluster. FusionIO has lots of case studies of companies doing this and achieving phenomenal performance while at the same time saving money.

 

[sol326]

While I've been setting up I have had a VMWare 'guru' locally helping me and he has since sort of bailed on me, so I'm trying to pick up the pieces. What I was told by him was that if I want to do websites hosted for myself that they take a lot more RAM and better off on their own box to max out RAM space for that function, due to me wanting multiple sites going in their infancy, and transferring them offsite once they pay for themselves. I also didn't want those to cause havoc on my production machine. So I thought I would go ahead and make a file server to save on harddrive space and to increase performance through SSDs. I have a prelim box built but only has HDDs we built on the fly but he stated my drives were slow and suggested finding better hard drives and I found the SSD route and haven't really heard from him since.

So that is sort of my scenario. Ask away for anymore clarification. ANY help I can get at this point is a plus.

 

[LokutusofBorg]

That still doesn't really provide any info that I can give specific advice towards. Web sites are generally not performance-intensive so you'll see little benefit from SSDs for anything you've talked about. I think you'd be better off building a RAID array of current generation SATA 7200 drives like the 7200.12s, the F3s or whatever WD's latest gen drive models are.

If you still need advice here, then what you probably need to detail out is what you're actually renting out as server space.