Of storage and file servers

[Gast]

So I'm looking at building a file server but after some research I'm not sure just what to do. I can't seem to find any articles/reviews to really make up my mind. So I'm looking for some help in pointing me the right direction.

I store a lot of stuff, for both me and friends. I've also had to delete a lot of stuff due to my limited storage space (1.5tb). A large section of the files are large in nature (400mb-10+gb), another decent chunk is of average size (20-400mb), and decent number (but small total size) of small files (<20mb most <1mb). I'd say that % of space is 70% large, 29% average, and 2% small. I'm tired of deleting and shuffling files around different drives to make space for new stuff.

Originally I was just going to add a huge storage array to my primary system (running Vista) and then just use shares for network sharing with a hot swap bay for faster file xfers. But I have read a few places that NTFS has issues with large storage areas. So I started looking for other solutions. I looked at NAS but then saw a few benchmarks and posts pointing to poor performance. I looked at DAS, but afaik that would leave the FS being NTFS. I looked at SAN, but I'm not quite sure if I could move away from NTFS.

I'm planning on using raid6. I want protection against data loss due to hardware failure. I don't need high availability or performance when there's a bad drive. I think I'd like the extra protection of raid6 vs raid5. On a similar note I'd also like tools to monitor for data corruption, but I'm not sure what I need, if anything.

I'd like the solution to be rackmount, as I would like to consolidate several pieces of hardware in my network closet (which is right behind by primary comp w/ plenty of ventilation). But I don't know where to look for server hardware reviews.

I'm not quite sure what kind of budget I want to set on this project yet. I'm thinking somewhere around 3k before drives. But that has room and time to move in both directions. I'm not really set on anything yet. I want to be well informed so I know the costs of the different features so I can decide what's the best price/features+performance ratio for me.


In summary:

Storage solution that's scalable (8+ tb).
Easy file sharing over network

Optimized for:

* One user
* High performance reads
* Large files


Need articles/reviews...

* explaining different filesystem's performance & issues in large storage arrays (esp NTFS).
* benchmarking (or at least explaining) SAN/DAS/NAS/other storage systems against each other at similar price points & setups.
* comparing rackmount server enclosures (particularly 20u(ish) size)
* on other random server equipment


Thanks for any help, advice, or insight you can give.

 

[JackMDS]

May be this can Help.

http://forums.anandtech.com/me...ght_key=y&keyword1=whs

http://forums.anandtech.com/me...ght_key=y&keyword1=whs

 

[Gast]

Thanks, but I don't think windows home server will work for my needs. WHS uses alot of space to provide drive failure backups. It basically acts as a psudo raid1 w/o the performance benefit. I'm guessing the performance is along the lines of cheap NAS, which doesn't provide the high performance reads for a single user that I want.

However it is an option I had not considered. Thanks for the links.

 

[ViRGE]

Yeah, you won't get the reads you want out of WHS. At best you get reads on par with normal reads from a single hard drive, other times it can be lower if the OS is busy doing something else such as a chkdsk. It's not slow, but if you're looking for RAID0-speed reads, you won't find it with WHS.

 

[MerlinRML]

Originally posted by: Gast
* explaining different filesystem's performance & issues in large storage arrays (esp NTFS).
* benchmarking (or at least explaining) SAN/DAS/NAS/other storage systems against each other at similar price points & setups.
* comparing rackmount server enclosures (particularly 20u(ish) size)
* on other random server equipment

I'm afraid I've got no articles to link to. I can point you at Chenbro, AIC/Xtore, and Supermicro for decent rackmount enclosures. You're going to have trouble finding a 20u case, though, as the push has been for higher density, and frankly if you're looking at that much stuff you're going to want to move to a SAN and add modular storage in 2u-4u enclosures. You might check out the 2u and 3u enclosures that can handle anywhere from 8-16 drives.

I will also point out that just about any modern SATA drive can sustain 60MB/sec (excepting the power-saving drives), and a gigabit network can theoretically handle 125MB/sec. As such, your network will be your bottleneck long before your drives are. The same can be said about your filesystem question.

NAS is just storage that doesn't actually connect to your machine, but rather is being made available to you from a file server somewhere. Network attached storage just means you can't access the physical disks (change partitioning, run diagnostics, etc) but can access the file system and the data stored on it. Some other system is actually handling all the physical access to the disks. Upside is centralized storage. Downside is network bottleneck, lack of physical disk access, and requires some management for security/user access, requires a "host" machine.

DAS just means the storage you're most likely used to working with, as it is directly connected to one machine, as opposed to going through some kind of switch and being accessible to multiple machines. You can access every piece of the disks from this machine. The server that hosts the NAS has DAS storage. Upside is that it is high performance, easy to manage, and usually is the lowest cost. Downside is that it is only usable by one machine unless that machine doubles as a network host, and depending on your implementation can be very limited scalability.

SAN refers to a common pool of storage that can be accessed by multiple machines and the access is controller through software. There is usually a layer of abstraction, so you're not talking to physical disks directly, but the computer is talking to something that looks and acts like a physical disk. Upside is very centralized, very scalable, and high performance. Downside is highest cost, lots of management for access and maintenance, additional software, and most difficult to diagnose problems.