Can you have too much storage?

[rchunter]

If you have a huge array just sitting mostly empty yeah. That's why I like unraid so much, you can add drives as you need them.

 

[smakme7757]

What is an External disk? Do you mean like a USB or Esata drive or is External some sort of network cloud based storage setup I'm not aware of?

I just use standard 3.5" cheap high capacity drives. And connect them with a dock.

 

[smakme7757]

I've come to realize that you can either:

1. Keep spending money, expanding storage, spending money ... forever ... trying to store all the crap that you really don't need.

or

2. End the cycle, figure out what you really do need. Keep things manageable.


I'm not going to tell anyone else that they absolutely don't need 175TB of gay S&M porn. Or that they don't need 2800 Blu-ray rips of movies that they'll never watch more than once in their lifetime ... but at some point it's a pretty idiotic waste of money, time, space, and electricity.

Yepp, agree with at.

 

[hasu]

Unless you are very systematic, storage can get really "messy" -- multiple copies all over. I have about 20TB mostly in Raid-1 (effectively under 10TB), mostly backup of backup of backups! Additionally I have about 6TB non-raid on HTPC with some very old recorded OTA tv-programs! Each time I buy a higher capacity drive, I copy over most files from the earlier drive to the new one, resulting in multiple drives in the archive with more or less the same old data. When I tried to retrieve some old pictures I found many corrupted jpeg files, and that prevented me from reusing those old drives. To make things worse I have more "backups" in portable raid drives as well (another 10TB max mostly 2.5" drives in CineRaid enclosures).

Edit: If really sort things out, I can squeeze the important stuff to under 1 TB (mainly family pictures and videos plus some important documents).

How do you guys handle such massive amount of data? How do you manage to have various drive sizes in a raid configuration?

 

[greenhawk]

How do you guys handle such massive amount of data? How do you manage to have various drive sizes in a raid configuration?

I have currently settled on the following setup after trying a few different approaches over the years. I aim for easy recovery from drive failures over anything else.

I use Stablebit Drive pool (paid) to collect all drives into one drive letter. It has the benefit of allowing a drive to be removed and the files on it read without anything special being done.

https://stablebit.com/DrivePool

I use file scavenger (paid) to recover files and such if a drive plays up (generally file allocation table issues)

http://www.quetek.com/prod02.htm

I use double killer to keep duplicate files under controll. Paid as the free version only works on files under 2GB.

http://www.bigbangenterprises.de/en/doublekiller/

And going forward I am looking to use something like Snapraid (file level raid) to protect from a drive failing (for non changing data, works with drive pool)

http://snapraid.sourceforge.net/index.html

 

[Red Squirrel]

When building a raid array I always plan for a certain disk size and use same size for an array. It's just cleaner. I try to stick to same make/model too. When I expand in the future I try to get the same make/model or something faster.

Currently I have two raid 10's and a raid 5. The raid 5 uses 1TB drives so I will eventually retire it and probably make another raid 10 with like 4TB drives or something. Or might just use the slots to expand the other two raids. One of the raid 10's only has 4 3TB drives so that's a good candidate for expansion. Though last time I tried expanding a raid 10 it would not let me but there was something messed up with that array. in fact, I STILL can't kill that array, there's 0 drives in it and it still wont let me take it offline. Kind of a pain. I had tried to make a degraded raid 10 with 7 drives, figuring it would make a 8 drive array with one shown as removed, but instead it actually made a raid 10 with 7 drives, somehow. So a note for next time, always make raid 10's in pairs, as it wont actually treat 1 drive as missing like you'd think.

 

[XavierMace]

Mine is split between a 12x2TB drive RaidZ2 pool and a 4x4TB drive RaidZ1 pool. Second pool is just used for snapshots/backups of important data.