Need large, cheap data archival (min. 8TB)

[Ken g6]

I record way too many TV shows, and I never get around to either editing out commercials and compressing them or deleting them. So I wind up with lots of archived data filling my drives, that I do want to access, but I don't need to access quickly. A Seagate SMR drive would seem ideal, except that I want the data stored reasonably reliably, and I see a lot of failure reports with this drive. I've also had too many bad experiences with external HDDs to trust them.

So, experts, what do you suggest? Are tape drives in a similar price range? Should I just get a BD-RW drive and a bunch of disks? Or is the above Seagate more reliable than it looks?

Edit: Oh, I suppose I could do online storage, but my Internet's fairly slow.

 

[aigomorla]

Raid Z using refurbished HGST's on FreeNAS or any free NAS Software on old hardware.

Just boot the system when you need to do your backups, and turn it off when your done.

66 dollar per drive:
https://www.amazon.com/HGST-Ultrast...=UTF8&qid=1497033879&sr=8-2&keywords=3TB+HGST

You would need at least 5 for Raid-Z. (well you can do Raid-Z with 3, but u will not meet your 8TB quota, and 4 drives is limited to Raid-Z2.)

330 dollars in drives... with 1 redundancy for failure for approx 12TB of storage.

Get old Hardware your not using for FreeNAS.
You will probably need about 4-8GB of Ram as well....

I will never trust Seagate again... ive lost 5 "NAS" drives from them. each of them dying 1-2 months after each other upon expiration of warrenty.

Im a full supporter of HGST and WD now...
Even then i will always recommend redundancy, and backups with redundancy.

 

[Elixer]

I haven't seen a cheap consumer tape drive unit, so, that is out.
BD-RW have come down in price, but, those have there own issues, and the storage per disc is still too low, so, I pass on that as well.

So, we are back to HDs. You could go to target and find clearanced drives, and shuck the case, or, I suppose https://www.amazon.com/Seagate-Expansion-Desktop-External-STEB8000100/dp/B01HAPGEIE/ which seems to be close to the lowest price it has been https://camelcamelcamel.com/Seagate-Expansion-Desktop-External-STEB8000100/product/B01HAPGEIE but, it is a Seagate and I really, really don't like their warranty BS they are pulling these days, not to mention the overall product reliability experience I had in the past with them. I haven't yet used this specific 8TB line though, so, jury is still out (http://www.storagereview.com/seagate_archive_hdd_review_8tb). Oh, and these are supposed to be the same internal drive that you were looking at before (SMR tech).

I have been wanting to get the HGST Helium drives, but, those are more expensive, $100 more.
Anyway, to make this rambling of mine stop, I would look for a HGST drive if I was gonna buy a HD today as reliability as the #1 concern, though, keep in mind that to really have true reliability, you need more than 1 unit, and backup to that as well..

 

[Shmee]

I like aigo's idea, but would go with WD reds. Not as cheap, but then I would not want to cheap out on this.

 

[seagate_surfer]

Hi Ken,

we would like to add one important thing that hasn't been mentioned yet:
the choice of your backup devices should be made in regards to the scenario you would like to use them in. It is a bit unclear if you are considering a NAS device for your backups or not...

Independent from picking a certain manufacturer, in a 24/7 environment you should use a NAS HDD ("IronWolf" in the Seagate world) whereas in a regular PC/Backup scenario a desktop drive ("BarraCuda" in the Seagate world) is sufficient.

Hope that helps...

 

[Ken g6]

Yeah, I think this thread kinda went off the rails. I have a 5TB fast drive and a 4TB green drive in my computer currently. The goal was to transfer data from the 4TB to an 8TB drive, and have more space. I wasn't thinking about RAID or backups. The video I record often isn't worth backing up - either I compress it before backing it up, or it'll be rebroadcast. A lot of this is probably stuff I'll never watch but can't quite throw away either.

 

[Oyeve]

Dude, 6-8 TB HD and an enclosure. Simple. People and their NAS and Raids. pishaw,

 

[tcsenter]

Yep, find Grade A off-lease/used/refurbed 2nd or 3rd Gen Core SFF or slim tower PC with Gigabit Ethernet, USB3.0, that will house two 3.5" drives (even if you have to buy one of those adapters to mount in a 5.25" bay) for ~$150 delivered (+/- $20). Use whatever OS it comes with (W7, 8.1, 10), or install open source alternative, create RAID1 mirror using two HDD. Way more CPU/RAM than you'd get from spending $250+ on a diskless NAS box or RAID enclosure.

 

[VirtualLarry]

Yes, but unless he wants his "server" to actually do trans-coding, then he would most likely be better off with a NAS. Less power, less heat, less noise, less hassle, and physically smaller.

Not necessarily cheaper, but you can get decent 4-bay QNAP NAS units (TS-431) for under $300 on sale or on ebay.

Sure, performance is likely not going to be better than a dedicated PC, but just for file-storage, that's kind of overkill, unless you're building a "real" server (unRAID, etc.), and filling a tower case with drives. The bottleneck is likely to be the network anyways.

 

[Charlie98]

Yeah, I think this thread kinda went off the rails. I have a 5TB fast drive and a 4TB green drive in my computer currently. The goal was to transfer data from the 4TB to an 8TB drive, and have more space. I wasn't thinking about RAID or backups. The video I record often isn't worth backing up - either I compress it before backing it up, or it'll be rebroadcast. A lot of this is probably stuff I'll never watch but can't quite throw away either.

I don't think any sort of NAS, RAID, etc is required... what you want is quite simple. Why don't you just add another 4-5TB drive to the 4TB one (assuming you have the room?) Or else it's just spend your money and take your chances with either a less expensive drive, or pony up the money for the HGST.

I will agree with a backup of some sort, however. I've got about 4.5TB of video stored on my HTPC's main 5TB drive, and while it wouldn't be earth shattering if it failed, it would cost me more in my time to re-rip and reload a new drive... so I have what were the old main drives (2x 3TB drives) as manual backup. You have to decide whether or not you are willing to lose that data... no matter which single drive you would purchase... without a backup.

 

[Malogeek]

IMO you should focus more on offline nightly processing of these recorded shows. Most likely they're being recorded in ancient MPEG2 which will give you very large files unnecessarily. Even if they're h264 you're still wasting space. You should think about setting up something like Handbrake scripts to run at night and compress everything to h265. Goodbye capacity problems.

 

[sdifox]

get two external hdds, duplicate copy of the content and call it a day.

or just torrent whatever show you want to watch instead of record and convert. internet is your backup

 

[Malogeek]

get two external hdds, duplicate copy of the content and call it a day.

That seems overly complicated when you could instead buy an external 2-drive that does mirroring automatically.

 

[sdifox]

That seems overly complicated when you could instead buy an external 2-drive that does mirroring automatically.

two enclosures safer than one

 

[WhoBeDaPlaya]

Get a WD MyBook / EasyStore 8TB for ~$180 at BestBuy.
Drive inside is a 5400rpm version of the HGST Ultrastar He8, so it's pretty much bulletproof from a reliability standpoint.
Bonus points for being PMR rather than sh*** SMR.

Dude, 6-8 TB HD and an enclosure. Simple. People and their NAS and Raids. pishaw,

When I moved from IA to TX, I FedEx-ed my HTPC sans HDDs, and drove the HDDs down.
What's that old saying again? Never underestimate the bandwidth of a car filled with HDDs
(latency is a killer though, and that is one JUMBO frame!)

 

[Ken g6]

IMO you should focus more on offline nightly processing of these recorded shows. Most likely they're being recorded in ancient MPEG2 which will give you very large files unnecessarily. Even if they're h264 you're still wasting space. You should think about setting up something like Handbrake scripts to run at night and compress everything to h265. Goodbye capacity problems.

That potentially sounds like a very good plan. The main problem with it is commercials. I have a couple of programs to edit MPEG2 losslessly to remove commercials. This is off-topic, but got anything like that for h264 or h265?

Oh, and h265 isn't a great option because almost nothing I have will play it back.

 

[Malogeek]

That potentially sounds like a very good plan. The main problem with it is commercials. I have a couple of programs to edit MPEG2 losslessly to remove commercials. This is off-topic, but got anything like that for h264 or h265?

Why not just process the files after they've been edited for commercials then? Use Handbrake CLI to convert from MPEG2 to h264 and you'll have huge benefits for file size.

Oh, and h265 isn't a great option because almost nothing I have will play it back.

Fair enough. Even h264 will be of great benefit for you though. I don't know what you're using for playback so can't comment regarding h265.

 

[Ken g6]

Why not just process the files after they've been edited for commercials then?

I do. But I don't edit commercials out frequently enough, so files build up, and I'm running out of space.

 

[sdifox]

I do. But I don't edit commercials out frequently enough, so files build up, and I'm running out of space.

Any particular reason you cannot torrent the shows already compressed and sans commercials?

 

[rchunter]

This is what I use for cutting commercials from my dish network captures.. Everything gets saved to h264 mkv.

https://www.videoredo.com/en/Products_TVSuite_V5.html

 

[Ken g6]

Any particular reason you cannot torrent the shows already compressed and sans commercials?

For starters, it's illegal. I could report you for just suggesting it.

This is what I use for cutting commercials from my dish network captures.. Everything gets saved to h264 mkv.

Requires Windows. Did I mention I run Linux? I could use it in my Windows VM...but this is getting complicated.

Why don't you just add another 4-5TB drive to the 4TB one (assuming you have the room?)

This is probably the most practical answer.

Get a WD MyBook / EasyStore 8TB for ~$180 at BestBuy.

Another potentially good answer. Especially for backups. Although a drive that requires helium makes me nervous that it will lose it at some point. Not, I suppose, within its practical lifetime, though.

 

[WhoBeDaPlaya]

Another potentially good answer. Especially for backups. Although a drive that requires helium makes me nervous that it will lose it at some point. Not, I suppose, within its practical lifetime, though.

I was apprehensive at first for the same reason, but with other options being SMR, it was Hobson's choice for me.
*knock on wood* My 32x He8 drives have been purring along just fine 24/7 for a good, long while now.

 

[Malogeek]

I do. But I don't edit commercials out frequently enough, so files build up, and I'm running out of space.

Irrespective of whether they've been edited or not, you can still process them automatically via CLI in linux on a nightly basis to at least get them in h264 format.

 

[sdifox]

For starters, it's illegal. I could report you for just suggesting it.


Requires Windows. Did I mention I run Linux? I could use it in my Windows VM...but this is getting complicated.

This is probably the most practical answer.


Another potentially good answer. Especially for backups. Although a drive that requires helium makes me nervous that it will lose it at some point. Not, I suppose, within its practical lifetime, though.

How is any more illegal when you are just skipping the editing and encoding step?

 

[Ken g6]

How is any more illegal when you are just skipping the editing and encoding step?

Because time-shifting is legal for personal, non-commercial use. Torrenting is not, since it involves distribution. Weird but true.