Question Long term data archival

[Red Squirrel]

What's everyone do for long term archival of data? Right now pretty much all my data is live on my NAS, and I have a lot of old early 2000's stuff on CDs. I know CDs can go bad but they do make archival grade ones so that may be an option. I'd probably do DVDs or Blurays now days.

I also have cold backups of all my data on hard drives. But reality is if the data on the live NAS goes bad in any way such as bit rot it will just eventually reflect in the backups. There is also the fact of accidentally deleting files without realizing or what not then that also gets reflected in the backups.

So I need to think of a better way, and ideally a read only way. Curious what everyone here does. I'm thinking things like pictures, videos, documents, code, various projects and that sort of thing.

 

[Muadib]

If you can handle the cost, then you want an LTO tape drive.

 

[WelshBloke]

 

[Red Squirrel]

If you can handle the cost, then you want an LTO tape drive.

I often look at LTO drives and get sticker shock, but maybe that would be the best bet. The tapes are not that much more expensive than hard drives so really it's just the drive that's expensive. LTO6 seems to be the sweet spot right now, LTO7 drives are like over 5 grand lol. LTO6 seems to be in the 1-2 grand range if I buy used on ebay. Still a lot of money though...

 

[dlerious]

I use M-disc blurays. I use dvdisaster to create an ECC image.

 

[Insert_Nickname]

50GB Blu-ray. Cheap, cheerful and do the job.

Further, they're read only. So nothing can mess with the data once written.

 

[igor_kavinski]

50GB Blu-ray. Cheap, cheerful and do the job.

Oldest backed up Bluray disc you have? Just want to get an idea of how long Bluray disc may last.

 

[Insert_Nickname]

Oldest backed up Bluray disc you have? Just want to get an idea of how long Bluray disc may last.

Oldest phase change media I have is a DVD-RAM (no, not a typo) burned in '98 I think. Still perfectly readable. Still have some ~'95 burned Kodak Gold CDs which are just as good as when they where burned. To use a cliché; they don't make 'em like that anymore. (MDISC should still be a good choice)

Blu-ray should last as long, provided they're not the LTH variety. Those use an organic recording layer, so are a bad choice for long term archiving.

 

[igor_kavinski]

Blu-ray should last as long, provided they're not the LTH variety. Those use an organic recording layer, so are a bad choice for long term archiving.

What's the easiest way to identify those? The recordable variety or even normal ones too?

 

[Muadib]

I often look at LTO drives and get sticker shock, but maybe that would be the best bet. The tapes are not that much more expensive than hard drives so really it's just the drive that's expensive. LTO6 seems to be the sweet spot right now, LTO7 drives are like over 5 grand lol. LTO6 seems to be in the 1-2 grand range if I buy used on ebay. Still a lot of money though...

Yes, I agree on the cost issue. I use M-disc currently, but one day LTO will be mine.

 

[Insert_Nickname]

What's the easiest way to identify those?

You can check the media code on this list;

https://blu-raydisc.info/licensee-list/discmanuid-licenseelist.php

Doesn't help when you buy of course, but the easiest way is to only buy 50GB or BDXL discs. Those are always phase change.

The recordable variety or even normal ones too?

Pressed discs are something else entirely. They'll likely outlast all of us.

 

[Red Squirrel]

I'm looking deeper at LTO, I think I was being greedy wanting to go with a higher generation, but if I go with LTO5 that gives me 1.5TB per tape, and the costs are not all that bad.

Found this listing for example:

HP LTO-5 ULTRIUM 3280 SAS EXTERNAL TAPE DRIVE EH900A 587238-001 | eBay

Compatible - 430066-001 - SAS Cable - External mini SAS (SFF8088) to (SFF8088). Compatible - SAS CABLE - SFF-8644/SFF-8. SAS 6GB HBA ADAPTER. THIS LINK for LTO6 EXTERNAL DRIVE include cables.

www.ebay.ca

A bit over $657 shipped. That $100+ shipping fee is still a big sting though, I find ebay shipping is always ridiculous. Can't seem to find many sites here that sell that sort of thing otherwise I'd just buy from a retailer as shipping is usually much more reasonable.

Just mostly humoring the idea at this point anyway. Maybe for now I will stick with just having multiple copies on HDDs.

 

[Muadib]

I'm looking deeper at LTO, I think I was being greedy wanting to go with a higher generation, but if I go with LTO5 that gives me 1.5TB per tape, and the costs are not all that bad.

Found this listing for example:

HP LTO-5 ULTRIUM 3280 SAS EXTERNAL TAPE DRIVE EH900A 587238-001 | eBay

Compatible - 430066-001 - SAS Cable - External mini SAS (SFF8088) to (SFF8088). Compatible - SAS CABLE - SFF-8644/SFF-8. SAS 6GB HBA ADAPTER. THIS LINK for LTO6 EXTERNAL DRIVE include cables.

www.ebay.ca

A bit over $657 shipped. That $100+ shipping fee is still a big sting though, I find ebay shipping is always ridiculous. Can't seem to find many sites here that sell that sort of thing otherwise I'd just buy from a retailer as shipping is usually much more reasonable.

Just mostly humoring the idea at this point anyway. Maybe for now I will stick with just having multiple copies on HDDs.

You have me looking now. Went to ebay, and the price for LTO-6 drives are doable. However, I don't have a sas external port to plug the drive into. I need to see if a sas to usb 3 adapter exists.

 

[Red Squirrel]

You have me looking now. Went to ebay, and the price for LTO-6 drives are doable. However, I don't have a sas external port to plug the drive into. I need to see if a sas to usb 3 adapter exists.

I'm finding lot of FC ones that are cheaper and more available, but are made to be housed inside auto loaders. I kinda wonder if they work stand alone. If I was to hookup a FC one to a fibre channel card would it show up in the OS or does it require more work? I have a bunch of FC cables and cards already, and I did have SAN enclosures hooked up at one point and each drive would show up individually in a /dev device so I'm thinking it might work the same. Very tempting.

What got me suddenly interested in this is I recently lost my cat and I have so many pictures of her over the years, but also other memories, going back almost 20 years. They are all on spinning disks, and a couple backups, but I feel it's just not enough. It would pain me to lose all of that! The pictures alone are around 300GB worth so optical media is doable, but would be tedious.

For data like pictures, that never changes after the fact, it makes sense to just archive it offline.

 

[igor_kavinski]

The pictures alone are around 300GB worth so optical media is doable, but would be tedious.

I think you could easily reduce the size to 100GB if you batch convert them to webp format.

 

[Insert_Nickname]

I think you could easily reduce the size to 100GB if you batch convert them to webp format.

Depending on source format JPEG XL might be worth a look. It can cut 15-20% off regular JPEGs, and is completely reversible.

 

[AnitaPeterson]

A long time ago, I learned the hard way that you need back-ups.

My method is somewhat primitive, but efficient:

Every HDD with important information is copied on another, identical-sized HDD, which is then labeled and dated, wrapped in an anti-static bag, enclosed in a little protective box and placed in a storage cabinet that also doubles as a Faraday cage.