Question I need to store 60G safely.

Lost_in_the_HTTP

Diamond Member
Nov 17, 2019
6,362
3,729
106
I have a folder of music that is just under 60Gb. It's taken a buttload of work and I don't want to lose it. Right now, it's on an SSD in daily use that I play the music from. I keep weekly Acronis backups on a second drive so at least i have that. Those backups are full disk though. I'd like another direct copy of just the folder itself, so I don't need Acronis to read it.

I see USB thumb drives larger than that available, but I've read some things about them not being stable. Or at least some of them.

I can't upload anywhere since I havve a slow-ish DSL connection and upload is only a couple of Mb which means it would take forever.

Suggestions?
 

VirtualLarry

No Lifer
Aug 25, 2001
53,283
7,705
126
Suggestions?
Get a NAS, a 4-bay prosumer unit, slap in four 14TB HDDs, that's 3x14TB net storage under RAID-5, or 42TB.

Buy a few 14TB external USB3.0 drives, and plug them in in a rotating fashion, and backup/rcopy the NAS's storage to the external HDD. Keep those as "Cold Storage", should anything happen to the NAS or it's data.
 
  • Haha
  • Like
Reactions: Muadib and Shmee

UsandThem

Elite Member
Super Moderator
May 4, 2000
15,691
6,665
146
I have a folder of music that is just under 60Gb. It's taken a buttload of work and I don't want to lose it. Right now, it's on an SSD in daily use that I play the music from. I keep weekly Acronis backups on a second drive so at least i have that. Those backups are full disk though. I'd like another direct copy of just the folder itself, so I don't need Acronis to read it.

I see USB thumb drives larger than that available, but I've read some things about them not being stable. Or at least some of them.
How about storing it on another small hard drive? I keep two local copies of all of my "can't lose" data. USB thumb drives are not what I would personally consider a reliable option. They are convenient for moving data to other PCs at times, but they are all about convenience and not reliability.
I can't upload anywhere since I havve a slow-ish DSL connection and upload is only a couple of Mb which means it would take forever.
Many of us old timers lived through the days of dial up modems, and I often had to download things like game patches overnight since they took so long.

Even if it's a slow upload, you really only have to do it once. You click upload, and after that you don't have to do anything else. Having a copy of "can't lose" data stored somewhere outside of your house is a wise decision as if you had something like a fire, you could lose your only copies.
 

Insert_Nickname

Diamond Member
May 6, 2012
4,586
1,166
136
I have a folder of music that is just under 60Gb. It's taken a buttload of work and I don't want to lose it.
60GB? An external HDD will do. Or external SSD if you prefer.

You can also keep a few copies on USB drives for extra portability. They're cheap and cheerful, so there is nothing wrong with buying a couple of 128GB drives.

Many of us old timers lived through the days of dial up modems, and I often had to download things like game patches overnight since they took so long.
Merely overnight? You where privileged... :D Could take a bloody day with my old 33.6 modem.
 

mikeymikec

Lifer
May 19, 2011
15,245
5,387
136
I see USB thumb drives larger than that available, but I've read some things about them not being stable. Or at least some of them.
However you've got one backup already and you're talking about a secondary backup. I'd just get a flash drive as big as you wish and copy / robocopy the files on to it.

The fact of the matter is you've got one backup which is a million times better than none, so for that to fail you need something horrible to happen to an SSD (unlikely), then your first backup fails (the probability of both of those happening together is very low indeed), and for your memory stick to also get corrupted somehow. The most likely way for this to happen is for something catastrophic to happen to your house. The question is, do you want to go as far as having an off-site backup? When my parents were alive, I had considered stashing a spare and encrypted hard drive in their house with an occasional backup I'd run on it (they lived around the corner from me), for example.

The other thing I'd do if I were you is to keep a record of when you've run the various types of backup, and if some types of backup only contain some bits of data (not a complete backup of everything you value), keep a note of what ought to be on where.
 

Lost_in_the_HTTP

Diamond Member
Nov 17, 2019
6,362
3,729
106
The fact of the matter is you've got one backup which is a million times better than none, so for that to fail you need something horrible to happen to an SSD (unlikely),


....
 

mikeymikec

Lifer
May 19, 2011
15,245
5,387
136

....
"unlikely" does not mean "impossible". In my line of work I've seen two SSDs fail, one within warranty.

My advice was based on you wanting backups on a tight budget.
 

MalVeauX

Senior member
Dec 19, 2008
644
167
116
Hi,

Realistically avoid any cheap memory based storage plan, like a thumb drive or something. Either use a quality SSD (will last longer than a HDD) that is offline and cold and refresh it periodically with a new one (pick a yearly interval), or use optical media (like bluray or mdisc bluray). Focus on having 3 physical copies that are separated. You could do SSD that is live/active that you use daily. Periodically back it up to an external SSD that is then unplugged and offline/cold and put it in a safe storage container of some kind. And for a 3rd, maybe burn a bluray disc or two and keep them somewhere else for long term storage. Inexpensive way to do it since you're talking 60Gb which is minimal these days.

For tight budget and truly long term storage, I would use optical media. Your file needs are small enough to fit on 2 bluray discs. Otherwise, 160~250Gb SSD that are fair quality are $30~50 and those are dirt cheap. You could get two and separate them physically and keep them offline/cold for pennies. They will outlast a hard disc.

Very best,
 

mindless1

Diamond Member
Aug 11, 2001
6,881
951
126
I use redundant USB flash drives for backup of certain folders (in this general storage capacity range) on the system's internal SSD (2nd SSD in system, not the OS SSD so OS partition backups don't keep including bulk/static files), and FreeFileSync + RealtimeSync companion app to synchronize them. I could set my partition backup to exclude folders but I have the 2nd SSD anyway so that's my plan.

Plug in a USB flash drive, Realtime sync updates the USB flash drive. Remove flash drive, plug in 2nd redudant flash drive, then Realtime sync updates it too. If there is a problem I'll notice it the next time I plug in the flash drive and still have the other two+ copies of the data.

USB flash drives, if flimsy design can be damaged, particularly the stress of plugging/unplugging or bumping while plugged in the USB socket, but otherwise I haven't had any reliability issues over several years of regularly scheduled backups, except certain makes and models known to overheat, which I just don't use for this in the first place.

Granted, data grows, so every few years I upgrade the capacity. Oldest flash drive I still have in the rotation is 7 years old... so by upgrading the capacity at this interval, I'm also avoiding the bit rot issue mentioned below, of leaving things on flash storage too long.

USB flash drives do need their contents refreshed (same files rewritten to them even if the files didn't change) every decade or so, or more often the more you exhaust their write cycles but I doubt you'll come near exhausting write cycles with this kind of storage use.

Besides, see above, I use two of them, and for certain things I also have on a 3rd and/or mSD card leftover from a phone, that can be read on the go with a phone. You put a file on the storage volume that determines what is backed up to it for Realtimesync to use to know this, so each USB flash drive can hold different things.

Certainly USB flash drives are slower, but I don't find it a big deal with my setup to just plug it in, let Realtimesync do the work, and unplug when it's done. At the same time it's better not to pick very slow drives, but music tends to be a few MB each track so not as bad as a cheap USB drive's drastic slowdown copying 1,000,000 x few-KB each sized files.
 
Last edited:

thecoolnessrune

Diamond Member
Jun 8, 2005
9,620
519
126
Just spend $6 / month on a Backblaze account and upload your stuff to the cloud. Seriously. You do it one time. If my parents could back up both of their laptops on their 768kbps DSL connection you can survive that one time backup. Putting it in the cloud will be more safe and effective than any extra flash drive / HDD Swap / or whatever methodology that you come up with that puts your data locally and thus still able to be knocked out by a simple fire or flood.
 
  • Like
Reactions: Ajay and coercitiv

mindless1

Diamond Member
Aug 11, 2001
6,881
951
126
^ What if somehow you get locked out of your cloud account, or the service goes under, or gets hacked, or you don't have internet access due to this fire/flood/etc catastrophic scenario, or you become a mass murderer and all your accounts are seized by law enforcement? ;)

A cloud is certainly another good, redundant way to store data, and I would do that for anything so important that not only you but the rest of your family needs utmost data retention possible for something more than just music files, but I don't consider it as reliable as a flash drive I can put in my pocket (and in fact do) + mSD card for my phone, so would never have it as the only backup. Any flash drive is flood-proof if you simply dry it out before use, but fire, if it's hot enough to destroy what's in my pockets, my being dead would limit my need to access the music thereafter. ;) So far, fires and floods have only killed me a handful of times over many years so they seem like worst case scenario opposed to storing a few music files.

Just sayin'... the moment you involve a 3rd party for something no longer under your control, the most you can do if things go wrong and you can't get your data back, is sue them. It might be hard to quantify financial loss on music files... unless you run a recording studio and keep good records.

I'm scratching my head wondering how people came to the conclusion that a decent USB flash drive isn't the most reliable backup possible. Yes things can go wrong, but also things can go wrong with a cloud, or an SSD (higher complexity and mechanical connection issues can make them less reliable in portable uses), or optical media. Even the old safe bet of tape storage in a fireproof, sprinkler/gas suppression system vault, is not foolproof if you don't have exclusive access and control over the site.

My point is, the fewer variables there are, especially those you can't control, the more you empower yourself to ensure your data is safe. Do put data on a cloud if it suits you but we're talking about under 60GB, which is about $8 worth of flash media storage, so a second point of redundancy for years, costs less than two month worth of a $6/mo. Backblaze account. Then again I advocated redundant USB flash drives so rather than less than two months, call it 4 months.

IMO, nothing beats having a local copy of your data. Expand your off-site options from there.
 
Last edited:

mikeymikec

Lifer
May 19, 2011
15,245
5,387
136
I've been thinking about the fact that my backup strategies don't yet involve off-site backups in any way, and one system I'm mulling over is a hybrid option for my most essential data: Back up to my phone into an encrypted veracrypt container. My phone has an SD card slot as well, so I can increase its storage if need be.
 

Insert_Nickname

Diamond Member
May 6, 2012
4,586
1,166
136
My phone has an SD card slot as well, so I can increase its storage if need be.
Put it on an SD card in your phone? I'd say that's a legit strategy, since you're mostly carrying your phone around with you presumably. SD cards have gotten rather large, so size shouldn't be a problem.

Just remember SD cards aren't reliable, so you'd need additional copies to be on the safe side.
 
  • Like
Reactions: mikeymikec

igor_kavinski

Platinum Member
Jul 27, 2020
2,544
1,304
96
*UNUSED* INTEL SSD 535 Series 180GB SSDSC2BW180H601 - FREE SHIPPING | eBay

Buy two of these. Put them in an external USB enclosure. Would cost you less than $100. Why? Intel's SSD reliability is legendary. Their annualized failure rate is less than 1%. They have tested their SSD reliability using a frickin' particle accelerator. This is what I would do to safeguard my precious data. At least once a month, refresh them in read-only mode using this:

DiskFresh - Refresh Hard Disk Signal (puransoftware.com)

To guard against the possibility of bit rot (cosmic ray bitflip is a bitch!), create a digest with this software:

ExactFile | Making sure that what you hash is what you get.

Then verify the digest once a month. It's multithreaded so more CPU cores should help. If you get an error saying it cannot find the files, make sure the base folder under which all the files are stored, is correct and then run it again.
 

Attachments

thecoolnessrune

Diamond Member
Jun 8, 2005
9,620
519
126
^ What if somehow you get locked out of your cloud account, or the service goes under, or gets hacked, or you don't have internet access due to this fire/flood/etc catastrophic scenario, or you become a mass murderer and all your accounts are seized by law enforcement? ;)

A cloud is certainly another good, redundant way to store data, and I would do that for anything so important that not only you but the rest of your family needs utmost data retention possible for something more than just music files, but I don't consider it as reliable as a flash drive I can put in my pocket (and in fact do) + mSD card for my phone, so would never have it as the only backup. Any flash drive is flood-proof if you simply dry it out before use, but fire, if it's hot enough to destroy what's in my pockets, my being dead would limit my need to access the music thereafter. ;) So far, fires and floods have only killed me a handful of times over many years so they seem like worst case scenario opposed to storing a few music files.

Just sayin'... the moment you involve a 3rd party for something no longer under your control, the most you can do if things go wrong and you can't get your data back, is sue them. It might be hard to quantify financial loss on music files... unless you run a recording studio and keep good records.

I'm scratching my head wondering how people came to the conclusion that a decent USB flash drive isn't the most reliable backup possible. Yes things can go wrong, but also things can go wrong with a cloud, or an SSD (higher complexity and mechanical connection issues can make them less reliable in portable uses), or optical media. Even the old safe bet of tape storage in a fireproof, sprinkler/gas suppression system vault, is not foolproof if you don't have exclusive access and control over the site.

My point is, the fewer variables there are, especially those you can't control, the more you empower yourself to ensure your data is safe. Do put data on a cloud if it suits you but we're talking about under 60GB, which is about $8 worth of flash media storage, so a second point of redundancy for years, costs less than two month worth of a $6/mo. Backblaze account. Then again I advocated redundant USB flash drives so rather than less than two months, call it 4 months.

IMO, nothing beats having a local copy of your data. Expand your off-site options from there.
I'm scratching my head wondering how you come to this conclusion at all. We're not even talking a single backup point here. We're talking a 2nd backup following a standard 3-2-1 strategy. Cloud storage is statistically far more resilient to issues than any single backup disk strategy you could come up with as a home user. Like again, you say you're scratching your head at this, but how on earth is basic statistics a head scratcher? A thumb drive more resilient than a cloud? The notion is just honestly absurd. Your what ifs that you base it on are equally absurd.

If you get locked out of your cloud account - You have another backup.
If the service goes under - You migrate your backup set to another service, just like you manage your other backup media.
It gets hacked - They restore from their own backups.
You don't have internet access due to fire / flood / etc. - You likely lost your thumb drive backup as well. It's pretty preposterous in this scenario to think you have everything you need in this scenario including something to play music on if you only just had your precious thumb drive with music on it, but a stop at your local library or McDonalds parking lot is just a bridge too far.
Your accounts are seized by law enforcement - You have another backup.

The reality is that thumb drives, SD Cards, etc. are all bargain basement devices with terrible reliability in general. Yes things go wrong in cloud, which is why you pay them to have the resiliency to deal with it.

Coming to the conclusion that a USB Flash Drive is more resilient than cloud is indeed a head scratcher. You need only look at the MTBF (if they even publish it) to reach the correct conclusion.
 
  • Like
Reactions: coercitiv

mindless1

Diamond Member
Aug 11, 2001
6,881
951
126
I'm scratching my head wondering how you come to this conclusion at all. We're not even talking a single backup point here. We're talking a 2nd backup following a standard 3-2-1 strategy. Cloud storage is statistically far more resilient to issues than any single backup disk strategy you could come up with as a home user.
Statistics include people who can't DIY so I have to assume the cloud storage customers have different liabilities and capabilities. It also doesn't rule out the getting locked out or service suspension, whatever. Mistakes happen. Getting your data back eventually, slowly, isn't as convenient to me as always having a few copies lying around. That was the thought years ago but it has proven to be reliable over actual use all that time.

Like again, you say you're scratching your head at this, but how on earth is basic statistics a head scratcher? A thumb drive more resilient than a cloud? The notion is just honestly absurd. Your what ifs that you base it on are equally absurd.
It's not absurd at all to recognize account lockouts happen, hacks happen, site internet access downtime happens. (particularly if a fire/natural-disaster enough to destroy local backup media).

If you get locked out of your cloud account - You have another backup.
If the service goes under - You migrate your backup set to another service, just like you manage your other backup media.
It gets hacked - They restore from their own backups.
If your flash drive fails, you have another backup too. Remember we're talking about only 60GB, I can literally buy a new flash drive every 2 months at lower cost, and have so many redundant copies that I'm tripping over them, if I felt I needed that many which I don't.

You don't have internet access due to fire / flood / etc. - You likely lost your thumb drive backup as well.
Not at all, as already stated I have one in my pocket right now, on my phone, on my desk, and something that's not sensitive data like music files, I could throw just about anywhere without security issues of someone else happening upon it.

It's pretty preposterous in this scenario to think you have everything you need in this scenario including something to play music on if you only just had your precious thumb drive with music on it, but a stop at your local library or McDonalds parking lot is just a bridge too far.
Your accounts are seized by law enforcement - You have another backup.
Yes, that other backup, and the other, are USB flash thumb drives, because more reliable for this purpose.

The thumb drive isn't "precious", that's kind of the point, a low cost reliable way to keep redundant local copie of data, so low cost that you can have redundancy coming out the wazoo if it worries you that much - it does not worry me, I've been doing this successfully since the first affordable flash drives hit the market, around 20 years ago.

The reality is that thumb drives, SD Cards, etc. are all bargain basement devices with terrible reliability in general. Yes things go wrong in cloud, which is why you pay them to have the resiliency to deal with it.
See above, I haven't lost a single bit of data over ~20 years of USB flash drive use. I have lost a single flash drive at a time, very few times over those decades, but I never advocated having only a single point backup of a single flash drive. Just like a cloud has redundant backups, so should a flash drive user. Does that cloud tell you every time they have a hard drive failure? Of course not, they have redundancy that seems invisible to the end user, but HDD failure is a far more routine thing for a cloud service than USB flash drive failure has been in my use.

Coming to the conclusion that a USB Flash Drive is more resilient than cloud is indeed a head scratcher. You need only look at the MTBF (if they even publish it) to reach the correct conclusion.
MTBF is meaningless for storage devices, mostly tells you infant mortality rate, but even then it is more applicable to cloud storage HDD failure. Besides, doesn't matter, when you have redundant media and it outlasts your required service life. In this music file storage application, it's not getting hammered with writes like cloud HDDs are. Even so, as stated previously I replace them every few years for capacity increase purposes, and it's nice that cost/capacity went up so much over the years too.

Details matter! Someone who can't factor in these details (average home user), is going to be more likely using the cloud service (if they can even figure out how to do that), as is someone with more sensitive data they can't have lying around on a flash drive, or a lot more data where it is cost prohibitive to use that much flash storage. That just isn't the case with 60GB of music.

If you are having failures from redundant USB flash drives to the point where you're losing data, then for whatever reason it isn't the right choice for you.

I suppose we'll both just keep doing what works well for each of us. I never argued that it wasn't better to have BOTH a flash drive AND cloud storage, but forced to pick one or the other for 60GB of music, USB flash drives every time... plus, it's kinda HANDY to have the music on my phone too!
 
Last edited:
  • Like
Reactions: Insert_Nickname

igor_kavinski

Platinum Member
Jul 27, 2020
2,544
1,304
96
Flash drive might sound like an inexpensive multiple backup media idea but trust me from my own experience, UFDs suck big time. I've learned not to entrust them with anything important without a copy on an actual drive, spinning rust or SSD. Their electronics are bad. You plug them one time too many into a bad front USB port, their electronics die. Your data is there on the chips but you can't access them. Cheap UFDs on sale? Don't buy multiple of the same brand/model! I did that and went on a spending spree buying at least 10 of the exact same thing. Result? They exhibited all the usual problems (file system becoming RAW / slow as dog / getting message "Do you want to format this drive?" when it worked fine the previous day). That's with a brand I never had problems with before (Lexar). Do yourself a favor and just put a cheap SSD in an external enclosure. They are designed to withstand dozens of hours of daily usage for at least 3 years or so. As long as you don't put them in some crappy enclosure of some unknown chinese brand, they might even outlive you.

And oh, don't even ask about the Sony branded Made in India flash drives I bought dirt cheap thinking, "It's a Sony!".
 

OlyAR15

Senior member
Oct 23, 2014
777
118
106
Boy, you sure had bad luck with flash drives. I've never had an issue with them. Up until last year, I had an old 512mb drive I bought decades ago, still worked, still held what limited data I put on it.
 

igor_kavinski

Platinum Member
Jul 27, 2020
2,544
1,304
96
Boy, you sure had bad luck with flash drives. I've never had an issue with them. Up until last year, I had an old 512mb drive I bought decades ago, still worked, still held what limited data I put on it.
The older drives are based on MLC. My old drives I bought from 2007 to 2011 are also still working. It's probably the move to really crappy TLC or worse QLC that led to my problems.

1637556582374.png

That's the most recent non-problematic flash drive I've found in a long time. Able to handle 5 simultaneous downloads and the files are saved without issue. Lesser drives have trouble when the file reaches 100% in Chrome. At that point, there is some extra I/O activity that chokes up drives with inferior flash or controller. That includes the Sandisk 256GB Ultra Flair drive which was MORE expensive than this Lexar model.
 

mindless1

Diamond Member
Aug 11, 2001
6,881
951
126
^ Sandisk Ultra Fair is known to have overheating issues. I happen to have one and can verify that.

yes, TLC or QLC will have fewer write cycles. I'm not so sure it will matter in an application of backing up a static set of 60GB music files, but if your use makes write cycles an issue, buy more capacity than you need so the write cycles are wear leveled over a larger area.

5 simultaneous downloads? We clearly do not have the same use scenario. I'm not, nor have I ever, tried to max out multiple I/O on a USB flash drive. Let's be reasonable about what they're good for and what they aren't.

If you need that much concurrent I/O they make flash drives with SSD controllers and DRAM cache. Of course they cost as much or more than same capacity SSD... but why even bother trying to download 5X direct to a flash drive? It seems like a misuse of the tech.
 

igor_kavinski

Platinum Member
Jul 27, 2020
2,544
1,304
96
but why even bother trying to download 5X direct to a flash drive? It seems like a misuse of the tech.
Well, apparently Lexar doesn't think so and I'm glad that they made a quality USB.

I don't know why USBs are relegated to being a cheap and disposable tech. If it involves data, it should be reliable. What if someone is working on a huge file at work and they need to leave early and then work on it from home late at night? A decent USB would copy the file in a few minutes (5 MB/s minimum write speed) and then NOT refuse to work when you plug it into your home PC. That's all that a USB has to do. Yet, I've had so many USBs fail in those requirements for me that I don't know what USB manufacturers think when they make these things. Sandisk particularly was the first one I had fail on me back in 2008. It's like they don't even have engineers. They just give schematics to interns and tell them to get the product out the door ASAP without proper testing and QC/QA.
 

ASK THE COMMUNITY