taltamir
Lifer
A common misconceptions is that if you have a backup or redundancy scheme of any sort your data is safe.
Backup and redundancy are different things, and neither protects against everything.
To properly protect your data you must list all the possible ways in which data can be lost, and then address each and every one of them (or the ones you think are pertinent & are within your budget)
Here are a list of common disasters and the ways to protect against them:
1. Fire - offsite backup, DO NOT use a firesafe as they are meant for paper documents (350 internal temp max... which will ruin your data, but not your paper documents)
2. Flood - offsite backup, waterproof media?
3. Swatting: Swatting comes from the term SWAT, SWATting is when someone places an anonymous call to the authorities, and lies, claiming that you traffic in illegal materials such as illegal gun trading or CP.
Even if you are completely innocent, your computers and drives will be held as evidence for years afterwards. Offsite backup is the only protection against this (as it will surely cause your business to fail otherwise). Note that this isn't advice to circumvent the law, if you are guilty they will likely go after your offsite backups as well. But if you are innocent this can save you. (I have read stories of people who run small software business from home having that happen to them, there was never any evidence found of any wrongdoing on their part, but the FBI kept their computers and data)
4. bit rot - use checksumming. The only HDD/SSD Checksumming filesystems currently are the googleFS (trade secret of google), ZFS (publically available), and BTRFS (almost done).
For optical media you can use http://dvdisaster.net/en/index.html
For individual files or folders containing important data you can use parchive recovery records http://en.wikipedia.org/wiki/Parchive
5. Cosmic rays bit flipping - see bit rot for solutions. add ECC ram and using robust software that can "verify written data" properly.
6. Drive failure - use redundancy (raid 1,5,6, 10/1+0/01/0+1 etc); or backup. Or if the data is important enough use both and even multiple backups. Note that neither RAID 0 nor JBOD are redundant and will not help.
7. Controller failure - use a storage scheme that allows you to replace the controller or does not require one. Any motherboard raid other then RAID1 is likely to result in dataloss in case of mobo controller failure. RAID1 should be fully portable though (but worth verifying before trusting with your data) and high end independent controllers can be swapped out with identical models.
8. A controller failure where your replaceable controller can no longer be found on the market - Upgrade your storage to ensure you are using controllers still available on the market or switch to a system that allows you to switch to other "controllers" (aka, OS based software raid like in linux, solaris, windows server, or open nas; or use controllers from a company that historically makes newer versions support older controller made arrays).
9. Theft - offsite backup OR/AND worthless media backups (DVD-R). they will take your fileserver, iphone, external drive, and flash drives. but unlikely to take your burned DVD-R collection.
10. Virus - NOD32 antivirus 😛. Joking aside. For a simple virus that just deletes all data have an external backup that you turn off / unplug except for when you manually connect it / turn it on to perform a backup. But those viruses are rare. To avoid silent corruption of data you should look at multiple rolling backups off of write once media (aka optical media)
11. Silent data corruption on HDD (due to a bad drive or PSU) - use a checksumming filesystem (only ZFS at the moment) AND redundancy (RAID1, 5, 6, etc). using only RAID will leave you with corrupt data as it has no way of knowing which copy of readable data is corrupt and which is correct. Using only checksumming filesystem will let you know it is corrupt, but without a non corrupt copy to recover from.
12. Sabotage - offsite rolling backup; internal corporate security.
13. Lightening strike - backup to optical media (CD/DVD/Bluray), or backup to an external drive that you physically unplug from both the power AND the PC when not in use AND doesn't sit on a metal surface. If it is sitting on the PC case, or if it is plugged into the PC via a USB cord, that is enough to fry it!
Also, use surge protectors on your PC / Server / External drive (not as safe as unplugging it, but it might save your data and hardware)
Also, please note that whatever method of data protection you use, it is CRITICAL that you:
1. Check to see that the backups are happennig as expected. (is it really backing up daily? or did a careless employee disable it 9 months ago and nobody noticed and turned it on since? did someone switch off the external HDD that is the target of the backup software? both cases are real examples I have seen)
2. Check that the backups are readable (is the CD you burned corrupt? corrupt beyond the ability of DVDisaster to repair?).
3. Check that all needed files are being backed up (for example, your firefox bookmarks cannot be backed up while the program is running. If you leave it on all the time then it will never get backed up)
Terms explained:
Rolling Backup: Rolling backup is what you have multiple backups from different time periods. For example, you have a complete backup of all your files from 1 week ago, 1 month ago, 2 months ago, 3 months ago, and 1 year ago.
Parchive: http://en.wikipedia.org/wiki/Parchive
ZFS: http://en.wikipedia.org/wiki/ZFS
Backup and redundancy are different things, and neither protects against everything.
To properly protect your data you must list all the possible ways in which data can be lost, and then address each and every one of them (or the ones you think are pertinent & are within your budget)
Here are a list of common disasters and the ways to protect against them:
1. Fire - offsite backup, DO NOT use a firesafe as they are meant for paper documents (350 internal temp max... which will ruin your data, but not your paper documents)
2. Flood - offsite backup, waterproof media?
3. Swatting: Swatting comes from the term SWAT, SWATting is when someone places an anonymous call to the authorities, and lies, claiming that you traffic in illegal materials such as illegal gun trading or CP.
Even if you are completely innocent, your computers and drives will be held as evidence for years afterwards. Offsite backup is the only protection against this (as it will surely cause your business to fail otherwise). Note that this isn't advice to circumvent the law, if you are guilty they will likely go after your offsite backups as well. But if you are innocent this can save you. (I have read stories of people who run small software business from home having that happen to them, there was never any evidence found of any wrongdoing on their part, but the FBI kept their computers and data)
4. bit rot - use checksumming. The only HDD/SSD Checksumming filesystems currently are the googleFS (trade secret of google), ZFS (publically available), and BTRFS (almost done).
For optical media you can use http://dvdisaster.net/en/index.html
For individual files or folders containing important data you can use parchive recovery records http://en.wikipedia.org/wiki/Parchive
5. Cosmic rays bit flipping - see bit rot for solutions. add ECC ram and using robust software that can "verify written data" properly.
6. Drive failure - use redundancy (raid 1,5,6, 10/1+0/01/0+1 etc); or backup. Or if the data is important enough use both and even multiple backups. Note that neither RAID 0 nor JBOD are redundant and will not help.
7. Controller failure - use a storage scheme that allows you to replace the controller or does not require one. Any motherboard raid other then RAID1 is likely to result in dataloss in case of mobo controller failure. RAID1 should be fully portable though (but worth verifying before trusting with your data) and high end independent controllers can be swapped out with identical models.
8. A controller failure where your replaceable controller can no longer be found on the market - Upgrade your storage to ensure you are using controllers still available on the market or switch to a system that allows you to switch to other "controllers" (aka, OS based software raid like in linux, solaris, windows server, or open nas; or use controllers from a company that historically makes newer versions support older controller made arrays).
9. Theft - offsite backup OR/AND worthless media backups (DVD-R). they will take your fileserver, iphone, external drive, and flash drives. but unlikely to take your burned DVD-R collection.
10. Virus - NOD32 antivirus 😛. Joking aside. For a simple virus that just deletes all data have an external backup that you turn off / unplug except for when you manually connect it / turn it on to perform a backup. But those viruses are rare. To avoid silent corruption of data you should look at multiple rolling backups off of write once media (aka optical media)
11. Silent data corruption on HDD (due to a bad drive or PSU) - use a checksumming filesystem (only ZFS at the moment) AND redundancy (RAID1, 5, 6, etc). using only RAID will leave you with corrupt data as it has no way of knowing which copy of readable data is corrupt and which is correct. Using only checksumming filesystem will let you know it is corrupt, but without a non corrupt copy to recover from.
12. Sabotage - offsite rolling backup; internal corporate security.
13. Lightening strike - backup to optical media (CD/DVD/Bluray), or backup to an external drive that you physically unplug from both the power AND the PC when not in use AND doesn't sit on a metal surface. If it is sitting on the PC case, or if it is plugged into the PC via a USB cord, that is enough to fry it!
Also, use surge protectors on your PC / Server / External drive (not as safe as unplugging it, but it might save your data and hardware)
Also, please note that whatever method of data protection you use, it is CRITICAL that you:
1. Check to see that the backups are happennig as expected. (is it really backing up daily? or did a careless employee disable it 9 months ago and nobody noticed and turned it on since? did someone switch off the external HDD that is the target of the backup software? both cases are real examples I have seen)
2. Check that the backups are readable (is the CD you burned corrupt? corrupt beyond the ability of DVDisaster to repair?).
3. Check that all needed files are being backed up (for example, your firefox bookmarks cannot be backed up while the program is running. If you leave it on all the time then it will never get backed up)
Terms explained:
Rolling Backup: Rolling backup is what you have multiple backups from different time periods. For example, you have a complete backup of all your files from 1 week ago, 1 month ago, 2 months ago, 3 months ago, and 1 year ago.
Parchive: http://en.wikipedia.org/wiki/Parchive
ZFS: http://en.wikipedia.org/wiki/ZFS
Last edited:
