Determining file corruption

[limer]

I had a game iso copied to my external HD (external case with HD). I took the external HD over to a family member's house and mounted the iso with Virtual CloneDrive. I received an installshield error that indirectly said that something was corrupt. After looking around on google and killing various installshield cache files, the same install error occurred.

I suspected that the file itself really was corrupt and thought I'd see what the MD5 for both the external HD and my local copy came up with. They were different. Worse than that, another large file on the external HD and a local copy had different hashes. So it appears that the HD is corrupt, the external chipset is having issues or the USB cable is bad.

I'll do more testing with Seatools for Windows and swap to a different drive, but this raises a new question:

How do I determine if the files I just now copied from the external HD (such as various documents) are corrupted without trying to test them individually. In other words, without opening all images in irfanview or test all docs with Word. Is this possible?

Any advice or direction would be appreciated.

Please, no holier-than-thou comments about how a backup would solve all this. Most of the data is backed up and what isn't I'll learn to live without.

 

[Fullmetal Chocobo]

Out of curiosity, are there format differences between the storage mediums (FAT32, NTFS, etc)? If so, I would use a hex / binary comparison tool to check for differences so that disk formats isn't a factor.

Do you have anything else to suggest you have corruption on the hard drive other than a CD that won't install and different MD5 checksums?

 

[limer]

File systems are both NTFS iirc. I'm at work now and will double-check when back at the house. In any event, I've never heard of different hashes just because the file system was different. Doesn't this go against what a hash should do?

If I download a file from a unix based web server that also gives me the MD5, should it not matter that I may be on a mac or windows PC?

I have no other evidence of corruption at the moment. We'll see what happens after a long Seatools test finishes.

 

[Fullmetal Chocobo]

File systems are both NTFS iirc. I'm at work now and will double-check when back at the house. In any event, I've never heard of different hashes just because the file system was different. Doesn't this go against what a hash should do?

If I download a file from a unix based web server that also gives me the MD5, should it not matter that I may be on a mac or windows PC?

I have no other evidence of corruption at the moment. We'll see what happens after a long Seatools test finishes.

I'm honestly not sure either. Just an idea. Any searching I do only comes up with inconclusive results. I figure it would be an easy check to see if that is the cause of the issue, if the md5 was the only issue.

 

[Nothinman]

File systems are both NTFS iirc. I'm at work now and will double-check when back at the house. In any event, I've never heard of different hashes just because the file system was different. Doesn't this go against what a hash should do?

If I download a file from a unix based web server that also gives me the MD5, should it not matter that I may be on a mac or windows PC?

I have no other evidence of corruption at the moment. We'll see what happens after a long Seatools test finishes.

For binary files you're correct, the hash should be the same regardless of the filesystem. However for plain text files it can vary because Windows, unix and OS X all use different line terminators. Actually, I guess OS X would have a mix since it's mostly unix now.

As for checking them all in batch, the md5sum command can take a file containing filenames and hashes and verify files against that list. But you'll have to generate the list file first.

 

[limer]

For binary files you're correct, the hash should be the same regardless of the filesystem. However for plain text files it can vary because Windows, unix and OS X all use different line terminators. Actually, I guess OS X would have a mix since it's mostly unix now.

As for checking them all in batch, the md5sum command can take a file containing filenames and hashes and verify files against that list. But you'll have to generate the list file first.

Interesting although I would think that a text file would also be the same. If the hash is the result of a working algorithm, your linux based editor's text file should be the same MD5 as on my Windows system except that windows will either refuse to display it, display it incorrectly or convert it on the fly. Granted, a saved conversion of that same file would have a different hash. I would think the original file would be the same.

In any event, something interesting from my end. I copied a small (~900KB) rar over to the drive and checked the MD5 on both and they match. Also checked:

Ubuntu 9.10 server i386 iso

55618AD5F180692F9DAC20CBFF352634

55618ad5f180692f9dac20cbff352634

against the ubuntu hashes online and they match. Onto Seatools.

 

[Nothinman]

Interesting although I would think that a text file would also be the same. If the hash is the result of a working algorithm, your linux based editor's text file should be the same MD5 as on my Windows system except that windows will either refuse to display it, display it incorrectly or convert it on the fly. Granted, a saved conversion of that same file would have a different hash. I would think the original file would be the same.

In any event, something interesting from my end. I copied a small (~900KB) rar over to the drive and checked the MD5 on both and they match. Also checked:

Ubuntu 9.10 server i386 iso

55618AD5F180692F9DAC20CBFF352634

55618ad5f180692f9dac20cbff352634

against the ubuntu hashes online and they match. Onto Seatools.

There's a other variables though. If you download the text files via FTP and specify text then the FTP server will likely fixup the line endings for you. And md5sum has an option for comparison in text mode (and it says it's the default in my man page) which could change the results depending on the OS it's run on.

 

[limer]

A quick update:

Tested the HD while out of the external enclosure with both Seatools for DOS and Powermax 4.23.

Both long tests pass. Maybe I'm wrong about the drive although I can't figure out how the data became corrupted quite yet. Will post back with more later.

 

[limer]

For now, I'm going to stop using the drive because I've got other problems/projects I'm distracted by. If anyone believes they know how to tech it further or just wants to chat about this topic, send me a pm or reply.