How do you compare THE SAME files in a folder when they're in different drives?

[Carrie Anne]

I have a C-Drive (Windows 10) and several EHD's (external hard drives).

The EHD's mirror what is on the C-Drive, I use the EHD's to back up the exact data, the exact number of files, that are found on the C-Drive.

Problem: Sometimes my C-Drive might have a folder that has way more files in it than the EHD does. For example, the My Pictures folder located on the C-Drive

might have 1,100 files in it

and the same folder (My Pictures) located on the external hard drive might have less,

950 files in it, for example.

Is there a procedure (or a software program) that will

quickly find, highlight, and point out

the 100 outstanding files so a person can transfer them over to the EHD?

My EHD's are remotely stored in a safe deposit bank at a chosen bank. I swap them out, or rotate them, about once a week.

Please dumb your answer down, way down, if you're a code king or code queen, because I'm a simple person who works in a deli, thank you

My OS: Windows 10

 

[Ketchup]

Well, it depends on what software you are using. Are the files you are missing actual pictures (in this case), or are they thumbnails, other files used by Windows?

If you want an automated way to compare the folders, Beyond compare is what I always use, for files and folders.
http://www.scootersoftware.com/

 

[Carrie Anne]

Today it's picture files, next month it might be video files or some other file.

Not sure if file type matters, any file would be the answer. Any run-of-the-mill digital file (music, movies, text, etc). In any folder that contains hundreds or thousands of files that people use for work, school and entertainment.

I have almost 4TB's of data and that amount continues to grow on a daily basis. In the thread starter I used a hypothetical folder, 'My Pictures.'

As it turns out, it's not really a hypothetical because due to extra stess in my life I currently have a mismatch in the My Pictures folder -

the My Pictures folder on the C-Drive contains

way more files

than the My Pictures folder on the EHD, it's about 100-200 files off.

Both folders (My Pictures) contain thousands of pictures - which it makes it hard to uniformly sync both folders when you have a discrepancy of 100-200 files.

Keep in mind this issue pertains to any standard, run-of-the-mill file that a non-techie backs up on an external hard drive: movies, music, text, etc - it's an issue that could happen to anyone when a person backs up their data to a portable or desktop external hard drive.

Because I create new files on a daily basis, I always have three Windows open on my desktop. One Window for the C-Drive and two Windows for two seperate hard drives. I do this because I like to back up my work whenever I create a new file and any file related to work, school, or entertainment - and because two of those EHD's will be placed in a safe deposit box at least once a week.

I'm usually very good at making sure that all 3 drives are uniform in number, but I recently moved, and that added a super amount of stress to my life. It is easy to make mistakes when dealing with 4TB's of data when you have an extraordinary amount of stress in your life.

Not all is bad: I have 69,359 files in the folders titled 'My Documents.' Luckily, that folder is uniform in all 3 drives (the C-Drive and the two EHD's). 'Uniform' means that the My Documents folder found on all 3 hard drives contain the same number of files: 69,359 files.

 

[Ketchup]

What program are you using?

 

[user1193526]

there probably a million apps to do what your asking

but to do it without any app, just past the bellow code into a .txt file and rename it to .bat and double click it... Its kinda Idiot Proof unless win10 operates differently.

the bellow file will allow you to specify the 2 directories to compare, and ask if you wont to copy/overwrite destination files, or simply identify file differences without copying.

you can drag and drop the directorys onto the command window after it opens so you don't need to type them, then hit Enter.

you can ofcourse change it however u wish, tho beyond the simple File Size Check, you'd probably wont an app that can run a simple CRC Hash against the SOURCE and BACKUP files.

just for example.
http://esrg.sourceforge.net/utils_win_up/md5sum/
the CRC for example would be substitued in the bellow script for the Size entry. If you go that route I'm sure someone if not me can elaborate 4u when the time comes.

@echo OFF
	setlocal ENABLEEXTENSIONS
	setlocal ENABLEDELAYEDEXPANSION

REM SET VARIABLES-------------------------------
	echo.
	set /P _DIR_SOURCE="ENTER Source Directory: "
	set /P _DIR_BACKUP="ENTER Backup Directory: "
	set /P _DIR_LOG="ENTER Directory to Save output Log: "
	set _DIR_SOURCE=%_DIR_SOURCE:"=%
	set _DIR_BACKUP=%_DIR_BACKUP:"=%
	set _DIR_LOG=%_DIR_LOG:"=%

	FOR /F "tokens=*" %%a IN ("%_DIR_LOG%") DO (set _FIND=%%~xa)
	IF "%_FIND:~,1%" NEQ "." (
		set /P _FILE_LOG="ENTER Log File Name: "
	)
	set _FILE_LOG=%_FILE_LOG:"=%

	FOR /F "tokens=*" %%a IN ("%_FILE_LOG%") DO (
		IF /I "%%~xa" NEQ ".txt" (set _FILE_LOG=%_FILE_LOG%.txt)
	)
	echo.
	echo Copy Files Not Found in Backup ?
	echo.
	echo      FROM : Source
	echo        TO : Backup
	echo.
	set /P _COPY="N/NO/ANY/BLANK=No   YES=Yes  ,ENTER: "
	echo.
	IF /I "%_COPY:~,3%" NEQ "YES" (
		set _COPY=REM
		set _REM=REM
		set _ECHO=NOTFOUND
	) ELSE (
		set _COPY=xcopy
		set _REM= 
		set _ECHO=COPYING
	)
	FOR /F "tokens=1,2 delims=: " %%a IN ('time /T') DO (
		set _TIMESTAMP=%DATE:~10%-%DATE:~4,2%-%DATE:~7,2%_%%a%%b
	)

echo.
echo ##################################################
echo 	START
echo ##################################################
echo.

	echo %_TIMESTAMP%>>"%_DIR_LOG%\%_TIMESTAMP%_%_FILE_LOG%"
	echo SOURCE  =  %_DIR_SOURCE%>>"%_DIR_LOG%\%_TIMESTAMP%_%_FILE_LOG%"
	echo BACKUP  =  %_DIR_BACKUP%>>"%_DIR_LOG%\%_TIMESTAMP%_%_FILE_LOG%"
	echo.>>"%_DIR_LOG%\%_TIMESTAMP%_%_FILE_LOG%"
	echo.>>"%_DIR_LOG%\%_TIMESTAMP%_%_FILE_LOG%"

REM STORE BACKUP FILE NAMES TO BUFFER-----------
	cd /D "%_DIR_BACKUP%"
	set _SKIP=1
	set _ADVANCE=1

	:A_LOOP
	set _COUNT=0
	set _BUFFER_%_ADVANCE%=/
	FOR /F "skip=%_SKIP% tokens=*" %%a IN ('echo.^&dir /A-D /B') DO (
		set /A _COUNT+=1

		IF !_COUNT! NEQ 50 (
			set _BUFFER_%_ADVANCE%=!_BUFFER_%_ADVANCE%!%%a/%%~za/
		) ELSE (
			set /A _ADVANCE+=1
			goto A_LOOP
		)
		set /A _SKIP+=1
	)


REM COMPAIRE SOURCE FILE NAMES TO BACKUP---------
	cd /D "%_DIR_SOURCE%"
	set _COUNT=0
	set _SKIP=1

	:B_LOOP
	FOR /F "skip=%_SKIP% tokens=*" %%a IN ('echo.^&dir /A-D /B') DO (
		set /A _SKIP+=1
		FOR /L %%b IN (1,1,%_ADVANCE%) DO (
			FOR /F "tokens=1-2 delims=/" %%c IN ("!_BUFFER_%%b:*/%%a/=\/!") DO (
				IF "%%c"=="\" (
					IF %%~za NEQ %%d (
						set /A _COUNT+=1
						set _NUM=   !_COUNT!
						echo.>>"%_DIR_LOG%\%_TIMESTAMP%_%_FILE_LOG%"
						echo  !_NUM:~-4! WARNING  SAME_NAME  but  DIFFERENT_SIZE>>"%_DIR_LOG%\%_TIMESTAMP%_%_FILE_LOG%"
						echo 	SRC  %%a  %%~za>>"%_DIR_LOG%\%_TIMESTAMP%_%_FILE_LOG%"
						echo 	BUP  %%a  %%d>>"%_DIR_LOG%\%_TIMESTAMP%_%_FILE_LOG%"
						echo.
						echo  !_NUM:~-4! WARNING  SAME_NAME  but  DIFFERENT_SIZE
						echo 	SRC  %%a  %%~za
						echo 	BUP  %%a  %%d
						echo.
%_REM%						echo  Overwriting the BACKUP File ?&echo.
%_REM%						set _SELECT=<nul
%_REM%						set /P _SELECT="N/NO/ANY/BLANK=No   YES=Yes  ,ENTER: "
%_REM%						IF /I "!_SELECT:~,3!"=="YES" (
%_REM%							echo  !_NUM:~-4! OVERWRITING  %%a>>"%_DIR_LOG%\%_TIMESTAMP%_%_FILE_LOG%"
%_REM%							echo.
%_REM%							echo  !_NUM:~-4! OVERWRITING  %%a
%_REM%							echo.
%_REM%							xcopy "%%a" "%_DIR_BACKUP%" /Q /H /R /K /Y 1>nul
%_REM%						)
					)
					goto B_LOOP
				)
			)
		)
		set /A _COUNT+=1
		set _NUM=   !_COUNT!
		echo  !_NUM:~-4! %_ECHO%  %%a>>"%_DIR_LOG%\%_TIMESTAMP%_%_FILE_LOG%"
		echo  !_NUM:~-4! %_ECHO%  %%a
		%_COPY% "%%a" "%_DIR_BACKUP%" /Q /H 1>nul
	)

echo.
echo ##################################################
echo 	FINISHED
echo ##################################################
echo.
pause

 

[Carrie Anne]

Update, before I say thanks.

I lucked out big-time: I found some software that will compare a folder that might have a million files in it, when said folder is kept safe on multiple EHD's and uniformity is critically important.

The software has turned out to be a lifesaver and a time saver too because a missing file in a folder that has thousands of files

used to take 30 minutes or a lot longer

to identify

but now takes about one minute of time to identify the missing file(s).

Also, besides cutting down the time it takes to hunt down a file(s), after acquiring the software I learned that precision is no longer important, I can be sloppy and not worry about uniformity when creating new files and then storing them on separate EHD's -

like the night before I go to the bank and I need to be 1000% sure that both drives are uniform because I'm going to put the drives in a safe deposit box.

As soon as I downloaded the software this layperson saw some things and I was immediately hit with a good feeling by what I saw:

1) When you open the program you see 5 icons and one icon is labeled 'Folder Compare' (eureka !!! ... those two words spoke volumes)
2) Click on Folder Compare and then a window opens. Said window is split in half. At this point I was still trying to figure it out when up near the menu bar I saw a lot of icons but the one titled 'Diff' stood out like a sore thumb and I asked myself
3) Does 'Diff' mean it will compare the same folder that is stored on different drives? - boy, lets hope so. I soon learned that, yes, it is indeed the icon that compares folders and that is why
4) The window is split in half, because you drag and drop one folder, from each separate EHD, and into each half of the screen in order to compare the contents
5) You then click on 'Diff' and the program then HIGHLIGHTS the files that are missing
6) All of the above takes about one minute

After stumbling on the software I went to bed that night feeling as if I had finally eliminated a huge problem from my life. The next morning I was of course interested in the price, because I had downloaded a free 30 day trial - I learned that it costs $50, pricey which is what I was kind of expecting.

Over priced? Oh heck yes, but

since it can potentially save me a ton of time,

because I go to the bank on regular basis to swap drives,

and because you can't and never should put COMPLETE trust into online storage companies like Jungle Disk to protect your data,

the over priced $oftware is worth it to me.

Several years ago Jungle Disk lost ALL of my data when they made changes to their sites interface. The change to their sites interface,

coupled

with the loss of every single digital file I had, may, or may not, have been a coincidence. I will never surrender COMPLETE trust to any online storage service.

Sorry, but I can't, and will not, give out the name of the comparison software because you probably know why.

I came back to say thanks for taking the time to answer my important call for help that I lobbed towards the community

and to wish everyone Happy Labor Day 2015

 

[ArisVer]

The EHD's mirror what is on the C-Drive, I use the EHD's to back up the exact data, the exact number of files, that are found on the C-Drive.

SyncBackFree can do what you want for free. They also have a Pro version with more features if you are willing to spend some money.

http://www.2brightsparks.com/freeware/index.html