upgrading hard drives for photo/video editing

[chiew]

hi all

a few questions...i'm getting a headache reading about hard drives; not that knowledgeable about all the terminology surrounding the topic

in any case, i've been using a 80GB western digital drive as my boot drive for 5 years now. if i recall correctly it is a 7200.7 drive...ancient by today's standards.

i am planning an upgrade to my rig that's going to involve core i7 and 6 gigs of RAM. however, there have been times where i think it is more my hard drives that is holding me back a little. loading pictures into photoshop or lightroom takes a decent amount of time, so i was planning on doing an upgrade to my hard drives

i've read a little on the subject, and there seems to be some debate over whether some of the faster 7200 rpm drives are a better deal than the 10,000 rpm drives. as far as i can tell, one major advantage the 10k drives hold is random access times. i'm not really willing to make the jump to SSD just yet, so that is out of the question.

i was wondering exactly what points i should notice an improvement in, if i were to get a 10k drive over a 7200rpm WD caviar black. the applications i am majorly using are adobe ones: photoshop, lightroom, and premiere. haven't started editing videos, but that will be soon enough (got a canon 5dmk2 recently).

i have considered a couple options:
1) replace boot with 10k drive (>100gb...150gb 10k raptor costs $170)
2) replace booth with a 7200 caviar black (640gb will cost $70)

if i were to do option 2, i would be tempted to spend a little more and get the 1TB version, since it would give me more space to hold photos as well as the OS. i would partion part of it to be the OS portion. however, if i were to put pictures i wanted to edit on the remaining partition, would this affect performance? (i'd imagine that since photoshop will be on the same drive, the head would need to go back and forth between the PS part and the part holding pictures..which would slow things down). is it better to have the files being edited on another drive? in which case i'd likely get another caviar black to hold pictures.

so in summary: two questions
1) where in the adobe editing workflow will i see an improvement if i were to get a 10k drive over a 7200 one?
2) is it better to keep files to be edited on a different hard drive than the one holding adobe photoshop?

thanks a lot in advance!

 

[pjkenned]

That's not a bad idea, and having an OS/App partition and a data partition is a good idea, or even an OS/App drive(s) and data drive(s).

If you are getting the VR, consider the 300GB model, just becaue I've seen it priced not that much higher than the 150GB model. If you are doing a lot of video/picture editing, space is at a premium. My raw HD footage is at least 1GB/ 90 sec so filling TB's quickly is a very easy thing to do. Even my old Canon 20D seems to be unable to stay under 4GB of pictures every time I take it out.

For video, the large 1TB+ drives are actually a lot better than people give them credit for as they can do 120MB/s sequential. For photo/video files stored on the hard drive are usually sequential reads/writes not random access.

If you have a lot of ram, then where photoshop is located means less, because you will be hitting ram for the program, not the hard drive. The VR will help the initial program load times over a 7,200rpm drive.

Once you are ready to get a bit more serious, it may be worth looking into Raid 5 to help boost performance, storage capacity, and give you some fault tolerance.

 

[chiew]

thank you so much for your information, pjkenned. Very much appreciated!

You mentioned that the 7200RPM drives is about 120mb/s. As a comparison, what is the VR's sequential data rate?

And yes, I frequently hit several gigabytes when I go out these days. Not hard when I shoot RAW..25 megs per picture! Within the next couple weeks, I have several shoots planned (at least 5 photoshoots and a wedding)...I wouldn't be suprised if I hit 100 gigs at the very least by the time it is over! My storage is going to take a hammering .

 

[Yellowbeard]

Here's what we have done in the past when my wife was heavy into editing. I'd suggest 4 or 5 or even 6 HDs. Our typical setup was a RAID-0 OS drive array, a RAID-0 data/working array, and then another drive as a backup. You could even use 2 drives in RAID-1 as a backup for your final work.

Run your OS and Apps on the system RAID-0 array. When you are actually working with the images and projects, there's no need for redundancy. You need fast thruput. So, you can work with your images on the RAID-0 working array, and save the finished products to the RAID-1 array or single data drive.

 

[Blain]

OS & Apps on one partition working with media on the second partition Sux.
OS & Apps on one drive working with media on a second drive doesn't Suck.

 

[Yellowbeard]

Originally posted by: Blain
OS & Apps on one partition working with media on the second partition Sux.

True, there is no advantage to this at all with the possible exception of OS corruption. There is no I/O advantage but, if the OS corrupts, the data on the second partition will be intact.

 

[pjkenned]

Originally posted by: chiew
You mentioned that the 7200RPM drives is about 120mb/s. As a comparison, what is the VR's sequential data rate?

Great question for Google, however, the relevant answer is not that much more. On platter hard drives, the answer is complex, but suffice to say, the VR is not going to be 200MB/s while a fast/large 7200rpm drive is 120MB/s. It is probably more of a +/- 15%. The VR's are just much better at random I/O scenarios.

Also, the suggestions of raid 0 + raid 1 are not bad, but I was trying to keep the costs reasonable at 3-4 drives total. It is important to remember that the failure formula for raid arrays with no parity is basically f^n where f is an individual drive's failure rate, and n is the number of drives in an array. As a result, a raid 0 array with 5 disks is 5 times more likely to lose your data (non-backed up) than a single disk.

Also, what cannot be underestimated is the time it takes to re-install everything if a drive crashes and you need to re-install from a backup. Prior to moving everything to raid 1/5/6, I would lose a half to a full day upon losing a drive. Now losing a drive merely means I swap another one in and have to suffer some degraded performance while the rebuild occurs.

 

[chiew]

thanks a lot for the replies, guys!

wow, raid 5 requires 4 drives. pretty expensive. but as you said, worth it down the road, perhaps.

so most people would suggest having a drive for OS & apps and a separate drive for media?
i think i will go with a WD caviar black for my media storage, but i'm still debating on whether to get a VR for my OS&apps drive, or just get another WD caviar black

my understanding so far is that a VR will just allow the program load times and startup times to decrease. actually working with the files is more of a sequential read/write issue, as well as a RAM issue. so it looks like i will forgo the VR...not sure if i'd pay an extra $150 or so just to decrease the amount of time it takes PS or windows to load.

i just feel bad getting a WD caviar black to just be boot + apps...the smallest one is 500GB which is way more than i need for OS&apps.

 

[Blain]

raid 5 only requires three drives, but who's counting.

 

[elconejito]

Whenever possible, for photo/video editing separate your OS/Apps from your working data. And if at all possible, use another drive for your scratch disk.

Don't feel bad about using only a portion of the drive. The bigger drives are generally faster, so the WD 1TB is faster than the 640GB, which is faster than the 500GB. If you don't want to get a TB Black, go for the 640GB black.

You can lessen the influence the drive has by getting enough RAM to keep your work in memory instead of writing to the disk.

 

[Yellowbeard]

Originally posted by: elconejito
Whenever possible, for photo/video editing separate your OS/Apps from your working data. And if at all possible, use another drive for your scratch disk.

Don't feel bad about using only a portion of the drive. The bigger drives are generally faster, so the WD 1TB is faster than the 640GB, which is faster than the 500GB. If you don't want to get a TB Black, go for the 640GB black.

You can lessen the influence the drive has by getting enough RAM to keep your work in memory instead of writing to the disk.

FWIW, RAID-5 is totally unnecessary in this case unless this is a mission critical 24/7 usage machine. Have you set a $$$ budget just for the HDs?

 

[chiew]

Originally posted by: Blain
raid 5 only requires three drives, but who's counting.

oh, nm then

 

[chiew]

Originally posted by: elconejito
Whenever possible, for photo/video editing separate your OS/Apps from your working data. And if at all possible, use another drive for your scratch disk.

Don't feel bad about using only a portion of the drive. The bigger drives are generally faster, so the WD 1TB is faster than the 640GB, which is faster than the 500GB. If you don't want to get a TB Black, go for the 640GB black.

You can lessen the influence the drive has by getting enough RAM to keep your work in memory instead of writing to the disk.

thanks for the advice. when you say separate drives, you are talking about separate physical drives, correct? not separate partitions?

as for the scratch disk...is the scratch disk supposed to perform well with random access or sequential? i have this vague concept in my mind that scratch is random access, in which case a velociraptor would be ideal.

do you have any experience editing with adobe? i know roughtly how much my still pictures take to edit, and its above 3 gigs including background programs. video editing i am sure will take more. my initial build is including 3x2gigs of ram, and i am getting a motherboard with 6 slots over the cheaper 4-memory-slot one because i want to be able to add 3 more sticks in the future.

 

[chiew]

Originally posted by: Yellowbeard
FWIW, RAID-5 is totally unnecessary in this case unless this is a mission critical 24/7 usage machine. Have you set a $$$ budget just for the HDs?

I don't think I will be running RAID5. My hard drive budget is currently $150ish to $300ish. The reason I picked that amount of budget is I want to replace two drives.

Currently I have
80gig 7200.7 boot drive (from my freshman year in college, dell desktop from 2004)
250gig 7200rpm for music/movies/word files, etc
2x500 gig 7200rpm drives..i think they're seagate server-style drives. These currently hold my stills and video from my DSLRs. One is the backup of the other..I just manually back up the drives. I initially wanted to RAID them to be mirrors, but my mobo doesn't do that.

My ideal set up
Fast boot drive
1TB caviar black for stills and movies from my DSLRs
2x500 gigs backing up the 1TB drive

For fast boot drive I'm currently weighing between VR and WD cav black...which is why my budget is $150-300 ($100 is for the 1TB cav black present in both cases, the extra $50 is for a small cav black or the extra $200 is for a VR). Of course I'd rather save money if the only advantage to the VR is faster OS boot time and faster startup of programs.

If I get a small cav black (640gb or so), I would partition off 140 gigs or so for OS&apps, and use the remaining <500 for music/movies/word files...general low-use data. This is making the most economic sense to me.

My money-stingy side is a tiny bit afraid I won't notice a huge difference going from
1) 7200.7 boot drive to WD caviar black boot
2) 7200rpm seagate server-style HD to WD caviar black for stills/videos from camera

But my performance hungry side is curious and wants to try out a cav black or VR as a boot drive. Something tells me that 7200.7 is pretty oudated and slow. in any case, i will need additional room for my stills/video, as after the next two weeks it wouldn't suprise me if i add 100gigs of files...lots of photoshoots coming up...so getting more space is definitely a must, which is why at the very least I am getting a 1TB cav black. I think my performance-oriented side is going to win out and I will definitely get a new boot drive to replace my 7200.7 one. I think another part of my reluctance is setting up a comp all over again...but I guess it will be worth it...I want a 64-bit OS to get access to more RAM.

basically the only debate i have left is whether to go with a $65 WD cav black 640gig hard drive boot drive with other random data files on it (not ones that will be edited...music, movies, word files) or a $200 or so 300gig velociraptor. spending as much as a decent graphics card on a hard drive kinda makes me uncomfortable...so i am leaning more towards a cav black boot...

i guess the main factor to springing for a VR is whether or not it will make a huge difference when editing...does the scratch disk better appreciate a drive with low latency or a drive with fast sequential read/write? if the former, that adds a point to the VR's argument, if the latter, as far as i have learned the VR and cav black are pretty much the same speed so its a tie...actually its a point for the cav black since its cheaper.


alright i'm ranting on ... sorry haha

 

[elconejito]

Hey Chiew, I'll try to answer as many of these as I can.

#1 - Yes, I mean separate drives. NOT separate partitions. The idea is to not have read/writes going on at the same time on the same drive.

If you have only one drive, the potential is there for Windows to be doing something (page file, indexing, AV scan, whatever), Photoshop to be using the scratch file, and you writing to the disk on a save. Thats not even including if you have other apps open and doing stuff.

[Overly complex example, I hope it makes sense. It's easier with pictures ]
I've never tried it, but I can only imagine having partitions is *worse* that just using one drive. Imagine a 500GB drive setup with the first 100GB is OS/Apps on C:, the remaining 400GB is for data on D:. Let's say you use up 30GB of the C: drive. So every time the head needs to move from the first 30GB of the disk (say writing to the page file) to the data partition (say to save a file) it has to skim over the intermediate **unused** 70GB to get to the data. And then the head needs to travel back to the first 30GB to continue what it was doing. Just a lot of unnecessary travel for the heads.
[End overly complex example]

#2 - The scratch disk (if you need to use it) is anything that won't fit in RAM. So it could be little things like "what was the last undo", or it could hold multiple states of the image your working on which might be huge.

#2a - If you can keep everything in RAM, then the hard drive is less of factor. Open the biggest file you have and check the scratch size in photoshop. As long as the scratch size (not document size) fits in RAM then don't worry too much about the hard drives. In my opinion, it's better to spend a few bucks on more RAM to prevent disk access in the first place than trying to spend a lot of money on hard drives. Only if you're doing really huge projects, like panoramas for example, would I shell out cash for fast disks (or setup RAID) because there just isn't enough RAM and it will definitely hit your HDD.

#3 - I've been using Adobe products as a graphic designer since the mid 90's. These days I live mainly in Dreamweaver Illustrator and photoshop (in that order) with Premiere Elements occasionally tossed in with misc other programs.

#4 - If you know you're going to be hitting the drive a *lot* (did you check your PS scratch size yet?), go for the VR. Otherwise stick with the Caviar Black. Let's say the VR is 20% faster (I just made up that number). If you only hit the drive for 2 secs at a time, lowering that time to 1.6secs is NOT worth it. However, if you hit the drive for 30 secs at a time, lowering that to 24secs might be worth it. The longer you read/write to the drive, the more you will notice the faster drive. Little writes, you aren't going to notice.

#5a - You will notice a difference from the 80GB you have now to the Caviar black. I don't know what the throughput is on that type of drive, but I went from one of the older 250GB drives which had a max STR of about ~65mb/s to the 640GB Blue which maxed out above 100Mb/s. Very noticeable.

#5b - Again, I don't know about those 500GBs you have, but I'd suspect it would be a noticeable difference.

Bottom line: If I were you, and it were my money I'd max out the RAM on that board (or as close to as I could) and buy 2 caviar blacks. They are plenty fast, and if you don't use the drive much (because it's all in RAM) you will never notice it.

Check your scratch sizes in Photoshop first!

Another alternative (I'm always down with saving a few bucks) is to buy 2x640GB caviar blacks ($65 each). 1st drive has OS/Apps, 2nd drive has working data. If you have movies, mp3s, etc I'd put them on the first drive (no partition). Then, use your 2x500GB as long term storage so once you've finished editing them stow them away here. Performance on the 500gbs doesn't really matter since you aren't editing them anymore so you can go ahead and just fill them up. This reduces the need for the TB storage.

Once prices drop a little bit, you can retire the 2x640GBs to long term storage also and then go pick up a pair of 1TB blacks (or whatever is the drive of choice at the time) to replace the 2x640GBs. Rinse and repeat this every year or two.

 

[chiew]

hi elconejito,

thank you so much for your detailed answer. it was really very helpful, thanks a lot for your time!

right now i am in the middle of applications for school, so i want a reliable computer more than tearing my current one apart and installing a OS (even though i'm sure things will go smoothly, it will give me a stress attack if the comp is down for even a couple days, hehe). but once its over, i will definitely do an overhaul of the computer, including more RAM and a x64 OS for more memory addressing, as well as a couple new hard drives.

again, thanks!

 

[pjkenned]

Originally posted by: Yellowbeard
FWIW, RAID-5 is totally unnecessary in this case unless this is a mission critical 24/7 usage machine.

Most people who get paid for photography/ videography like to never lose data. Sometimes this means a backed up Raid 0 array, raid 5 is another strong option (along with normal backup). I had a friend in college that lost quite a bit of money due to a hard drive crash losing wedding pictures that she still held the copyright to.

Raid 5 works with 3 drives, but you start seeing big benefits with more drives since you basically get the full capacity of the next added drives + a performance bump for each additional drive over 3.

I 100% agree about going x64 and more ram though.