Speeding up the unzipping proces?

[MarcVenice]

When unzipping big files it takes like over 10 minutes for a 10GB file. But, my e7300 @ 3,8ghz is loaded at 25/30% tops. Is my hdd being the bottleneck?

I do have 2 hdd's, so I tried unzipping 1 file on 1 hdd, and the other on my other hdd, but things completely slowed down to a crawl. Suggestions? I unzip a lot of big files all the times, and before I start throwing money at it, I'd like to know if there's anything else I can do.

 

[faxon]

get a velociraptor, you're probably running into a HDD bottleneck like you were thinking. just out of curiosity though, which hdds are you using now?

 

[Rhoxed]

try to extract from one drive to the other

or raptor, or RAID 0 - a 12GB file takes my 2 640's about 2.5 min

 

[Keysplayr]

Definitely the HDD. SSD's are a great step toward alleviating this constant system bottleneck, but still a long way to go.
The best you can do is RAID0 3 SSD drives. Beyond that I think would be diminishing returns due to I/O limits on the SATA bus.
Unfortunately the best drives belong to Intel, and they are very expensive for their GB size. But they are the fastest out there right now.

 

[MarcVenice]

Originally posted by: faxon
get a velociraptor, you're probably running into a HDD bottleneck like you were thinking. just out of curiosity though, which hdds are you using now?

Two samsung f1 500gb drives. I might buy a 1tb drive to keep backups and put the two f1's in raid 0. Reckon it's worth it though, on my onboard raidcontroller, a ICH9R?

The intel ssd's are nice, but a little to expensive for me right now. Thanks for confirming it's a hdd-bottleneck.

 

[Idontcare]

Originally posted by: MarcVenice

Originally posted by: faxon
get a velociraptor, you're probably running into a HDD bottleneck like you were thinking. just out of curiosity though, which hdds are you using now?

Two samsung f1 500gb drives. I might buy a 1tb drive to keep backups and put the two f1's in raid 0. Reckon it's worth it though, on my onboard raidcontroller, a ICH9R?

The intel ssd's are nice, but a little to expensive for me right now. Thanks for confirming it's a hdd-bottleneck.

What is the general make-up of the files that are unzipped that need to be written? Lots of small (20-100KB) files or lots of large (20-100MB) files?

If it is <100KB stuff then you will be rate-limited by access times on the spindle-drive, better to use an SSD, if it is >20MB stuff then you will write-bandwidth limited and you are better off with raid-0 spindle drives for cheaper but high write-bandwidth.

 

[MarcVenice]

Good call Idontcare. I actually did think about that, and I'm moving larger files, upwards of 50mb to 500mb. Any suggestions on stripe size ?

I wanted to benchmark loading times with raid on a onboard raid-controller, so I've went ahead and ordered a 1tb f1. It's cheapest here and quiet so yeah ... I might consider getting two seagate 10.12 500gb drives those, with a single platter. Not sure if it's worth it though, it does mean I have to get rid of my samsung f1's.

 

[Idontcare]

Originally posted by: MarcVenice
Good call Idontcare. I actually did think about that, and I'm moving larger files, upwards of 50mb to 500mb. Any suggestions on stripe size ?

I wanted to benchmark loading times with raid on a onboard raid-controller, so I've went ahead and ordered a 1tb f1. It's cheapest here and quiet so yeah ... I might consider getting two seagate 10.12 500gb drives those, with a single platter. Not sure if it's worth it though, it does mean I have to get rid of my samsung f1's.

64KB stripes seem to be a good "one size fits most" approach.

You lose effective bandwidth for any files <128KB but your controller spends less time piecing the file back together or breaking it apart (less processing overhead).

 

[Philippart]

try unzipping big files on a ramdisk if you have enough ram

EDIT: well I doubt you have more than 10gb ram

 

[geokilla]

Are you using WinRAR? WinRAR is multithreaded now so you might have to check something in WinRAR to make it use the additional cores and stuff.

 

[Flipped Gazelle]

Originally posted by: MarcVenice
When unzipping big files it takes like over 10 minutes for a 10GB file. But, my e7300 @ 3,8ghz is loaded at 25/30% tops. Is my hdd being the bottleneck?

I do have 2 hdd's, so I tried unzipping 1 file on 1 hdd, and the other on my other hdd, but things completely slowed down to a crawl. Suggestions? I unzip a lot of big files all the times, and before I start throwing money at it, I'd like to know if there's anything else I can do.

Perfect time to grind and brew some yummy coffee! :sun:

 

[MarcVenice]

Flipped: http://catalog.ebay.com/Philip...Q_pidZ56730629QQ_tabZ2

Takes roughly 1 second to push the button Maybe 5 seconds to put in a coffee-pad

Using the latest winrar. And no, I don't have more then 10GB of ram

 

[OCGuy]

This is the only way.

 

[Flipped Gazelle]

Originally posted by: MarcVenice
Flipped: http://catalog.ebay.com/Philip...Q_pidZ56730629QQ_tabZ2

Takes roughly 1 second to push the button Maybe 5 seconds to put in a coffee-pad

Well then, how about baking some

Have you experimented with different unzipping programs?

 

[Idontcare]

Originally posted by: MarcVenice
I do have 2 hdd's, so I tried unzipping 1 file on 1 hdd, and the other on my other hdd, but things completely slowed down to a crawl.

Are you saying here that you unzipped two files simultaneously? Or are you saying you unzipped one file, reading it off of one drive and writing the unzipped contents to the second drive?

Be sure you check where the program's temp/cache directory is stored. A lot of these programs first extract the info into the temp location, and then they copy it to the target location.