Can someone explain HDD bottlenecks to me? SATA, SATAII, PCI-E 1x, IDE, etc

[fuzzybabybunny]

Situation 1:

I'm sitting here transferring about 80GB of data from a Seagate 320GB 7200.10 SATAII drive to a 400GB Samsung Spinpoint SATAII drive. The average transfer rate is 30MB/s, ranging from 40MB/s to 20MB/s.

SATAII is supposed to have a max transfer rate of ~375MB/s, but I can see that this doesn't make any difference between PCI 266MB/s because hard drives can't even come close to this value, or even PCI 133MB/s, let alone PCI-E 1x at 500MB/s.

Situation 2:

When I transfer files from my Windows box (SATAII) to my Linux box (IDE) over a gigabit network, the transfer speeds are exactly the same as transferring between SATAII drives within my Windows box (30MB/s)

Huh? Why is transferring over the network the same speed as transferring internally? Shouldn't internal transfers be a lot faster?

Situation 3:

Doing a benchmark with HDTach, I get these results for my Seagate 320GB:

Burst: 250 MB/s
Average Read: 60 MB/s
Sequential Read: Ranges from 80MB/s to 40MB/s

Doing a benchmark with HDTach, I get these results for my Samsung 400GB:

Burst: 185 MB/s
Average Read: 65 MB/s
Sequential Read: Ranges from 80MB/s to 40MB/s

Why are these values so far off from real-world tests? Real world tests don't even come close to 60MB/s, the slowest value in the HDTach tests.

Is it safe to say that my lack of speed is due entirely to the hard drives themselves being slow?

Raptor 74GB vs. Gigabyte's i-RAM

http://www.anandtech.com/storage/showdoc.aspx?i=2480&p=10

In looking at the 1.76GB file copy, the Raptor took 96s and the i-RAM took 32 seconds.

1000 x (1.76GB / 96s) = 18.3 MB/s
1000 x (1.76GB / 32s) = 55 MB/s

Why is the Raptor so slow at 18.3MB/s? This is slower than my 320GB and 400GB... I know that there should be variations, but should they be this large?

Looking at the iRAM, I'm dissappointed. The i-RAM is attached to the PCI bus, but it's performance of 55 MB/s is still far under the PCI bus specs.

 

[bob4432]

if you are transfering random data, which is usually a lot of small pieces, yes hdd speed will go down quite a bit. if you are transferring larger files, like hundreds of MB or even GBs in size, transfer rate will go up to what would be considered the max of the hdd. this is what hdtach measures, the max for large files, not little files.

this is because say you transfer 1000 1MB files, your hdd has to find them transfer them and then start all over again. if you transfer a 1GB or 10GB file, you hdd finds it, transfer one giant file where it can open up and use all of the mechanical speed (which is much less than the interfaces, even ata100/133).

this is why your GbE lan shows the same speed - the lan is not your bottleneck, the hdd is

hope this helps

 

[Paperdoc]

I've seen several HDD tests of data transfer rates in real-world situations. Most agree that, using current SATA II (300 MB/s max) units you will actually achieve about 30 to 40 MB/s transfers. What others have said about the impact of random small-file seeks and copies versus sustained large sequential file read / write copies is a part of the story, but only a part. The rest I don't undestand, but that is the reality.

So, why is a gigabit network copy running at the same speed as an internal copy? Because the limit is in the disk operations, not in the data transfer bus operations.

 

[metalmania]

My transfer rate between IDE drives is usually 35 MB/s by internal IDE cable or by external 1394 link.

 

[Madwand1]

Originally posted by: fuzzybabybunny
Why are these values so far off from real-world tests? Real world tests don't even come close to 60MB/s, the slowest value in the HDTach tests.

The slowest values in the HDTach tests are actually around 40 MB/s -- the minimum rates. The transfer rate varies significantly along the drive, and slow down further when the drives get crowded / fragmented. HDTach works at the drive level, not the file system level, so it doesn't show fragmentation effects.

As mentioned above, with a mix of files, you probably have a lot of smaller files -- these will incur a lot of overhead to process; security checking, working on the directory + file itself, starting & stopping the transfer, etc., and will not come close to the maximum potential transfer rate.

E.g. Going from a local RAID 0 array to an early partition in a 7200.10 PATA drive:

M:\test\test0>xxcopy 10.gb d: /y

XXCOPY == Freeware == Ver 2.93.1 (c)1995-2006 Pixelab, Inc.

-------------------------------------------------------------------------------
M:\test\test0\10.gb 10,000,000,000
-------------------------------------------------------------------------------
Directories processed = 1
Total data in bytes = 10,000,000,000
Elapsed time in sec. = 156.5
Action speed (MB/min) = 3833
Files copied = 1

That's around 64 MB/s. HDTach benches this drive at 40-78 MB/s or so; average 66 MB/s. Around the location of the file, I read between 72 to 75 MB/s, so the file system, system load, etc. (this is also the OS/swap/temp internet/etc. drive), are taking a good 10 MB/s or so off the maximum performance here, even though it's a very large file. The file was fragmented into 7 fragments in this case, which should not have had a large impact..

Originally posted by: fuzzybabybunny
Raptor 74GB vs. Gigabyte's i-RAM

http://www.anandtech.com/storage/showdoc.aspx?i=2480&p=10

In looking at the 1.76GB file copy, the Raptor took 96s and the i-RAM took 32 seconds.

1000 x (1.76GB / 96s) = 18.3 MB/s
1000 x (1.76GB / 32s) = 55 MB/s

Why is the Raptor so slow at 18.3MB/s? This is slower than my 320GB and 400GB... I know that there should be variations, but should they be this large?

Looking at the iRAM, I'm dissappointed. The i-RAM is attached to the PCI bus, but it's performance of 55 MB/s is still far under the PCI bus specs.

The details of the tests themselves are very important. The above is a test designed to highlight the advantage of the i-RAM -- it's a file copy from the device to the same device. This is the worst way to do file copies for typical devices -- it involves a lot of seeking; thrashing back and forth between the source and the destination. At best, you'd expect 1/2 the transfer rate that the device is capable of for a single transfer; generally a fair bit less. This test is also highly sensitive to the location of the files, and the state of the file sysem, etc., so the numbers cannot be compared to anything else other than the drives and their contents themselves. It's fairly useless as a strict test; it only serves to illustrate the seek advantage here -- the i-RAM against what should be one of the best consumer hard drives for seeks.

 

[Airs]

Why are these values so far off from real-world tests? Real world tests don't even come close to 60MB/s, the slowest value in the HDTach tests.

There's writing involved in your real world tests. The HDtach numbers are pure read-based.

 

[Rubycon]

Improving seek time is the fastest way to improve random read times. This is particularly useful when a drive/array is used in a multi user environment. (ie server)

This is why 15k spindle speeds are used in heavy server loads whereas on the desktop they are seldom needed.

It gets far more complicated if you connect the disks to an intelligent host that performs caching of its own, however.

 

[Matthias99]

Originally posted by: airs
Sure the reads are quick, but there are writes involved too in your real world tests.

Sequential reads and writes tend to be roughly the same speed. Reads are faster, but usually not by a huge amount.

If you transfer tons and tons of very small files (or very heavily fragmented big files), seek times will start to cut into your transfer rate significantly. If your drive seeks in ~10ms on average, and can transfer ~60MBps sustained, each seek takes the same amount of time as transferring ~600KB of data. If you are transferring many (unfragmented) 1MB files, the workload looks like this:

(seek)->(transfer 1MB)->(seek)->(transfer 1MB)->...

With a ~60MBps STR, a 1MB transfer takes ~15ms. So the drive would end up spending almost half its time seeking. Whereas if you transfer 1GB files, you'll get much closer to the actual STR of the drive.

 

[Airs]

Let me re-phrase: comparing an artificial single drive read benchmark number to real world dual drive (one reading and one writing) is not apples to apples.

That said, I agree that file size plays a larger part.

 

[Seekermeister]

It's probably off the target of the subject being discussed, but when I used Dantz Express to clone one drive to another, it transferred about 19.5GBs in less than two hours. Since the OS being cloned was not just one big file, and the transfer rate was alot more than ~40 - 50MBs, how does that translate in terms of seek time?

 

[bob4432]

Originally posted by: Seekermeister
It's probably off the target of the subject being discussed, but when I used Dantz Express to clone one drive to another, it transferred about 19.5GBs in less than two hours. Since the OS being cloned was not just one big file, and the transfer rate was alot more than ~40 - 50MBs, how does that translate in terms of seek time?

not really sure - when i use acronis to create a image of my c drive (15K) and put it on my e drive (10K) that are both scsi drives on the same controller, it only takes ~7MINs and my c drive has ~18GB of data - os/apps/game/pagefile, etc

 

[Matthias99]

Originally posted by: bob4432

Originally posted by: Seekermeister
It's probably off the target of the subject being discussed, but when I used Dantz Express to clone one drive to another, it transferred about 19.5GBs in less than two hours. Since the OS being cloned was not just one big file, and the transfer rate was alot more than ~40 - 50MBs, how does that translate in terms of seek time?

not really sure - when i use acronis to create a image of my c drive (15K) and put it on my e drive (10K) that are both scsi drives on the same controller, it only takes ~7MINs and my c drive has ~18GB of data - os/apps/game/pagefile, etc

When you "clone" a whole drive or partition (as opposed to backing up files), normally it does a block-level copy -- it doesn't care about the filesystem, fragmentation, etc. You can think of it as sequentially transferring all the disk blocks that make up the filesystem, rather than copying each file individually. Not sure what you mean by "19.5GBs in less than two hours" and then referencing 40-50MBps... 20GB in 2 hours would be a whopping 2.75MBps (20,000MB over 7200 seconds).

7 minutes is 420 seconds... 18,000MB in 420 seconds is ~45MBps. Sounds about right for a block-by-block clone... Acronis also normally does compression, which can slow it down a little.

When I get home I can run some tests and give you HDTach numbers from my system.

Edit:

Numbers:

HDTach:

Drive 1 (Samsung SP2504C 250GB):
Random access: 14.0ms
Average read: 60.7MB/sec. (~72MB/sec. max, ~40MB/sec. min)
Burst read: 163.9MB/sec. (interesting, since this is a SATA150 drive...)

Drive 2 (Maxtor 7Y250M0 250GB):
Random access: 13.9ms
Average read: 49.8 MB/sec. (~60MB/sec. max, ~35MB/sec. min)
Burst read: 112.8MB/sec.

I'm not exactly sure why the Maxtor is noticeably slower. They're both on the same controller on the motherboard. I didn't defrag the drives beforehand, but I have Norton Systemworks defrag them every week, so they're usually very well-optimized.

Copying my "EA GAMES" folder (1,968,505,213 bytes, 1515 files in 216 folders): 1 minute 16 seconds. 1969MB in 76 seconds = 25.9MB/sec.

Copying a single large video file (2,135,134,234 bytes, 1 file): 53 seconds. 2135MB in 53 seconds = 40.0MB/sec.

Many small files = lousy copy performance due to the extra seeking.

 

[bob4432]

Originally posted by: Matthias99

Originally posted by: bob4432

Originally posted by: Seekermeister
It's probably off the target of the subject being discussed, but when I used Dantz Express to clone one drive to another, it transferred about 19.5GBs in less than two hours. Since the OS being cloned was not just one big file, and the transfer rate was alot more than ~40 - 50MBs, how does that translate in terms of seek time?

not really sure - when i use acronis to create a image of my c drive (15K) and put it on my e drive (10K) that are both scsi drives on the same controller, it only takes ~7MINs and my c drive has ~18GB of data - os/apps/game/pagefile, etc

When you "clone" a whole drive or partition (as opposed to backing up files), normally it does a block-level copy -- it doesn't care about the filesystem, fragmentation, etc. You can think of it as sequentially transferring all the disk blocks that make up the filesystem, rather than copying each file individually. Not sure what you mean by "19.5GBs in less than two hours" and then referencing 40-50MBps... 20GB in 2 hours would be a whopping 2.75MBps (20,000MB over 7200 seconds).

7 minutes is 420 seconds... 18,000MB in 420 seconds is ~45MBps. Sounds about right for a block-by-block clone... Acronis also normally does compression, which can slow it down a little.

When I get home I can run some tests and give you HDTach numbers from my system.

with acronis i have is set to normal compression so that does sound about right

 

[Seekermeister]

Matthias99,

When you "clone" a whole drive or partition (as opposed to backing up files), normally it does a block-level copy -- it doesn't care about the filesystem, fragmentation, etc. You can think of it as sequentially transferring all the disk blocks that make up the filesystem, rather than copying each file individually. Not sure what you mean by "19.5GBs in less than two hours" and then referencing 40-50MBps... 20GB in 2 hours would be a whopping 2.75MBps

Math has always been a weak point for me, but I feel worse about Dantz's performance than my own.