Backblaze: reliability update

[Soulkeeper]

https://www.backblaze.com/blog/hard-drive-reliability-update-september-2014/

Looks like seagate took a hit there, and the WD red drives did worse than I expected too.


Anyone with a facebook account want to post if they could separate their seagates by place of origin (china vs. thailand) ?

 

[Dahak]

Hmm... that does not look good for 3TB+ Drives.

But there could be two reason as mentioned in the article.

1) could be something with the "It may be that those drives are less well-suited to the data center environment."

2) External Drive harvesting, meaning they take out the drive that comes in the usb external enclosures.

Although they did not track the ones that came out of an enclosure, at least they did not mention it

 

[ronbo613]

I used to like Seagate, then I bought mostly WD hard drives. The past year or so I've been buying HGST(Hitachi), which is a subsidiary of WD. My largest WD drives are 2TB Reds and I have a few 1TB RE4's, all are working well. I'm a little sketched at the larger capacity drives, I have two 4TB HGST drives and so far, they are working fine.

Gotta stay ahead of the curve; no loyalty for hard drive companies anymore. Reliability gets my money.

 

[dave_the_nerd]

Hmm... that does not look good for 3TB+ Drives.

I'm curious about this. I suspect is has more to do with the amount of I/O on an individual drive. Assuming an even distribution of data to all the disks in an farm, a 3TB disk next to a 1TB would be 3x as likely to get hit, right?

 

[ethebubbeth]

My NAS is on 24x7 and hasn't had any drive failures yet. It's 24 drives with a 50/50 split between 3TB reds (WD30EFRX) and 3tb 5k RPM Hitachi (HDS5C3030ALA630). I've had 8 hitachis for 3 years, the remaining 4 I bought used on eBay and the WD Reds are about 8 months old (they were purchased new).

Here's my output from camcontrol devlist:
=========================================
<ATA WDC WD30EFRX-68E 0A80> at scbus0 target 24 lun 0 (da0,pass0)
<ATA WDC WD30EFRX-68E 0A80> at scbus0 target 25 lun 0 (da1,pass1)
<ATA Hitachi HDS5C303 A580> at scbus0 target 26 lun 0 (da2,pass2)
<ATA Hitachi HDS5C303 A580> at scbus0 target 27 lun 0 (da3,pass3)
<ATA WDC WD30EFRX-68E 0A80> at scbus0 target 28 lun 0 (da4,pass4)
<ATA WDC WD30EFRX-68E 0A80> at scbus0 target 29 lun 0 (da5,pass5)
<ATA Hitachi HDS5C303 A580> at scbus0 target 30 lun 0 (da6,pass6)
<ATA Hitachi HDS5C303 A580> at scbus0 target 31 lun 0 (da7,pass7)
<ATA WDC WD30EFRX-68E 0A80> at scbus0 target 32 lun 0 (da8,pass8)
<ATA WDC WD30EFRX-68E 0A80> at scbus0 target 33 lun 0 (da9,pass9)
<ATA Hitachi HDS5C303 A580> at scbus0 target 34 lun 0 (da10,pass10)
<ATA Hitachi HDS5C303 A580> at scbus0 target 35 lun 0 (da11,pass11)
<ATA WDC WD30EFRX-68E 0A80> at scbus1 target 0 lun 0 (da12,pass12)
<ATA WDC WD30EFRX-68E 0A80> at scbus1 target 1 lun 0 (da13,pass13)
<ATA WDC WD30EFRX-68E 0A80> at scbus1 target 2 lun 0 (da14,pass14)
<ATA WDC WD30EFRX-68E 0A80> at scbus1 target 3 lun 0 (da15,pass15)
<ATA WDC WD30EFRX-68E 0A80> at scbus1 target 4 lun 0 (da16,pass16)
<ATA WDC WD30EFRX-68E 0A80> at scbus1 target 5 lun 0 (da17,pass17)
<ATA Hitachi HDS5C303 A5C0> at scbus1 target 6 lun 0 (da18,pass18)
<ATA Hitachi HDS5C303 AA10> at scbus1 target 7 lun 0 (da19,pass19)
<ATA Hitachi HDS5C303 A580> at scbus1 target 8 lun 0 (da20,pass20)
<ATA Hitachi HDS5C303 AA10> at scbus1 target 9 lun 0 (da21,pass21)
<ATA Hitachi HDS5C303 A580> at scbus1 target 10 lun 0 (da22,pass22)
<ATA Hitachi HDS5C303 AA10> at scbus1 target 11 lun 0 (da23,pass23)

Here's the power on hours for the drives:
=========================================
Drive RAW_VALUE
da0 5799
da1 5307
da10 27350
da11 29129
da12 5818
da13 5174
da14 5797
da15 5121
da16 5816
da17 5185
da18 23061
da19 5824
da2 20881
da20 29120
da21 5823
da22 29108
da23 5823
da3 28116
da4 5797
da5 5311
da6 21089
da7 16288
da8 5817
da9 5189

So there's my anecdotal evidence to contribute.

 

[Concillian]

Hmm... that does not look good for 3TB+ Drives.

But there could be two reason as mentioned in the article.

1) could be something with the "It may be that those drives are less well-suited to the data center environment."

2) External Drive harvesting, meaning they take out the drive that comes in the usb external enclosures.

Although they did not track the ones that came out of an enclosure, at least they did not mention it

I think it's clearly at least #1, though I would suggest that there's potentially something in the Backblaze system design that is just plain not very compatible with the design of these drives.

If you check this article:
http://www.zdnet.com/who-makes-the-...s-backblaze-has-updated-its-stats-7000034008/

You see this information presented:

There are some hard drives that Backblaze has tried and won't use because they "just don&#8217;t work in the Backblaze environment". These include the energy-efficient "Western Digital Green 3TB drives and Seagate LP (low power) 2TB drives. Both of these drives start accumulating errors as soon as they are put into production," says Backblaze. "We think this is related to vibration."

There is something about the design and use of these systems that is inappropriate for the consumer level hard drives. I have no idea what that is, but there seems to be strong evidence of this.

Based on that, I'm not sure how directly this data translates to a typical usage scenario in a home PC. I feel like outside of the Hitachi's, the failure rates are high, and I find it hard to believe either WD or Seagate would be very successful with these kinds of failure rates on drives in actual consumer use.

 

[Soulkeeper]

You might be onto something.
Makes me wonder how much noise/vibration their environment has.

Edit: https://www.backblaze.com/blog/petabytes-on-a-budget-how-to-build-cheap-cloud-storage-2/
upto 45 drives in that one, dunno if that represents the majority of their configs or not ?

 

[Dahak]

You might be onto something.
Makes me wonder how much noise/vibration their environment has.

Edit: https://www.backblaze.com/blog/petabytes-on-a-budget-how-to-build-cheap-cloud-storage-2/
upto 45 drives in that one, dunno if that represents the majority of their configs or not ?

Yea, one thing that stands out is this

A note about drive vibration: The drives vibrate too much if you leave them sitting as shown in the picture above, so we add an &#8220;anti-vibration sleeve&#8221; (essentially a rubber band) around the hard drive in between the red metal grid and the drives

Really, a rubber band.. that is all they use for anti-vibration... seeing as this is custom designed rack would you not think that they would design it with something akin to the anti vibration grommets that most cases use on the HDD trays to put across the grid

There was something that I read a long while back (5-6 years) and I believe it was in some documentation for one of the hard drive manufactures, it stated that you should not mount your hard drives vertically. So I wonder if this comes into play as well. Not so much for a few drives(>5), but maybe there some odd resonance vibration that is affecting them too, one you start hitting 10+

 

[Coup27]

Interesting read, thanks. Confirms my feelings for a while that Seagate just aren't as reliable as WD. Impressed by Hitachi but looking at the prices, they are considerably more.