Over a Petabyte of storage in a single namespace or volume!

[Voo]

3 HDD under best case scenario to match 1 SSD.

What about... 3 SSDs?

For more than three times the price? Great deal that! Basically you try to sell "Pay more for less performance and only a tenth of the capacity!" - I'm sure people will flock to that marketing

Filling up the drive is a completely sequential problem, so if you're limited by price (and most people are) you can get a higher sequential write with HDDs than SSDs for the same price (ie HDDs will copy the stuff faster than SSDs) - and at the same time get MUCH, MUCH more space.

For mass storage under the mentioned circumstances (ie random r/w not important) I don't see any reason to go with SSDs at this point in time. If SSDs somewhere in the future offer the same capacity for a similar price we can talk about it again, but at the moment we're far, far away from that.

Also funny you say AI and computer vision isn't happening - I assume I should tell that some colleagues of mine, they may be a bit surprised though.. (and considering some classical papers in these areas are decades old that's somewhat entertaining )

@Puppies04 An OS drive has other requirements than mass storage - ie random r/w and access latency in itself ARE important metrics which changes the picture completely. We're only talking about mass storage here, eg storing TBs of pictures, movies or backups. And yes you can get a really good price for some of the small SSDs especially if you consider deals so the argument isn't completely fool proof - but then the small SSDs also get worse write speeds and I limited myself to "larger SSDs" to avoid nitpicking in that regard

 

[exdeath]

Less performance? You kidding? The fastest single HDD is what, 125 MB /s?

Lol first generation SSDs have that beat and current SSD are nearly 5 times that from a single drive. And it just gets worse in massive arrays where SSDs sequential will hit TB/s speeds.

Sequential speed is important too. Doesn't matter if you can put a million pictures on it, how long does it take to put then there in the first place, or run backups? Just moving 100 GB off a HDD feels like draining the ocean with a straw.

I realize I'm going overboard here and be being fanatical about SSD speed, but cost per GB is the ONLY thing spindles have left. If you look at cost per GB/s, it takes 5 HDDs to match the performance of a single SSD even in just sequential. And if that file is two or more parts, forget it.

As for capacity, I don't care. Its about time we get back to the days when a printer driver wasn't a 400 MB download anyway... its mostly wasted space since people use it lazily just because its there. Hoarding: Buried Alive Digital Edition anyone? To be fair though I'm used to limited space, since I've had nothing but Cheetahs and Raptors since their debut, so moving to SSD isn't constraining to me at all. I've never been one to need to save the entire internet to my PC :AWE:

You can probably safely ignore me from here on out, I won't be satisfied until our data storage and retrieval capabilities are 1:1 with the CPU/bus speed. Only then will we have the next major sci fi to reality revolution, seeing that instant access computers is one of the remaining sci fi staples we've yet to realize and every major field in computing is now kneecapped by its nonvolatile storage speed.

 

[Voo]

Lol first generation SSDs have that beat and current SSD are nearly 5 times that from a single drive.

Nope, modern HDDs get 150mb/s sequential speed and more (haven't seen those new 1tb platter drives tested). And 5times 125mb would be 625mb/s - you severely overestimate SSDs. See here. 380mb/s (the revo drive with 4raided SSDs obviously not counting) is quite a bit lower than 625mb.

And what the hell are you talking about, we're ONLY talking about sequential speed here when transferring data. And if you really don't need the capacity, great for you - but what exactly are you doing arguing about mass STORAGE if you don't need any? And if you really think nobody needs more than 250gb - that's nice for you, other people like to store backups, media (look what a single BR movie takes) or photos (guess what size the average raw generated by my camera has) or isos have.

To keep it really simple:
HDDs sequential r/w: 150mb/s
SSDs sequential w: 380mb/s (and that's the fastest, largest SSD on the market basically)
SSDs sequential r: 420mb/s (not the same SSD as for writing, so in reality you'd lose a bit on one metric)

So a SSD is equal to 2.5 HDDs for writing and 2.8 HDDs for reading. And as a nice bonus you get more than 10times the capacity with more performance out of the 3 drives.

Or to keep it really simple: For sequential speed you can get MORE performance/$ out of cheap raided HDDs than expensive SSDs.

So you can turn it around all you want: There's no reason to go with SSDs for mass storage.

 

[alyarb]

I predict someone will be exabyte-stage left soon.

alyarb likes this.

 

[exdeath]

Guess you completely missed the 500 + MB/s class SATAIII SSDs that have been out for several months now.

I admit Im a SSD snob. I still use HDDs for backup... which is fitting seeing as how they were invented to replace tape reels some 60 years ago...

 

[Voo]

Guess you completely missed the 500 + MB/s class SATAIII SSDs that have been out for several months now.

Ok and which one those would be? Because as far as I see the AT storage bench doesn't miss any special controllers. Marvell? SF2200? Intel? All there.

And even if, then we'd have to add a fourth drive to certainly beat it - still cheaper, even more space and as fast. But then only if there were such an ominous SSD that could get 500mb/s write speed out of compressed data (read may be possible though especially if one uses as SSD and not iometer [but then one should maybe use the same benchmark for all numbers, don't you agree?], but then 420 or 500 doesn't change the argument much. But write? I beg to differ)

 

[taltamir]

Nope, modern HDDs get 150mb/s sequential speed

Can you name a spindle HDD that gets 150mb/s ?

And 5times 125mb would be 625mb/s

people don't don't run their 5 drive arrays in RAID0. A very expensive controller can make a HDD array reach those kinds of speed (with more than 5 drives)... but you are comparing HDD raid to SSD raid.

The real argument against that guy is that SSDs are EXPENSIVE.
Its like saying "I can't see why doesn't anyone use plasma instead of CRTs" back when they cost more then a new midrange car (most expensive plasma screen I saw was a 50 inch screen for over 20,000$ I think it was 21k IIRC which is more than you pay for a brand new full sized sedan)

Give it another 10-15 years and everyone will be using SSD exclusively. But only because prices went down not because suddenly people would "see the light"

 

[Voo]

Can you name a spindle HDD that gets 150mb/s ?

The first HDD storage review article lists this - not exactly 150mb/s but 140mb in iometer should be 150 in as if HDDs behave similar to SSDs in that regard.

people don't don't run their 5 drive arrays in RAID0. A very expensive controller can make a HDD array reach those kinds of speed (with more than 5 drives)... but you are comparing HDD raid to SSD raid.

Oh I think 3 HDDs in Raid0 are still somewhat reasonable and if we want redundancy we can add some HDDs and go with a more reliable RAID - still cheaper than the SSD. The argument is just that at the moment SSDs are expensive enough that you can RAID together a handful cheap HDDs and get better sequential performance for a similar price and at the same time much more capacity.

Give it another 10-15 years and everyone will be using SSD exclusively. But only because prices went down not because suddenly people would "see the light"

Don't see why I should argue with that - if we get cheap SSDs with large capacities compared to HDDs that'll be great.

 

[taltamir]

The first HDD storage review article lists this - not exactly 150mb/s but 140mb in iometer should be 150 in as if HDDs behave similar to SSDs in that regard.

Well, he said 125mb/s
You said 150mb/s
The review is showing 139mb/s
Yours is a little closer, but only a little.

 

[Voo]

Well, he said 125mb/s
You said 150mb/s
The review is showing 139mb/s
Yours is a little closer, but only a little.

Yeah and I also cited an imaginative SSD that gets 380w and 420r in iometer, taking the best scores of both (and rounding up). Somehow I thought it was quite obvious that having the numbers exact to the byte wouldn't really change much of the argument, but I can retype it with 150 replaced with 140 if you want to.

 

[taltamir]

Yeah and I also cited an imaginative SSD that gets 380w and 420r in iometer, taking the best scores of both (and rounding up). Somehow I thought it was quite obvious that having the numbers exact to the byte wouldn't really change much of the argument, but I can retype it with 150 replaced with 140 if you want to.

its fine actually. rereading you seem to have been refuting his point that SSDs are 5x the sequential speed of spindle HDD. Which is definitely wrong. I didn't notice that this was what you were doing (due to reading failure on my part).

Here are some more accurate speeds for SSD sequential if someone is curious
http://www.anandtech.com/show/4712/the-crucial-m4-ssd-update-faster-with-fw0009/3

 

[LarryRAguilar]

What game do you host? Not COD 4 by any chance! I addicted to that dang game! :biggrin:

Actually the 4 bottom SAN enclosures were basically headed to the garbage so I managed to snag them. Given the age and the fact that it's proprietary (can't pop my own drives in it) I only use it for low risk stuff like backups... on top of my existing solution.

The middle server is actually most of my environment. Have like 5 VMs on there from torrents VMs dedicated to my game server dev/test environment. The server has 3.7TB of storage. The top server is a firewall and then the switches are on the back. Mostly all personal stuff. I would love to host services given I have the physical room, but with 5mbps down and 512k up, it would kinda suck.

So it's mostly just hobby stuff. The SAN is fiber channel too. It's kinda neat as it's hooked up to a HBA on that little 1U server that's just above and the OS sees all the drives. I only have the top enclosure allocated right now. I turn it on maybe once a month to run a full backup of my main server. I was writing an article on mdadm raid the other day so decided to use one of the bottom enclosures and all the drives started failing lol. This was actually hosting a corporate environment at one point. They got rid of it just in time.

It looks pretty with all the blinking lights though.

 

[Red Squirrel]

What game do you host? Not COD 4 by any chance! I addicted to that dang game! :biggrin:

Nah, it's an older game, Ultima Online. Been running a shard since around 2007. http://www.uovalor.com

 

[LarryRAguilar]

Exdeath, I agree with you, and I think technology is moving towards SSDs. The problem lies in the cost of SSDs, at the current cost, it is not cost effective to implement a solution using SSDs. Just give it time!

 

[LarryRAguilar]

Speaking of movies, check this out!
Believe it or not:Avatar takes 1 petabyte of storage space, equivalent to a 32 YEAR long MP3.

 

[ViviTheMage]

I bet those are 7200RPM disks ... not really something i'd want to host a bunch of VM's on.

Maybe my D drive, but nothing OS/database ...

still chool .

 

[LarryRAguilar]

Yeah, I think they are 3TB 7200 SAS HDD's, but Im sure you could use 3TB 15k SATA drives.

 

[KentState]

Exdeath, I agree with you, and I think technology is moving towards SSDs. The problem lies in the cost of SSDs, at the current cost, it is not cost effective to implement a solution using SSDs. Just give it time!

That's where tiered storage comes in to play. Move the most access dataed up to SSD as needed in the background and you get the benefit of large slow disk prices and SSD speed. Only a very small percentage of data is continually accessed in most businesses so why spend the money on the 75% that can sit in near line SAS.

BTW, ZFS is not unlimited in size so a 1PB or even 100PB server is possible. Only takes enough controllers, space and processing power to pull it off.

 

[mv2devnull]

Damn, so yeah 1PT in a single 42U rack is quite impressive. Looks like they have 360 3TB drives, and their enclosures look like they hold 45 drives. Guess they're just designed more space efficiently.

I'm sure that at least Hitachi has a "high density storage expansion tray". 48 HDD in 4U (but only 38 15k rpm SAS HDD due to cooling), and that you can link 10 of them to their SAN controller. So around 450 drives per rack. That thing would weight a ton though.

Your regular enclosure allows you to hotswap from the front. In those "high density" models the backplane is at the bottom of the enclosure and you push the drives down to it.

 

[LarryRAguilar]

That's where tiered storage comes in to play. Move the most access dataed up to SSD as needed in the background and you get the benefit of large slow disk prices and SSD speed. Only a very small percentage of data is continually accessed in most businesses so why spend the money on the 75% that can sit in near line SAS.

BTW, ZFS is not unlimited in size so a 1PB or even 100PB server is possible. Only takes enough controllers, space and processing power to pull it off.

BTW:ZFS is unlimted in size, it is a software RAID. There a couple of software companies that offer this technology, but i think this one is using Nexentastor.

 

[taltamir]

ZFS isn't unlimtied. If using 4k sectors a 32bit FS can store 16TiB of data, while ZFS could store 1267650600228229401496703205376 TiB using 4k sectors.
aka 1237940039285380274899124224 PiB (petabyte)
aka 1208925819614629174706176 EiB (exabyte)
aka 1180591620717411303424 ZiB (zettabyte)
aka 1152921504606846976 YiB (yottabyte)

There are no names for units above yottabyte

It is a very very large number, but it isn't unlimited.