True SSD capacity - GiB and GB vs overprovisioning?

[lehtv]

I got a question for those who understand SSD's better than myself. True usable SSD capacity is different from advertised capacity, but why exactly, and is there a way to calculate it reliably?

Kingston's client SSD's are overprovisioned by 7%, see http://www.kingston.com/en/ssd/overprovisioning. So a 480GB drive will have roughly 447GB of actual capacity and that is what Windows reports as well, e.g. see Kingston UV400 review by legitreviews. But Windows reports GiB (1024^3 bytes), and I've thought both SSD and hard disk capacities are advertised in GB (1000^3 bytes), is this not true?

Finally, do you know if the overprovisioning % is different for different manufacturers or even among different product families?

 

[TheELF]

But Windows reports GiB (1024^3 bytes), and I've thought both SSD and hard disk capacities are advertised in GB (1000^3 bytes), is this not true?

Finally, do you know if the overprovisioning % is different for different manufacturers or even among different product families?

Yes ssd and hdd have always been advertised in a decimal matter (Gb) since it's less confusing for average joe,while all OSs report in binary.

overprovisioning does differ and the drive utilities often even allow you to change the amount of it.

 

[dave_the_nerd]

Finally, do you know if the overprovisioning % is different for different manufacturers or even among different product families?

It is.

I got a question for those who understand SSD's better than myself. True usable SSD capacity is different from advertised capacity, but why exactly, and is there a way to calculate it reliably?

Kingston's client SSD's are overprovisioned by 7%, see http://www.kingston.com/en/ssd/overprovisioning. So a 480GB drive will have roughly 447GB of actual capacity and that is what Windows reports as well, e.g. see Kingston UV400 review by legitreviews. But Windows reports GiB (1024^3 bytes), and I've thought both SSD and hard disk capacities are advertised in GB (1000^3 bytes), is this not true?

SSDs start with a round number of GiBs of storage, since the NAND chips are made with that. So starting with 512GiB of NAND, chop of 7%, you get just about 512 GB (1000^3). Chop off some more for an SLC cache (some SSDs do that) and you may even end up with less than that.

But you're absolutely right that they're sold using GB and show up in Windows using GiB.

 

[dave_the_nerd]

...while all OSs report in binary.

MacOS reports in decimal. So do some versions of Linux.

I think Windows can be told to work either way...?

 

[lehtv]

How can they be sold using GB, like hard disks, if the NAND means they have a round number of GiB capacity, unlike hard disks? They can't end up with a round number of GB to advertise... Whereas a 1TB hard disk is exactly 10^12 bytes.

Considering the Kingston UV400 drive, am I doing this right:

Exactly 480 GiB to start with -> 7% overprovisioning so 480/1.07 -> left with 448.598... GiB reported as 447 GiB in Windows. But that should round to 449, not 447... Maybe the overprovisioning amount is not exactly 7%?

479,967,834,112 bytes = 447.0048789978027 GiB

Would suggest overprovisioning % is 1 - (480 / 447.004...) -> about 7.38%

 

[Puffnstuff]

I know how you feel when the drives are advertised to have a certain capacity only to install it and see that windows reports it another way showing much less capacity. There should be an agreed upon reporting standard that drive manufacturers and OS makers both adhere to in order to avoid consumer confusion.

 

[Hellhammer]

How can they be sold using GB, like hard disks, if the NAND means they have a round number of GiB capacity, unlike hard disks? They can't end up with a round number of GB to advertise... Whereas a 1TB hard disk is exactly 10^12 bytes.

1TB doesn't have to be exactly 1000^4 bytes. See the IDEMA spec for calculating advertised capacities (it's a download link so don't be surprised).

Considering the Kingston UV400 drive, am I doing this right:

Exactly 480 GiB to start with -> 7% overprovisioning so 480/1.07 -> left with 448.598... GiB reported as 447 GiB in Windows. But that should round to 449, not 447... Maybe the overprovisioning amount is not exactly 7%?


479,967,834,112 bytes = 447.0048789978027 GiB

Would suggest overprovisioning % is 1 - (480 / 447.004...) -> about 7.38%

Kingston UV400, like all client drives advertised as 480GB, start with 512GiB of raw NAND. Since the usable space is 480GB, the over-provisioning is:

1 - [ (480*1000^3) / (512*1024^3) ] = 12.7%

Some manufacturers mistakenly consider over-provisioning as the capacity between 512GB and 480GB (which yields 6.3% OP), but that is incorrect because the Gibi to Giga translation must be taken into account to provide an accurate percentage.

 

[TheELF]

Exactly 480 GiB to start with -> 7% overprovisioning so 480/1.07 -> left with 448.598... GiB reported as 447 GiB in Windows. But that should round to 449, not 447... Maybe the overprovisioning amount is not exactly 7%?

You do not start at 480GiB, 480 is already with the overprovisioned part subtracted.

The rest is just because of the conversion from GiB to Gb.
http://wintelguy.com/gb2gib.html

 

[JimmiG]

You do not start at 480GiB, 480 is already with the overprovisioned part subtracted.

The rest is just because of the conversion from GiB to Gb.
http://wintelguy.com/gb2gib.html

Correct.
480 gigabytes = 447 gibibytes, which is what Windows reports in your screenshot.

The over-provisioning is completely transparent to the OS and handled by the drive itself. The raw capacity might be 500 gigabytes, 512 gigabytes or even 1024 gigabytes, but the drive is sold as 480 gigabyte drive, and reports a capacity of 480 gigabytes.

 

[lehtv]

Ah, that makes more sense!

What about drives advertised as 512GB or 500GB? Do they start with 512GiB as well, but they just have a smaller % of overprovisioning compared to 480GB drives?

In case of 512GB:
1 - [ (512*1000^3) / (512*1024^3) ] = 6.87%

In case of 500GB:
1 - [ (500*1000^3) / (512*1024^3) ] = 9.05%

 

[nerp]

I know how you feel when the drives are advertised to have a certain capacity only to install it and see that windows reports it another way showing much less capacity. There should be an agreed upon reporting standard that drive manufacturers and OS makers both adhere to in order to avoid consumer confusion.

This has been the way things have been since forever, though. People have been whining about advertised/real capacity since I first touched a computer in the early 1980s.

 

[hojnikb]

Ah, that makes more sense!

What about drives advertised as 512GB or 500GB? Do they start with 512GiB as well, but they just have a smaller % of overprovisioning compared to 480GB drives?

In case of 512GB:
1 - [ (512*1000^3) / (512*1024^3) ] = 6.87%

In case of 500GB:
1 - [ (500*1000^3) / (512*1024^3) ] = 9.05%

Exactly. But to complicate things even more, there is actually more than 512GiB of raw flash on a 512GB/500GB/480GB, but it's not accessible

The secret is that the NAND dies are not really 128Gbit – they are in fact much larger than that. SanDisk could not give us the exact size due to competitive reasons, but told us that the 128Gbit number should be treated as MLC for it to make sense. Since TLC stores three bits per cell instead of two, it can store 50% more data in the same area, so 128Gbit of MLC would become 192Gbit of TLC. That is in a perfect world where every die is equal and there are no bad blocks; in reality TLC provides about a 30-40% density increase over MLC because TLC inherently has more bad blocks (e.g. stricter voltage requirements because there is less room for errors due to narrower distribution of the voltage states)..

 

[Hellhammer]

Exactly. But to complicate things even more, there is actually more than 512GiB of raw flash on a 512GB/500GB/480GB, but it's not accessible

There is more if you count the spare bytes, but those are pretty much solely used for ECC purposes and aren't freely accessible by the controller.

SanDisk Ultra II was a special case because it combined TLC NAND with a controller that wasn't designed for TLC. Hence SanDisk had to resort to a ton of parity to correct errors with subpar BCH ECC. The Ultra II was purely a show to the investors that SanDisk can do TLC because from the economical perspective it makes zero sense to take a 128Gbit MLC die, treat it as TLC and then use most of the gains in extra ECC to make TLC work in the first place.

 

[mv2devnull]

Example. This unit was marketed as "128GB":

$ sudo gdisk -l /dev/sda
GPT fdisk (gdisk) version 0.8.10

Found valid GPT with protective MBR; using GPT.
Disk /dev/sda: 250069680 sectors, 119.2 GiB
Logical sector size: 512 bytes
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 250069646

$ sudo smartctl -i /dev/sda
smartctl 5.43 2012-06-30 r3573 [x86_64-linux-2.6.32-642.1.1.el6.x86_64] (local build)
Copyright (C) 2002-12 by Bruce Allen, http://smartmontools.sourceforge.net

=== START OF INFORMATION SECTION ===
Model Family:     Samsung based SSDs
Device Model:     SAMSUNG SSD 830 Series
Firmware Version: CXM03B1Q
User Capacity:    128,035,676,160 bytes [128 GB]
Sector Size:      512 bytes logical/physical

No over-provisioning in sight (but 830 is "old tech" by now).

Filesystem metadata does consume some space too, so looking for "available space" of "empty disk" can show less than the size of a partition.

 

[hojnikb]

There is more if you count the spare bytes, but those are pretty much solely used for ECC purposes and aren't freely accessible by the controller.

SanDisk Ultra II was a special case because it combined TLC NAND with a controller that wasn't designed for TLC. Hence SanDisk had to resort to a ton of parity to correct errors with subpar BCH ECC. The Ultra II was purely a show to the investors that SanDisk can do TLC because from the economical perspective it makes zero sense to take a 128Gbit MLC die, treat it as TLC and then use most of the gains in extra ECC to make TLC work in the first place.

So sandisk made a special die just for ULTRA II ?

Or is it common for other NAND makers to have "128Gbi" die, thats actually much bigger ?

And how much of that extra die space is addressable by the flash controller ?

 

[Puffnstuff]

This has been the way things have been since forever, though. People have been whining about advertised/real capacity since I first touched a computer in the early 1980s.

Just because it's the status quo doesn't make it right and it should be changed.

 

[Cerb]

Example. This unit was marketed as "128GB":

$ sudo gdisk -l /dev/sda
GPT fdisk (gdisk) version 0.8.10

Found valid GPT with protective MBR; using GPT.
Disk /dev/sda: 250069680 sectors, 119.2 GiB
Logical sector size: 512 bytes
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 250069646

$ sudo smartctl -i /dev/sda
smartctl 5.43 2012-06-30 r3573 [x86_64-linux-2.6.32-642.1.1.el6.x86_64] (local build)
Copyright (C) 2002-12 by Bruce Allen, http://smartmontools.sourceforge.net

=== START OF INFORMATION SECTION ===
Model Family:     Samsung based SSDs
Device Model:     SAMSUNG SSD 830 Series
Firmware Version: CXM03B1Q
User Capacity:    128,035,676,160 bytes [128 GB]
Sector Size:      512 bytes logical/physical

No over-provisioning in sight (but 830 is "old tech" by now).

128GiB = 137,438,953,472B, not 128,000,000,000B. That's a typical 7% or so.

 

[mv2devnull]

128GiB = 137,438,953,472B, not 128,000,000,000B. That's a typical 7% or so.

Yes, depending on what has been marketed.

The user accessible capacity is clearly 119.2 GiB, which equals 128 GB.
If the "128GB" in product description means 1000-based 128 GB, as is usual in the marketing, then the user accessible capacity equals the marketed value.
If the "128GB" in product description means "128 GiB physical capacity of which only 119.2 GiB is user accessible", then there is visible overprovisioning.

 

[nerp]

Just because it's the status quo doesn't make it right and it should be changed.

Well, when something has been a standard for decades, it's a lot harder to make the change. Another way of dealing with this is when you buy a storage device, assume that the advertised capacity is unformatted and you will expect a little less. Then you no longer get upset.

 

[Mike64]

This has been the way things have been since forever, though. People have been whining about advertised/real capacity since I first touched a computer in the early 1980s.

:thumbsup:

Before the Internet, it was bad enough (I got/get that not everyone read Byte or Dr Dobb's Journal.) 20+ years after it (when virtually everyone in the US, their grandmother, and their pre-schoolers) has Internet access), my blood pressure now jumps about a dozen points every time I see Yet Another Stupid Product Review complaining about it. It is one of many things that makes me wonder how so many people manage to make it through their days without becoming roadkill...

Well, when something has been a standard for decades, it's a lot harder to make the change. Another way of dealing with this is when you buy a storage device, assume that the advertised capacity is unformatted and you will expect a little less. Then you no longer get upset.

Seriously. Buying computer storage is not like buying meat at the supermarket. How many people go shopping for a storage device because they've actually calculated what they'll need over a given time period and decided on "X TB"?/:biggrin:

 

[dave_the_nerd]

:thumbsup:

Before the Internet, it was bad enough. 20+ years after it, my blood pressure now jumps about a dozen points every time I see Yet Another Stupid Product Review complaining about it. It is one of many things that makes me wonder how so many people manage to make it through their days without becoming roadkill...

As storage capacities have gotten larger, both the absolute and relative differences between binary and decimal has gotten larger.

When a 40 MB hard drive was normal, it formatted to 39MB (2.4% loss) and nobody really cared, if they even noticed. The manufacturers said "formatted capacity something something" and people accepted it.

But when your brand new 4TB HDD is "missing" more storage space than your last computer HAD? (275GB or so, or about 7% of the total) That's worth calling customer service.

I think psychologically, certain capacities suck more than others. For instance, 750GB drives which are ~690GiB That '6' is a kick in the teeth.

 

[Mike64]

I think psychologically, certain capacities suck more than others. For instance, 750GB drives which are ~690GiB That '6' is a kick in the teeth.

I think you must just be a much more "understanding" person than I am.

I mean, sure, the first time someone buys a flash drive and discovers this heinous miscarriage of deceptive advertising, I can understand their being mildly annoyed, and conceivably even calling or emailing the company whose product they bought. But worked up enough to post angry screeds in the guise of "product reviews" or demand that the industry change decades of what is, after all, basically a harmless practice when one is familiar with the basis for it? I mean, no one ever gets the mileage per gallon car companies advertise and kids' toys are never as much fun as they look in the commercials either, but you don't often see people foaming at the mouth all over the Internet over those things...

 

[dave_the_nerd]

I think you must just be a much more "understanding" person than I am.

I mean, sure, the first time someone buys a flash drive and discovers this heinous miscarriage of deceptive advertising, I can understand their being mildly annoyed, and conceivably even calling or emailing the company whose product they bought, but worked up enough to post angry screeds in the guise of "product reviews" or demand that the industry change decades of what is, after all, basically a harmless practice when one is familiar with the basis for it? I mean, no one ever gets the mileage per gallon car companies advertise and kids' toys are never as much fun as they look in the commercials either, but you don't seen people foaming at the mouth all over the Internet over those things...

I spent a lot of years doing desktop support. Seeing it from the viewpoint of the most-ignorant-user-possible is pretty much automatic.

But seeing that kind of whaarrrgargbl from somebody who's pretending to be an authority (doing storage reviews) is kind of a telling sign.

 

[Mike64]

I spent a lot of years doing desktop support.

Oh dear. In my book, that's up there just couple of rungs below teaching junior high school. Somebody's got to do it, but thank God that somebody isn't me.:biggrin: You have my sympathy and respect.

But seeing that kind of whaarrrgargbl from somebody who's pretending to be an authority (doing storage reviews) is kind of a telling sign.

To be fair, the foaming I meant was the ranting one-star reviews on retailer websites in which people angrily vow never to buy another of Company X's products as long as they live because the drive's capacity didn't exactly match the stated spec... (And fwiw, if one is going to get worked up over flash memory spec in ads or product listings, I think speed specs are much more fertile ground... "Transfer" speed? Sequential read/write speeds as a primary guide to "typical" performance? Etc) I will say I don't think I've ever seen a professional/"serious" reviewer more than mention the issue in passing, nor for that matter, a blog-reviewer, though I don't pay much attention to the latter except for any hard numbers I might come across when googling for specs or benchmarks...

And yes, I know this thread is about SSDs, but the principle is the same and frankly, if anything, I'd expect more from someone buying a replacement for an internal PC component than a plug&play device like a flash drive or card.

 

[nerp]

As storage capacities have gotten larger, both the absolute and relative differences between binary and decimal has gotten larger.

When a 40 MB hard drive was normal, it formatted to 39MB (2.4% loss) and nobody really cared, if they even noticed. The manufacturers said "formatted capacity something something" and people accepted it.

But when your brand new 4TB HDD is "missing" more storage space than your last computer HAD? (275GB or so, or about 7% of the total) That's worth calling customer service.

I think psychologically, certain capacities suck more than others. For instance, 750GB drives which are ~690GiB That '6' is a kick in the teeth.

Fair points. There is a psychological element here.