Long term performance of SSDs. Analysis of Intel SSDs.

[n7]

Don't stop with the friendly banter, but please ensure it stays friendly banter.

n7
Memory/Storage Mod

Now continue please...

 

[Idontcare]

Originally posted by: fuzzybabybunny
Just read this recent article:

Long-term performance analysis of Intel Mainstream SSDs
http://www.pcper.com/article.php?aid=669

It shows very significant performance decreases as SSDs are used for longer periods of time.

Intel came out officially and has stated they cannot duplicate the findings of the pcper article:

Intel replies to solid-state drive 'slowness' critique

After a technology review site claimed Intel solid-state drives slow considerably after extended use, Intel said it has not been able to duplicate the results.

http://news.cnet.com/8301-13924_3-10168084-64.html

 

[Cookie Monster]

Sp basically Intel is saying that in the "real" world situation, average PC users wont be affected so much with the degradation of a SSD in performance which is probably true. However denying the fact that when SSDs are used heavily, they can really take a hit when it comes to performance as shown by PcPer and there is no lie about this.

 

[IntelUser2000]

coolvariable:

I'm pretty sure I can say the same to you. Did you even click on the link?? If you click on it and see the HDTune results the Samsung fluctuates massively on the write benchmark, which if it was as you claim the numbers would be higher.

You seriously believe that factory driven write cycle will simulate the true steady-state of the actual drive??

We did not say we were experts but these are facts.

 

[IntelUser2000]

Originally posted by: taltamir

And taltamir, I'm pretty sure its not NCQ that does the wear levelling part of the Intel SSD because what if NCQ was disabled?

I never said NCQ does wear leveling, i said NCQ aware wear leveling can reduce the impact of steady state and your write amplification. And according to one of the articles on the subject the intel controller does just that... but I have only read it in one location and it might be bull. However it makes sense.
Wear leveling at its simplest will try to, by itself, write each block to the least written to block on the drive. If you combined it with NCQ it will still try to do the same, but it will now have the added benefits of doing read (512K circuit)-modify with SEVERAL pending 4k writes at once-erase entire 512K-write modified data.

Right I guess I made a mistake. What I meant to say is the NCQ doesn't do anything to reduce write amplification rather the controller.

NCQ is probably an aid to make it even better because it can put outstanding IOs before the controller even needs it.

 

[coolVariable]

Haha.
Looks like the PCPer article caused another price drop for the intel 80GB x-25M:
$358!!!! On Newegg! Can't wait to see another price drop, once the Vertex lands.

http://www.newegg.com/Product/...x?Item=N82E16820167005

 

[Idontcare]

Originally posted by: Cookie Monster
Sp basically Intel is saying that in the "real" world situation, average PC users wont be affected so much with the degradation of a SSD in performance which is probably true. However denying the fact that when SSDs are used heavily, they can really take a hit when it comes to performance as shown by PcPer and there is no lie about this.

I see Intel saying two things - first they appear to be claiming that even if a bench were to induce some form of performance degradation in the drive, a bench doesn't reflect real-world usage patterns and as such they would not expect customers to experience the same degradation issues (unless they run the bench, then their system is fubar'ed).

The second thing Intel appears to be saying is that regardless of the above, they can't get the benches to cause the same problems that pcper accomplished. This is a big discrepancy. If pcper's results cannot be duplicated then that is very bad for pcper's credibility.

 

[magreen]

Originally posted by: Idontcare

Originally posted by: Cookie Monster
Sp basically Intel is saying that in the "real" world situation, average PC users wont be affected so much with the degradation of a SSD in performance which is probably true. However denying the fact that when SSDs are used heavily, they can really take a hit when it comes to performance as shown by PcPer and there is no lie about this.

I see Intel saying two things - first they appear to be claiming that even if a bench were to induce some form of performance degradation in the drive, a bench doesn't reflect real-world usage patterns and as such they would not expect customers to experience the same degradation issues (unless they run the bench, then their system is fubar'ed).

Yeah, Intel said the same thing about the FDIV bug.

The second thing Intel appears to be saying is that regardless of the above, they can't get the benches to cause the same problems that pcper accomplished. This is a big discrepancy. If pcper's results cannot be duplicated then that is very bad for pcper's credibility.

They certainly have every motive to not succeed in duplicating the results themselves (at least not admitting to it), and they may only be testing in what they call "normal usage" scenarios, not the "extreme" scenarios of pcper.

 

[magreen]

Originally posted by: taltamir

Originally posted by: coolVariable
Link?
Samsung and Mtron ship their SSDs after a full write cycle, that is steady-state!
Don't really see where you could get another performance drop from.

Then you obviously don't understand what steady state is.

Originally posted by: Viper GTS

Originally posted by: coolVariable
Link?
Samsung and Mtron ship their SSDs after a full write cycle, that is steady-state!
Don't really see where you could get another performance drop from.

There is more to it than just a lack of unwritten cells available. Intel documents that it varies by workload/access pattern, and switching workloads requires a new cycle of conditioning the drive to obtain steady-state results.

I believe taltamir is correct overall in this debate, but he has not explained himself fully, and is getting sidetracked into semantics and debating the definition of the term "steady state," instead of the main issue which is the performance degradation.

coolVariable, I believe the issue here is that you are characterizing the long-term performance degradation that pcper reported by using the term "steady state." And you are saying that if manufacturers perform an entire full write cycle before shipping, the drive will be in "steady state" already.

I do not actually know the definition of the term "steady state" as applied to an SSD drive, and I don't think it's useful to resolving this debate. It's not even clear to me that there is a unique definition, so I will avoid the term.

What is useful to resolving this debate, however, is determing whether there is a performance degradation that can occur even after a full write-cycle to the drive. CoolVar, you claim that after a full write cycle, any performance degradation would have already occured and there will therefore be no further performance degradation. You claim that for this reason, the Samsung and mtron drives are free of degradation.

This is contradicted by the characterization of the performance degradation reported in the pcper article, and assuming that their article is correct, there is a performance degradation that occurs even after a full write cycle.

The pcper article claims that the performance degradation is due to "internal fragmentation as a result of the mix of small / large files being written to it."(1) A complete write cycle, i.e. merely filling all available blocks, wouldn't accomplish this fragmentation.

Additionally, the pcper article says that taking a fresh drive and merely imaging 20GB of OS files back to it causes a drop in performance. Their explanation of it is, "What you see is the result of internal fragmentation caused primarily by write combining. The drops in read speed are also caused by this process."(2) Now, assuming that their explanation is correct, the degradation is not simply due to the sectors having already been written and that the blocks must then be erased before being written, causing a slowdown. The performance degradation has much more to do with fragmentation.

I do not know whose definition of "steady state" is correct, but on the issue of performance degradation, I believe taltamir is correct.

 

[Viper GTS]

For what it's worth the testing I did on a 32 GB SLC drive over-wrote the entire drive in about 45 minutes. We saw degradation over a period of several hours before it finally leveled off and we took our final measurements.

Viper GTS

 

[GourdFreeMan]

Originally posted by: Idontcare
I see Intel saying two things - first they appear to be claiming that even if a bench were to induce some form of performance degradation in the drive, a bench doesn't reflect real-world usage patterns and as such they would not expect customers to experience the same degradation issues (unless they run the bench, then their system is fubar'ed).

I suspect this is what Intel is actually saying, but since the cnet article does not quote its source, we can only speculate. In any case it seems cnet, or the Intel marketing department that contacted them seems to presume "real world" use consists of installing the Windows OS once, doing some occasional web browsing, playing minesweeper, listening to music, and that is about it. God forbid any user actually benchmark their hardware, compile large applications, fill their SSD up, put a pagefile on their SSD when system memory is inadequate, or run a database server on an SSD.

Originally posted by: Idontcare
The second thing Intel appears to be saying is that regardless of the above, they can't get the benches to cause the same problems that pcper accomplished. This is a big discrepancy. If pcper's results cannot be duplicated then that is very bad for pcper's credibility.

This has to be patently untrue. Intel's own correspondance with ssdworld when this issue was originally unconvered four months ago clearly admits the problem exists, and even offers work-arounds. Here is the relevant quote directly from Intel:

SSDs all have what is known as an ?Indirection System? ? aka an LBA allocation table (similar to an OS file allocation table). LBAs are not typically stored in the same physical location each time they are written. If you write LBA 0, it may go to physical location 0, but if you write it again later, it may go to physical location 50, or 8.567 million, or wherever. Because of this, all SSDs performance will vary over time and settle to some steady state value. Our SSD dynamically adjusts to the incoming workload to get the optimum performance for the workload. This takes time. Other lower performing SSDs take less time as they have less complicated systems. HDDs take no time at all because their systems are fixed logical to physical systems, so their performance is immediately deterministic for any workload IOMeter throws at them.

The Intel ® Performance MLC SSD is architected to provide the optimal user experience for client PC applications, however, the performance SSD will adapt and optimize the SSD?s data location tables to obtain the best performance for any specific workload. This is done to provide the ultimate in a user experience, however provides occasional challenges in obtaining consistent benchmark testing results when changing from one specific benchmark to another, or in benchmark tests not running with sufficient time to allow stabilization. If any benchmark is run for sufficient time, the benchmark scores will eventually approach a steady state value, however, the time to reach such a steady state is heavily dependant on the previous usage case. Specifically, highly random heavy write workloads or periodic hot spot heavy write workloads (which appear random to the SSD) will condition the SSD into a state which is uncharacteristic of a client PC usage, and require longer usages in characteristic workloads before adapting to provide the expected performance.

When following a benchmark test or IOMeter workload that has put the drive into this state which is uncharacteristic of client usage, it will take significant usage time under the new workload conditions for the drive to adapt to the new workload, and therefore provide inconsistent (and likely low) benchmark results for that and possibly subsequent tests, and can occasionally cause extremely long latencies. The old HDD concept of defragmentation applies but in new ways. Standard windows defragmentation tools will not work.

SSD devices are not aware of the files written within, but are rather only aware of the Logical Block Addresses (LBAs) which contain valid data. Once data is written to a Logical Block Address (LBA), the SSD must now treat that data as valid user content and never throw it away, even after the host ?deletes? the associated file. Today, there is no ATA protocol available to tell the SSDs that the LBAs from deleted files are no longer valid data. This fact, coupled with highly random write testing, leaves the drive in an extremely fragmented state which is optimized to provide the best performance possible for that random workload. Unfortunately, this state will not immediately result in characteristic user performance in client benchmarks such as PCMark Vantage, etc. without significant usage (writing) in typical client applications allowing the drive to adapt (defragment) back to a typical client usage condition.

In order to reset the state of the drive to a known state that will quickly adapt to new workloads for best performance, the SSD?s unused content needs to be defragmented. There are two methods which can accomplish this task.

One method is to use IOMeter to sequentially write content to the entire drive. This can be done by configuring IOMeter to perform a 1 second long sequential read test on the SSD drive with a blank NTFS partition installed on it. In this case, IOMeter will ?Prepare? the drive for the read test by first filling all of the available space sequentially with an IOBW.tst file, before running the 1 second long read test. This is the most ?user-like? method to accomplish the defragmentation process, as it fills all SSD LBAs with ?valid user data? and causes the drive to quickly adapt for a typical client user workload.

An alternative method (faster) is to use a tool to perform a SECURE ERASE command on the drive. This command will release all of the user LBA locations internally in the drive and result in all of the NAND locations being reset to an erased state. This is equivalent to resetting the drive to the factory shipped condition, and will provide the optimum performance.

Here is the source. You will note percentage wise the problem is just as bad as the PC Perspective article, and that the problem even occurs on the X-25E SLC SSD.

 

[GourdFreeMan]

Originally posted by: magreen
I do not know whose definition of "steady state" is correct, but on the issue of performance degradation, I believe taltamir is correct.

The term "steady state" comes from Intel in its characterization of the performance of its SSDs, as evinced in its reply to ssdworld I quoted above. It refers to how Intel's wear-leveling algorithms adapt as an Intel SSD becomes "full" internally, and the algorithm tries to adapt to past user IO behavior to mitigate performance issues. Intel isn't explicit in describing how this works; there could be some sort of internal defragmentation, or simply a change in the method by which sectors are filled with clusters of user data. However it works, it is actually a bit of a misnomer, as there are situations where a steady state would be impossible. For example, if I regularly did both video editing and compiling of a project with many small sources files that changed frequently, I would be in a situation where the drive would try to adapt to one IO pattern, when I switch to precisely the situation that adaption would hurt.

The "steady state problem" with SSD performance is a term coined on the Notebookreview forums to describe the performance degradation of Intel SSDs and possibly other drives after large amounts of data is written to them in small random writes. It would probably be better to call this the "internal fragmentation problem" or the "redirection fragmentation problem" when applying it to SSDs in general, but the name "steady state problem" has struck because of Intel's original response. To my understanding, it encompasses both the fragmentation of the internal LBA map, and the necessity to erase and rewrite more sectors than would be necessary on a HDD when the internal sector size is larger than file system cluster size after the drive has been "filled" internally. As long as the write-leveling algorithms in an SSD are sensible it should not significantly affect writes beyond simple fragmentation until every sector on drive has been written to at least once. That does not necessarily mean the problem has had its maximal performance impact after a single "filling" of the drive internally, however. It is a problem that does not go away with reformatting by the OS in current versions of Windows, but rather requires a special tool to blank the SSD internally.

Intel's response to ssdworld suggests the "steady state problem" should affect all SSDs, however the extent to which this is true has not been properly investigated in my opinion -- it may vary from SSD controller to SSD controller. Also coolVariable's assertion that Samsung drives are immune to this problem because they have been written to once before distribution, has no source other than a forum poster's bold assertion on Notebookreview, and a presumption based on a Chinese article that doesn't seem to have anything to do with the "steady state problem" at all. Take it with a grain of salt until exhaustive testing is done. In published articles Samsung's drives have been shown to be free of the issues plaguing JMicron-based drives, but I have yet to see an article doing testing like the PC Perspective article.

Footnote: When I write "full" or "filled internally" above I refer to the situation where, as far as the controller knows, every internal sector has data on it, but this does not necessarily mean the drive is full because SSD sectors can be larger than filesystem clusters. By an SSD sector I mean the minimal size block that the drive permits to be erased internally, which AFAIK is what the controller deals with internally when working with an LBA allocation table. When I use the term LBA, I refer to Logical Block Address of SSDs as refered to in Intel's response to ssdworld.

 

[taltamir]

magreen, that was beutifully eloquent... I think we have definitely had a lack of communication. Kudos on your research too.

btw, there is a program called windows steadystate: http://en.wikipedia.org/wiki/SteadyState

 

[magreen]

Cheers! :beer:

 

[faxon]

this is why im going to be sticking with platter drives until flash aware file systems become a reality, or some other equally powerful fix is implimented. until then, its Vraptors or SAS drives for me

 

[IntelUser2000]

So here's the impression I got reading lots of SSD reviews.

DON'T BENCHMARK NEEDLESSLY!!

Especially if you run things like IOMeter, which is an extremely stressful server-oriented benchmark program.

 

[Idontcare]

Originally posted by: IntelUser2000
So here's the impression I got reading lots of SSD reviews.

DON'T BENCHMARK NEEDLESSLY!!

Especially if you run things like IOMeter, which is an extremely stressful server-oriented benchmark program.

I'd be happy if a review consisted of nothing more than:

1. 1. SSD product name (so I know what to look for on newegg)
   2. Estimated availability date at web-tailers like newegg
   3. MSRP and estimated street price
   4. Warranty details
   5. CrystalDiskMark bench screenshot

Anything else we care about won't be captured by the tailored benches these guys are using anyways. Anything else the reviewers might show us is just boilerplate that won't actually assist us in downselecting to the drive of choice.

ATTO isn't going to show you something that HD Tach or HD Tune isn't going to show you when it comes to making the decision to purchase or avoid a given SSD.

You pretty much have to wait for a trusted forum colleague to report on their real-world feedback.