Optane SSDs available at Newegg - $2 per GB!

[Hail The Brain Slug]

I'm wondering if AMD and Intel are planning on release CPUs and chipsets with higher amounts of PCIe lanes so consumer level hardware can use 8x PCIe NVMe SSDs. PCIe v5 x8 should make for a really fast SSD for 4K video editing/rendering and tasks that need really high performance storage?

PCI-e 5.0 is going to quadruple bandwidth per lane, so a 4x link that currently maxes out at 4 GB/s theoretical will have ~16GB/s bandwidth. Do you somehow think that's not enough? I think we are far away from consumer/prosumer grade SSD's needing an 8x link for 32 GB/s bandwidth, let alone a 4x link at 16 GB/s.

AFAIK we don't even have any 3.0 16x enterprise SSD's that will saturate the bus.

 

[BonzaiDuck]

PCI-e 5.0 is going to quadruple bandwidth per lane, so a 4x link that currently maxes out at 4 GB/s theoretical will have ~16GB/s bandwidth. Do you somehow think that's not enough? I think we are far away from consumer/prosumer grade SSD's needing an 8x link for 32 GB/s bandwidth, let alone a 4x link at 16 GB/s.

AFAIK we don't even have any 3.0 16x enterprise SSD's that will saturate the bus.

Over at "Operating Systems," I posted a thread seeking more information about a "Windows Experience Index" from Windows 10. I found a freeware replacement for what it been the WEI in Windows 7. My scores drooped for the CPU at about 8+, but the rest of them were 9+. Usually, you find the "bottom line" overall score defined by the lowest component score to be identical to the "storage" result.

That one is the highest, and after cleaning up and optimizing my disks and closing background software, the CPU score was shown about 9.0 with a second try. But I can only attribute the storage result to my caching configuration. Yeah -- just ran it again -- storage remained at 9.2, CPU showed 9.2 -- everything else was 9.2 except for graphics, which was 9.9

Why do I mention that? Well, I have the second-tier GTX-1070 in PCIE x16 slot #1. Ordinarily this would be using all 16 lanes. But I had put a 250GB caching NVME in the x16 slot #2 position, so the graphics card only has 8 lanes.

Sadly, the NVME in that #2 slot is only using 4 of its 8 lanes. To utilize a card fitting two NVME M.2 drives in that slot, the board has to be capable of PCIE bifurcation, and some Z170 boards feature it -- but not mine. And ASUS has no plans to revise the BIOS for inclusion of that feature.

I'm coming away from this thread with an impression that Optane, even in its most forward-looking usage with servers, utilizes some sort of "pairing" and therefore what might come under the heading "caching." And I remember back in 2011 with the Z68 systems, when people were skeptical about ISRT. I migrated to the hardware-agnostic PrimoCache (software) option, and there has been more skepticism. Certainly, we want to hear the criticisms, and I can't simply dismiss what I've heard -- they're mostly valid.

But I can't imagine when or how soon my computer usage is going to saturate the potential of this machine. My hardware and software configuration on this Skylake had fully matured by last summer, and I'm still sitting down to this machine feeling a sort of triumph and satisfaction, as though someone had just given me a Nissan GT-R Nismo ($175,000) for my birthday. "Let's take it out on a country road for a spin," and it has way more unused guts left -- for what? Running LinX at full-bore, mostly!

 

[IntelUser2000]

No consumer storage system needs above 2GB/s bandwidth. DRAM requires high bandwidth because it uses the bandwidth for compute.

No practical storage system needs above 500MB/s. The only reason you need above that is for transferring files. PCIe x4 for drives are about as useful as having 8 cores on smartphones. Not workstations, not high end PCs, Smartphones.

 

[BonzaiDuck]

No consumer storage system needs above 2GB/s bandwidth. DRAM requires high bandwidth because it uses the bandwidth for compute.

No practical storage system needs above 500MB/s. The only reason you need above that is for transferring files. PCIe x4 for drives are about as useful as having 8 cores on smartphones. Not workstations, not high end PCs, Smartphones.

No disagreement there. But don't call me foolish -- I knew that in that in the first place. I just wanted to see what I could build. And I built it.

 

[cbn]

Perhaps, then, we'd have a decade to prepare ourselves. Basically, it looks as though you're saying that the Intel DIMMs move a caching arrangement higher in the pyramid, and that the combination would make up for any speed shortcomings of the 3D XPoint DIMMs.

But I'm still trying to wrap my brain around the persistence aspect. With that type of capacity -- density -- you wouldn't need any slower storage devices except for uploading OS, programs and data into the XPoint DIMMs?

With NVDIMM-P (which uses 3DXPoint or NAND in addition to DRAM) the OS would just be on the non-voltaile part of DIMM. The same would be true for Workstations using Apache Pass DIMMs (ie, pure Optane DIMMs) in addition to regular DIMMs (ie, dram only DIMMs). Hard drives and SSDs would only be required if additional capacity were needed.

 

[cbn]

The 118GB is now available on Newegg.

P.S. It looks like this model of SSD is not available at too many retailers. I wonder if it is going to be a limited edition type release?

 

[BonzaiDuck]

The 118GB is now available on Newegg.

P.S. It looks like this model of SSD is not available at too many retailers. I wonder if it is going to be a limited edition type release?

Is the 800p in any way crippled for being limited to two PCIE lanes or x2?

 

[cbn]

Is the 800p in any way crippled for being limited to two PCIE lanes or x2?

It is actually the controller that is holding things back to levels even below PCIe x 2.

For example, both the 58GB and 118GB 800p have only 100 MB/s more Sequential read compared to the 32GB optane (1450 MB/s vs. 1350 MB/s).

The Sequential writes (limited by the controller) are capped around ~640 MB/s for both 58GB and 118GB 800p.

 

[cbn]

Yesterday, after debating on 32GB Optane vs. 58GB Optane (for a Windows "Browser build" boot drive) I decided to go with 58GB 800p.

Looking forward to finally trying out 3DXPoint. (Drive should probably be here early next week)

 

[cbn]

My Crystaldiskmark results of 58GB 800p on MSI A68HI AC motherboard (OS: Windows 10 Pro):

Not sure why the 4K read is low?

 

[IntelUser2000]

My Crystaldiskmark results of 58GB 800p on MSI A68HI AC motherboard (OS: Windows 10 Pro):

Not sure why the 4K read is low?

https://www.tweaktown.com/articles/8073/amd-ryzen-ssd-storage-performance-preview/index.html

AMD systems get lower storage performance than Intel. Now that's with Ryzen and FM2 may get lower. With Optane the results may be further exaggerated. You may try playing around with Balanced/High Performance settings to see if it changes.

Just for fun I ran the benchmark on my system. This is with 2TB Seagate being cached. Looks like you are capped at 175MB/s for some reason. 140MB/s QD1 Read is still insanely good. That's 4x best NAND SSDs.

 

[cbn]

Despite the low 4K score web browsing with heavy paging out is really impressing me.

My current Task Manager Screen shot:

(Opening new pages or clicking on the first pages I opened is very fast)

In fact, I am thinking (at this point) I could get by with 2GB RAM DDR3 1600 if all I wanted to do is browse the web with this machine. (Will do a test with this later on)

 

[cbn]

2GB RAM + 58GB 800p working great for browsing.

 

[IntelUser2000]

2GB RAM + 58GB 800p working great for browsing.

That's, awesome.

So if you have higher amounts of RAM, Windows uses RAM as sort of a cache for your applications. You should definitely feel a performance difference if paging and if the drive is too slow.

I feel like moving the 16GB one to my other system and getting the 58GB 800P. Ahh, why did they have to price it so high? The 32GB Optane Memory is only $50! $50 Canadian dollars I tell you!

 

[Alpha One Seven]

If I hadn't just added 2 x 2TB PCIE SSD's I would have been interested in this sale.
It's always AFTER I make a purchase. lol

 

[BonzaiDuck]

OK. I'm curious.

First, how is cbn using his 800p? As a system boot disk? Or -- what motherboard features is he using? I think he's just using the 58GB 3DXPoint as a boot disk, and I get that notion from his CrystalDiskMark screenies.

Second, IntelUser2000 seems to be using the Intel 800p to cache a 2TB drive, but his CrystalDiskMark screenie suggests to me he's only accessing 1TB or 979GB of the Seagate HDD.

Is this ISRT caching? Or is it a newer feature on his motherboard?

I'll go ahead and post some screenies of my own, explaining why I'm not salivating over the 800P drives like Alpha One Seven. I've explained what software I'm using to cache drives on my sig Skylake system, which I was using for a few years before I'd even heard of 3DXPoint.

Here are the benchies (Magician, Anvil Benchmark, CrystalDiskMark) for my boot-system NVME 960 Pro [drive C:], which is cached to more than 8GB of RAM, enabling deferred writes to improve the write benchtests. The drive is installed in my PCIE x16 #3 slot which only offers x4 lanes, and therefore uses four of the twenty lanes provided by the chipset:

Here are the same three benchtests on a Crucial MX300 SATA SSD cached to a 250GB 960 EVO with 230GB caching volume, and then cached in a two-tier arrangement to another 8GB of RAM. The RAM allocations are separate for the two drive configurations, and the EVO is installed in PCIE x16 #2 -- robbing my graphics card of 8 lanes out of the 16 provided by the CPU:

Now, I could add two more screenies which show the real-time information from the caching-program's GUI, but I'll spare the readers and simply provide one of them. We're interested in the hit-rate on the cache after it's been running for several hours of computer use, and the hit-rate between the two drive configurations is similar, or in the range between 80 and 100%:

The hit-rate for the boot NVME is 94%, if you can't read the faint green print at bottom-right. The hit-rate for the SATA SSD configuration is 83%, and has been seen to climb to 100%.

The hit-rates would inform any ideas about how these benchtest scores would appear in real-world usage. And it's important to note that usage is a personal profile, or something that will vary over time as well as over different users.

With the RAM -caching, I have no need to worry about the better 4K performance I might see with the 3DXPoint 800P device.

Of course, I could say that I spent $130 for the EVO drive and another $180 for an additional RAM kit matching the original. The PrimoCache license and download is $30 for one computer. I had entertained getting 32GB RAM from the start when I built the system, but chose the 16GB kit initially. I was getting similar benchtests with the original 16GB kit, but I wanted bigger caches and just "wanted to see for myself."

You can make critical remarks about the expense -- the extra RAM, for instance -- but those were the choices I had, and my choices were driven just by curiosity.

I'd be interested in Alpha One Seven's usage and benchmarks for his two PCIE SSDs. But are they in RAID? Are they AHCI drives? Or are they NVME?

 

[nosirrahx]

The main drawbacks to RAM cache (which I use BTW) is that it is easily exhaustible, does not persist across reboots and does not aid in booting.

Its great on a high memory system that has been up for a while especially if only a few apps care constantly being used.

Its kind of terrible where multiple huge apps are rotated as it is impossible to cache everything you also have reduced working RAM to work with.

In the last build I stacked Optane with RAM cache so when RAM cache is exhausted I fall back to Optane speed, not SATA speed.

The following bechmarks are from RAM cache on top of Optane and Optane on its own.

 

[IntelUser2000]

Second, IntelUser2000 seems to be using the Intel 800p to cache a 2TB drive, but his CrystalDiskMark screenie suggests to me he's only accessing 1TB or 979GB of the Seagate HDD.

Is this ISRT caching? Or is it a newer feature on his motherboard?

No, I have the 2TB split into two partitions using diskmgmt. I ran the CrystalDiskMark just to be sure, and it benches the same. I didn't really give much thought into it when I did this. Optane Memory still counts it the same way.

There's a difference between ISRT and Optane Memory. The latter only works with Optane Memory, and has some Optane specific optimizations.

 

[BonzaiDuck]

No, I have the 2TB split into two partitions using diskmgmt. I ran the CrystalDiskMark just to be sure, and it benches the same. I didn't really give much thought into it when I did this. Optane Memory still counts it the same way.

There's a difference between ISRT and Optane Memory. The latter only works with Optane Memory, and has some Optane specific optimizations.

Second paragraph: "Optane Memory" is a motherboard feature for caching slower storage devices running on SATA, etc.?? Short comments are good -- I need to "catch up." Could you point me to (for example) an ASUS mid- to flagship-range board -- probably Coffee Lake or Kaby Lake -- which implements the feature? Or - correct me.

Yes -- assume I know that you would know if there were something wrong with your Seagate. You could have two different volumes -- as would I. But in my configuration, either volume was only accessible to one of two operating systems -- at least because they are not assigned drive letters in the other OS. I did the same thing with the Seagate Barracuda [128MB cache] 2.5" drive that I used for six months in the caching configuration. I would not have replaced it with the Crucial -- or I would not have bought the Crucial so soon -- if I didn't suspect a problem with the 128MB cache in the set of possible sources -- independent or acting together. About once every six weeks, the Macrium backup would fail and the drive would need a quick CHKDSK repair sequence. Techno-lust got the best of me, because I might have sorted it out. [Consider all the trouble with dual-boot I may have had with Creator Builds.]

I don't think the disk-error problem I had with the drive was for it being faulty, but that's possible. I also don't think I had any "offline writes" occurring to the wrong volume, but that could also still be possible, despite the cautions and pains taken.

Also, to be honest, I suspect I had a recurring hiccup that occurred once a month just around the time of Windows Updates. The system would blue-screen with an orderly shutdown and a mini-dump file. That problem disappeared after updating my Primo to version 3.0. I think the updates would crash the cache, and now it doesn't.

But it still might have been a driver conflict or bug.

I remember the occasional hiccup with ISRT when I used it on a Z68 system. Howsoever, "long is the road, and hard is the way" -- but I have no more hiccups.

 

[BonzaiDuck]

The main drawbacks to RAM cache (which I use BTW) is that it is easily exhaustible, does not persist across reboots and does not aid in booting.

Its great on a high memory system that has been up for a while especially if only a few apps care constantly being used.

Its kind of terrible where multiple huge apps are rotated as it is impossible to cache everything you also have reduced working RAM to work with.

In the last build I stacked Optane with RAM cache so when RAM cache is exhausted I fall back to Optane speed, not SATA speed.

The following bechmarks are from RAM cache on top of Optane and Optane on its own.

And so Intel provides the software/firmware.

Intel would include a means of enabling deferred-writes, which they had done with ISRT. But it's possible that Intel might not have included the "Prefetch Last Cache" option with a "Start at Windows Boot" feature. The cache will be repopulated from persistent storage after an normal shutdown or restart.

I say that because of your observation: " . . . does not persist across reboots and does not aid in booting.. . . "

From this end, I only see the disadvantage of "doesn't aid in booting," because the system has to reload previous cache contents. Truth is -- I haven't yet figured out for sure how this is done, since I've yet to discover any files on disk created by Primo that would function in that way -- like pagefile-sys or hyberfil.sys. I'm wondering if the cache doesn't get saved in pagefile.sys, but it's unlikely since the system-managed pagefiles total about 10GB on my system.

 

[nosirrahx]

From this end, I only see the disadvantage of "doesn't aid in booting," because the system has to reload previous cache contents. Truth is -- I haven't yet figured out for sure how this is done, since I've yet to discover any files on disk created by Primo that would function in that way -- like pagefile-sys or hyberfil.sys. I'm wondering if the cache doesn't get saved in pagefile.sys, but it's unlikely since the system-managed pagefiles total about 10GB on my system.

If I am understanding the documentation correctly the RAM cache begins filling as soon as the driver loads so that by the time your desktop loads a chunk of cached content from the previous session is already set to load from RAM.

 

[BonzaiDuck]

If I am understanding the documentation correctly the RAM cache begins filling as soon as the driver loads so that by the time your desktop loads a chunk of cached content from the previous session is already set to load from RAM.

A nominal plus-positive, then, for the Intel Optane Memory feature. I can see I need to update my CrystalDiskMark version.

But from the numbers that are actually comparable, we're in the same league, so to speak . . .

Primo on my system will "release the L1 [RAM] cache on hibernation." If the checkbox isn't selected, the system won't go into hibernation -- or -- it attempts to do so, and then comes up running within 30 seconds.

With the Intel configuration, can you pause and resume the caches?

 

[nosirrahx]

With the Intel configuration, can you pause and resume the caches?

The Optane cache software is absolutely pathetic when it comes to options. You can turn it on and you can turn it off, that is it.

They clearly tried to make it as idiot proof as humanly possible but they went way overboard IMO.

I actually would not have a problem with idiot proof mode if there was an advanced mode option.

This has come up on the Intel forums and they claim that in the future Optane software might have more options.

 

[nosirrahx]

If I am understanding the documentation correctly the RAM cache begins filling as soon as the driver loads so that by the time your desktop loads a chunk of cached content from the previous session is already set to load from RAM.

BTW, in this post I am talking about Primo cache.

 

[BonzaiDuck]

BTW, in this post I am talking about Primo cache.

So . . . you're experimenting with Primo Cache?

Then, what are those "special features" IntelUser2000 cites?

I get hammered a lot for my enthusiasm about the software from Shanghai. "Bad ROI on the RAM," "extra clock cycles," etc. And I remember when ISRT came out, and I was posting my experiences with it -- I built my first Sandy Bridge system to include ISRT of a 64GB SSD and a hard disk. The word was described this way: "It's just a stopgap measure anticipating larger SSDs." So there was resistance to caching strategies all along. It "adds complexity." "Windows has its own cache" -- so many criticisms. Lately, XavierMace has leveled stern remarks: to paraphrase, "Nobody needs a system with more than 500MB/s sequential throughput."

At this point, looking back on the past year with this system, Primo has been more reliable and versatile than ISRT ever was. I was going through a few bumps in trying to make SSD-caches work for both Win 7 and Win10 OSes without conflict. And when I screwed things up in my experiments, nothing was damaged, nothing lost. There were some orphaned VSS keys, and at one time I had to remove an NVME, change the configuration, reinsert it and recreate the caching volume.

Mainstreamers would probably be overwhelmed. But I got my MOJO workin' now!!

I could always fret and worry that China's PLA Cyber-War Division has hooks in my system: Romex is HQ'd in Shanghai. But their support uses English like the Indians do -- better than the English. They have a useful and well-maintained forum. Ah! Double-jeopardy -- I'm using Kaspersky for my AV . . . . But there are three myths at play: The Russians and Kaspersky myth, another myth about China, and the "free-market." Companies would not incline to lose market-share with some . . . software scandal . . . . If only SuperSpeed SuperCache offered Primo's features . . . .

But I'm good. It seems so, anyway . . . .