Discussion Optane Client product current and future

cbn · Apr 5, 2018

I do agree about the on package HBM and Pure Optane DIMMs being a logical progression, but that aside I am wondering about how different ways of stacking Optane could affect its performance tuning?

If additional TSVs are involved to increase parallelism per Optane die I'm thinking a DIMM slot would no longer provide enough bandwidth for even a modest amount of Optane.

If true, then we would be looking at moving the on-package HBM onto the processor die to make room for on-package (High TSV) Optane.

IntelUser2000 · Apr 5, 2018

cbn said:
If additional TSVs are involved to increase parallelism per Optane die I'm thinking a DIMM slot would no longer provide enough bandwidth for even a modest amount of Optane.

For system memory, it won't matter, because its basically all about QD1 performance.

Let's look at latencies first:
Optane SSD: 10us rated
DRAM: 60ns for Skylake
And let's say Optane DIMM: 500ns

You can translate that into QD1 4KB IOPS. The equation is Queue Depth / latency. 1/10us = 100K

So 10us means 100K IOPS using the QD1 4KB block size metric*. That means,

DRAM: 17 million IOPS
Optane DIMM: 2 million IOPS

You can also translate that into bandwidth. The equation is IOPS x block size.

Optane SSD: 400MB/s
DRAM: 64GB/s
Optane DIMM: 8GB/s

A fast NAND NVMe drive can achieve 2GB/s sequential transfers, but only does 13K IOPS 4KB QD1. That's 77us, or only 52MB/s.

Now, you see why parallelism doesn't matter. Optane DIMMs can't reach DRAM bandwidth because the media latency is too high. The 2GB/s bandwidth for a NAND NVMe drive has nothing to do with 2GB/s in DRAM. You'll be able to make a passable PC with PC133 SDRAM, but not with 2GB/s NAND NVMe drive.

*This isn't entirely accurate as for superfast systems like Optane DIMM and DRAM, 4KB block sizes slow it down. Instead, they transfer data in 64Byte chunks, which is the size of a cacheline in CPUs. It would be more accurate to say DRAM achieves 1.1 billion IOPS @ 64Byte.

cbn · Apr 6, 2018

IntelUser2000 said:
Now, you see why parallelism doesn't matter. Optane DIMMs can't reach DRAM bandwidth because the media latency is too high. The 2GB/s bandwidth for a NAND NVMe drive has nothing to do with 2GB/s in DRAM. You'll be able to make a passable PC with PC133 SDRAM, but not with 2GB/s NAND NVMe drive.

Below is a patent from Micron:

http://appft.uspto.gov/netacgi/nph-Parser?Sect1=PTO1&Sect2=HITOFF&d=PG01&p=1&u=/netahtml/PTO/srchnum.html&r=1&f=G&l=50&s1="20160276022".PGNR.&OS=DN/20160276022&RS=DN/20160276022

Some embodiments include architectures in which two or more memory array decks are vertically stacked. One or more of the stacked decks is configured to have different operational characteristics relative to others of the stacked decks. For instance, one or more of the decks may be configured to have rapid access times suitable for utilization in XIP (execute in place) applications and/or dynamic random access memory (DRAM) emulation applications, and one or more others of the decks may be configured to have stabile, possibly slower access, storage suitable for utilization in long-term storage applications. Further, one or more of the decks may be configured to have more endurance than others of the decks. For instance, one or more of the decks may be suitable for a lifetime of approximately 100,000 cycles, whereas one or more others of the decks may be suitable for about 1,000,000 cycles (in other words, at least one of the decks may have a durability of at least about 10-fold more cycling times than another of the decks). The difference between the endurance of the decks may result from structural differences between the decks. For instance, a deck with higher endurance may have reduced thermal disturb and/or other memory-loss mechanisms as compared to a deck with less endurance. However, the deck with less endurance may have other advantages (for instance, faster access times, etc.) as compared to the deck with higher endurance. Accordingly, each memory array deck may be tailored for applicability relative to specific memory functions.

According to that reduced latency (access time) can be achieved by sacrificing endurance of the 3DXpoint layers.

Endurance can be regained by having slower writes. (I am sure there are some exceptions though)

Increasing parallelism regains the speed lost by the slower writes.

IntelUser2000 · Apr 6, 2018

cbn said:
According to that reduced latency (access time) can be achieved by sacrificing endurance of the 3DXpoint layers.

They may or may not do this. Patents are very unreliable sources of information on future products though. Original 3D XPoint roadmap indicated we'll see 2nd generation of it sooner than later.

One site is listing April 17 as the availability date for Optane M10. Can't find much info on it other than the name, and capacity. I think there's a good chance M10 will be Optane Memory for mobile.

Japanese site listings are also showing what the + designations for Core chips really mean. The Core i5+ and i7+ chips have Optane Memory in the box. PC Canada has early listings too.

cbn · Apr 8, 2018

VirtualLarry said:
Intel, IMHO, has Optane positioned exatly wrong. It's the lower-end systems, budget ones that ship with HDDs that need Optane caching. Higher-end rigs will just ship with a larger primary SSD, thus making Optane virtually useless in its current market positioning.

Comparing specs of Coffee Lake Celeron and Coffee Lake Pentium the gap between Celeron G4920 and Pentium G5400 seems too large:

So I am thinking Intel could easily have plans for 20th Anniversary Celeron by using that large gap.

Maybe a more memory focused Celeron with 3MB L3 cache and Optane memory support to "sorta kinda" go along with the theme of 300A (released 1998)? This with SMT added to the 3.2 GHz (or better) clock frequency.

(This working on B360 or better board)

cbn · Apr 10, 2018

Based on the following information on how the Intel memory caching works (link below originally provided by IntelUser2000 here):

http://surprizingfacts.com/what-is-intel-optane-part-1-optane-memory-blog-of-intel-surprizingfacts/

Secondly, while the RST system is running, the driver will continuously generate caching. And here there is one important difference between Optane Memory modules of different capacities: on a 16GB device, only block level caching is supported, on a 32GB device, block level caching and file leveling (both work simultaneously). In the case of block caching, the decision to cache a block occurs instantly at the time of the I / O request. In the case of file caching, the driver monitors the frequency of access to files and puts it all in a special table, which then (at the time of system downtime or according to the user's schedule) is used to determine which files are left in the cache, which are deleted and which are added.

Both types of caching use rather clever algorithms for caching decision making – I can not describe them deeply, but for general understanding I note that, for example, video files are not cached (yes, the driver looks at File extension), the size of the file is taken into account, the type of load is determined – caching preferences are given to random access rather than sequential, which makes sense due to extremely slow operation of hard disks in random access operations, etc. On the Internet, I met some negative comments on the topic that "the cache will be immediately overwritten with data," "16GB capacity is not enough for anything" and the like – usually reviews from people who have never tested Optane Memory. I have not yet heard negative feedback about the performance of this solution from any of our partners I work with.

.....We know the Optane memory optimizes for small size files and random access.

So when Intel moves the Optane from NVMe to using a DDR interface (and 4K read speed jumps from 300 MB/s to 500 MB/s to 8GB/s) How fast does the Sequential Read need to be on a NAND based SSD (or hard drive array) in order for it to not be a bottleneck? (Average use case scenario)

nosirrahx · Apr 10, 2018

cbn said:
.....We know the Optane memory optimizes for small size files and random access.

So when Intel moves the Optane from NVMe to using a DDR interface (and 4K read speed jumps from 300 MB/s to 500 MB/s to 8GB/s) How fast does the Sequential Read need to be on a NAND based SSD (or hard drive array) in order for it to not be a bottleneck? (Average use case scenario)

The faster Optane cache gets the more harsh the switch to non-cached data will be.

I think we will eventually see a delineation between Optane for HDD and Optane for SSD.

They already do this BTW but it is undocumented. The 900P is the perfect Optane cache for a large SATA SSD since it caches far more and is far faster than the 16/32GB offerings targeted towards HDDs.

For reasons I can't understand the best enthusiast option for fast + huge all SSD storage is not even being advertised by Intel.

PCIe gen 4 will take care of the sequential issue effectively doubling it. For 4KQ1T1 you don't actually need to make huge leaps to feel it as the files you are reading are already so small that past a certain point the CPU requesting reads starts to get in the way. This is why many games see little improvement in load times when switching from a SATA SSD to NVMe or even Optane. The read access is already not the bottleneck for those games.

cbn · Apr 17, 2018

IntelUser2000 said:
For system memory, it won't matter, because its basically all about QD1 performance.

Let's look at latencies first:
Optane SSD: 10us rated
DRAM: 60ns for Skylake
And let's say Optane DIMM: 500ns

You can translate that into QD1 4KB IOPS. The equation is Queue Depth / latency. 1/10us = 100K

So 10us means 100K IOPS using the QD1 4KB block size metric*. That means,

DRAM: 17 million IOPS
Optane DIMM: 2 million IOPS

You can also translate that into bandwidth. The equation is IOPS x block size.

Optane SSD: 400MB/s
DRAM: 64GB/s
Optane DIMM: 8GB/s

A fast NAND NVMe drive can achieve 2GB/s sequential transfers, but only does 13K IOPS 4KB QD1. That's 77us, or only 52MB/s.

^^^^ Thanks very much for that formula. (That is very useful information)

P.S. I noticed that 4K QD1 write is higher on the NVMe NAND based SSDs compared to SATA SSDs:

https://www.anandtech.com/show/12512/the-intel-optane-ssd-800p-review/5

https://www.anandtech.com/show/12348/the-samsung-860-pro-512gb-and-4tb-ssd-review/5

So does there happen to be a formula we could use for predicting 4K QD1 writes for Optane on a DDR interface?

IntelUser2000 · Apr 18, 2018

cbn said:
So does there happen to be a formula we could use for predicting 4K QD1 writes for Optane on a DDR interface?

No, that's not possible. Many factors determine performance, and only the manufacturer will know it by testing. The formula I put above applies to write too though, assuming you know a number.

cbn · Apr 19, 2018

cbn · Apr 19, 2018

Here is a motherboard I didn't know existed (credit to Virtual Larry for finding this and posting in another thread):

https://www.msi.com/Motherboard/B250M-PRO-OPT-BOOST.html

Dayman1225 · Apr 28, 2018

Intel Optane 905P Series, 960GB

M.2 Version exists according to Asrock.

and a new U.2 verison

Declaration of Conformity for 905P

Looks like a "new gen" of Optane is around the corner

IntelUser2000 · Apr 28, 2018

Looks like the 905P is Mansion Beach Refresh. Interesting. Idle power is 2W higher at 7W according to Newegg.

The Core i + branding isn't just for Optane Memory. It can be systems that feature ANY Optane device.

Dayman1225 · Apr 28, 2018

Non render image of the 905P along side the Ruler Form Factor P4500 series.

Source Here

nosirrahx · Apr 28, 2018

905P will likely be my OS drive on my new workstation. I am due to start building in about a month or so.

I wonder if this is just a capacity refresh or if we get more speed.

IntelUser2000 · Apr 28, 2018

Yea, I've seen the M.2 version too. Your link doesn't work BTW.

It just looks like a refresh that brings more form factors and greater capacities. Up to 1.5TB.

Dayman1225 · Apr 28, 2018

Full original Pic for Asrock M.2 905P is from Chiphell AFAIK. Suggests PCIe Gen 3x4? so Carson Beach? I am probably reading into it to much though lol.

Dayman1225 · Apr 28, 2018

IntelUser2000 said:
Your link doesn't work BTW.

That's odd... Working fine for me, even if I click the hyperlink from here.

IntelUser2000 · Apr 28, 2018

It works now.

M.2 905P looks like Mansion Beach Refresh on a M.2 slot. That's why its x4. Carson Beach, if its coming, isn't here yet. Carson Beach is a successor to Brighton Beach, which is 800P and likely has power saving features just like its predecessor.

Starting with 16.0.2.1086 Optane RST driver, secondary drives are supported for acceleration: https://www.intel.com/content/www/us/en/support/articles/000027987/memory-and-storage.html

You can just choose which drive to accelerate before enabling it.

IntelUser2000 · Apr 28, 2018

M10 Optane Memory 16GB to be available in 2242(42mm length) form factor.

cbn · Apr 28, 2018

Regarding the cache method differences between 16GB (block level) and 32GB Optane Memory (File level and Block level) listed in post #106, I have been wondering specifically how paging out (page file) would differ?

Specifically does the Optane Software handling the 32GB load the page file onto the Optane? If so, I am thinking there could be substantial difference between the two for a system used for browsing if the primary drive (being cached) is HDD.

See one problem I can imagine with the 16GB during page out (if it goes to the Optane) is that some data would need to be evicted (to make room for the page out) thereby making the primary drive write speed (not the Optane write speed) the bottleneck.

cbn · Apr 29, 2018

IntelUser2000 said:
M10 Optane Memory 16GB to be available in 2242(42mm length) form factor.

I can imagine we will see this in Chromebooks then.

Maybe even ARM chromebooks too.

(Speaking of ARM, I do think it will be very interesting to see how NVMe Optane would compare to NVMe Z-NAND/SLC NAND for paging out during Browsing.....not only for Chrome but for Firefox Browser Builds as well.)

Firefox Browser Build = Any Linux distro used primarily for (Firefox) browsing, but can also use (and load additional) local apps as well. (See below for example)

IntelUser2000 · Apr 30, 2018

It's more likely we'll see the 2242 M10 on laptops and some ultrabooks.

Most chromebooks probably don't even support NVMe. They often skip SATA and use eMMC, probably because its cheaper. Geminilake based Chromebooks may support it, but the question is how many manufacturers will go for it, if at all for just 16GB storage? There's also the question of how low power the M10s are. The 800P review on TH shows its acceptable, but nowhere near Chromebook level. The top level NVMe drives are behind SATA drives in power consumption, and 800P is middling in the field of NVMe drives.

Specifically does the Optane Software handling the 32GB load the page file onto the Optane? If so, I am thinking there could be substantial difference between the two for a system used for browsing if the primary drive (being cached) is HDD.

There probably isn't a big difference in browsing. I'm using the 16GB so I can vouch for that. What 32GB is good for is scenarios where data swapping can occur because there's lot of activity going on. The 16GB doesn't know the difference so it just keeps the most recent data. The 32GB can optimize over time. I can tell you on my system when I need to reinstall drivers or reboot the OS after being on for many days it slows down for some time.

cbn · Apr 30, 2018

IntelUser2000 said:
Most chromebooks probably don't even support NVMe. They often skip SATA and use eMMC, probably because its cheaper. Geminilake based Chromebooks may support it, but the question is how many manufacturers will go for it, if at all for just 16GB storage?

Yes, the only ChromeOS device I know of that has NVMe storage is the Core i7 model of this Chromebook. (The 128GB and 256GB models of the same notebook (both Core i5) have eMMC for the "SSD" ).

I guess it depends on how much the Android apps mentioned in this article catch on, but I do know that there are many Chrome books with 16GB storage.

And maybe for some people having a faster, but smaller capacity storage is more desirable? (re: faster booting and can use the Optane NVMe as a memory extender). With that noted, I do wonder how much power the 16GB Optane NVMe uses while paging out during browsing? This compared to the power consumption of other hardware configurations.

P.S. With Ubuntu (or some other Linux like it) being a very small install like ChromeOS (Example: Full install of Ubuntu 18.04 LTS only takes up 5.8GB + swap partition) it would be great to see it (or another Linux like Ubuntu MATE or Linux Mint MATE) offered as an alternative to ChromeOS. Not sure why this hasn't been happening? Battery life holding things back? (Hopefully since the article below was written back in Oct. 2016 things have changed? Would like to see Ubuntu (and other Linux dsitros) make a better showing against ChromeOS. (The various Linux distributions have a lot of positive benefits (including privacy) that ChromeOS doesn't have.)

https://www.datamation.com/open-source/ubuntu-vs-chromeos-work-flows.html

Chromebooks have outstanding battery life. I've seen them running 8 to 10 hours without missing a beat. Obviously watching Netflix or the like can significantly reduce those numbers. But regardless, the batteries included with Chromebooks seem to blend well with ChromeOS itself.

Ubuntu (Unity edition) has horrible battery life on even the most bleeding edge, highest capacity batteries.Why power management isn't a priority out of the box always amazed me. Thankfully some Ubuntu spins like Ubuntu MATE offer decent power management out of the box. This is done with a tool called TLP. Using TLP, Ubuntu (or any distro) can detect when you're connected to power or running on your notebook's battery. This feature significantly increases your available battery life under Ubuntu. Going even deeper, you can also make additional changes (and save them) using PowerTOP.

The best choice is: If you're needing a simple web browser on a laptop with great battery life, ChromeOS wins on the Chromebook. Ubuntu with TLP installed is good, but the ChromeOS compatibility with the Chromebook is better. Whether or not you choose to paint this as a hardware vs software issue is up to you.

P.S. Forgot to mention I do own a 16GB Optane now and have using with Linux on the desktop. I've been impressed and I keep on wondering if perhaps its usage as RAM Extender for browsing is one of its best kept secrets.

Brahmzy · May 1, 2018

Whoa! What’s this 905p all about!?!
I JUST freakin bought 3 900ps - you’ve gotta be kidding me.
Release ETA??

Discussion Optane Client product current and future

Lifer

Elite Member

Lifer

Elite Member

Lifer

Lifer

Senior member

Lifer

Elite Member

Lifer

Lifer

Golden Member

Elite Member

Golden Member

Senior member

Elite Member

Golden Member

Golden Member

Elite Member

Elite Member

Lifer

Lifer

Elite Member

Lifer

Senior member