Fastest interface for solid state storage?

[Carson Dyle]

I'm a bit out of the loop when it comes to the latest and greatest developments in PC hardware. What's considered the best interface these days for solid state storage in a desktop PC?

 

[PliotronX]

NVME is the one and it is confusing because the physical form factor is known as m.2 but there are ordinary SATA SSDs available as m.2 so you have to be aware of that and your motherboard or adapters support one way or the other.

 

[Carson Dyle]

Ok, so I quickly dug this up in wikipedia. Yeah, I'm confused.

NVM Express devices exist both in the form of standard-sized PCI Express expansion cards[2] and as 2.5-inch form-factor devices that provide a four-lane PCI Express interface through the U.2 connector (formerly known as SFF-8639).[3][4] SATA Expressstorage devices and the M.2 specification for internally mounted computer expansion cards also support NVM Express as the logical device interface.[5][6]

So, an NVME device typically plugs into a PCIe slot? The m.2 interface is less desirable?

 

[Campy]

Simply put, NVME lets you access storage media over a PCIe bus. So NVMe is all PCIe, but there are different connectors and you don't always plug anything into a standard PCIe slot. Motherboard M.2 slots are typically wired for NVMe(PCIe).

M.2 is most common and it's great because you skip all the hassle with power and data cables, and it can support up to x4 PCIe speeds, roughly 3.5GB/s. Where you seem to be getting confused is that some M.2 form factor SSDs use the SATA interface rather than NVMe, so you need to check the SSD you're buying to see if it has the common SATA speeds 550MB/s -ish or 1000+MB/s NVMe speeds.

Also, it's up to the motherboard manufacturer whether or not to have SATA or NVMe support on their M.2 slots. For instance most Intel motherboards have two slots and both support NVMe, but one of them can be toggled to use SATA, in which case one of the regular SATA ports on your motherboard will be disabled and the SATA controller will instead be used for the M.2 SATA drive.

 

[Carson Dyle]

Thanks. That clears things up quite a bit.

 

[Wall Street]

I don't know if this was explained quite well enough yet. NVMe is a software/hardware protocol to allow SSDs to be connected to the system through the PCIe bus. There are basically three types of NVMe drives:

1. m.2 drives look like a new type of drive which is the shape of a stick of gum with one end of the gum stick slotting into a ~1 inch long socket on the motherboard. Most m.2 drives communicate through a PCIe 3.0 x4 interface. The physical m.2 slot usually also supports mini PCIe cards and this slot is used for wifi in some laptops and other small form factor computers. Be careful because the m.2 slot also supports SATA interface which is much slower, and in fact some m.2 slots only support SATA, so check that your motherboard supports NVMe over its m.2 port. As an example is the Samsung 970 pro.

2. The fastest but least practical option is full size PCIe cards. These can also connect over NVMe, which is basically the language for any storage device to communicate over PCIe. This is mostly not for home PCs and it usually used in servers. These devices can have much better cooling and draw more power than an m.2, but for desktop use the extra performance is very negligable. An example is the Intel 900p.

Here is an m.2 card plunged into a full PCIe card. This shows the size difference and the fact that a simple adapted shows that they both communicate through the same PCIe language.

3. There are also connectors which were designed for connecting higher speed NVMe drives in traditional 2.5 inch form factors. These are called u.2 and SATA Express. u.2 is the leftmost interface in the picture below while two SATA express connections are in the middle of the picture below and each SATA connection consists of the smaller connector plus the two SATA ports immediately to the right of that plug. I wouldn't recommend these because they are not widely adopted. Basically only Intel is making u.2 drives and pretty much nobody makes SATA Express drives despite the standard being a couple of years old already.

 

[BonzaiDuck]

And some minor clarification to Wall Street's splendid summary. There are constraints on bandwidth and PCIE lanes. For instance, my Z170 board and the Skylake processor offers 16 PCIE lanes managed by the processor.

There are another 22 PCIE lanes offered by the chipset, and supporting additional motherboard slots and devices/components. In a board with three PCIE slots which appear as full-length x16 slots, the third and lowest slot may allow only PCIE x4, depending on BIOS settings, and it competes with other resources like USB_34 the 19-pin USB 3.0 port of a motherboard, some of the x1 PCIE slots and so forth. If you configure the board to offer up PCIE x4 in this last slot, the four lanes belong to the chipset and communicate with the processor via DMI.

One might experience slightly lower benchtest results of an NVME connected to this third slot as I describe, because there are other resources and devices competing for the chipset PCIE lanes, and only four at a time can communicate with the processor.

So, then, one may have a decision about reserving the PCIE x16 #2 slot for a second graphics card (SLI or Xfire), or using it for an NVME adapter card. In this latter possibility, the number of lanes used by the main graphics card in PCIE x16 #1 would drop from 16 to 8 lanes, and the graphics card would suffer a 1% performance hit (which is nothing, really). If you want to take advantage of all 8 lanes in the #2 slot, you would need the motherboard to implement PCIE bifurcation, and you would purchase an adapter card which might configure up to four NVME cards.

Well . . . perhaps I got carried away, but I hope that's all useful information . . . .

 

[thecoolnessrune]

3. There are also connectors which were designed for connecting higher speed NVMe drives in traditional 2.5 inch form factors. These are called u.2 and SATA Express. u.2 is the leftmost interface in the picture below while two SATA express connections are in the middle of the picture below and each SATA connection consists of the smaller connector plus the two SATA ports immediately to the right of that plug. I wouldn't recommend these because they are not widely adopted. Basically only Intel is making u.2 drives and pretty much nobody makes SATA Express drives despite the standard being a couple of years old already.

U.2 on the drive end is SFF-8639, and is widely accepted in the commercial space, as, like Ethernet, it is simply a tighter spec revision of the same 2-port SAS connector that has been used for ages, first for 3Gb SAS, then 6 Gb SAS, then 12Gb SAS, and then eventually revising to accept 4x PCIe as well. The square-ish port you see on the motherboard is SFF-8643, again, the widely accepted port for internal 12Gb SAS, and NVMe Enterprise drives.

I agree on the Home use side it's rare to see this interface (because only Intel seems to be re-badging Enterprise drives as "Enthusiast" drives), but commercial drives with Power Loss Protection, Hot Swapping, and much less thermal throttling (because of the case acting as a heatsink), more likely than not come with this interface.

If for some reason you wanted to remotely locate your M.2 drive (better thermals, easier access, hot-swap, etc.) there are enclosures available to convert your M.2 drive to an SFF-8639 interface.

 

[paperfist]

I always thought m.2 slots blew all others out of the water?

 

[nosirrahx]

I always thought m.2 slots blew all others out of the water?

M.2 is just a form factor and can be SATA (SATA 600 specifically) or NVMe (which can be PCIe Gen 3 X 4 lanes although there are ways to make it run slower).

PCIe gen 3 X 4 lanes is the current "fast" interface and can be connected via M.2, U.2 or an actual PCIe slot.

Those PCIe lanes can be connected to the CPU through the chipset (almost all cases) or directly to the CPU. In theory a SSD would be faster (less latency) when directly connected to the CPU but in practice the difference would almost never be noticeable.

Intel's VROC technology allows you to connect multiple SSDs directly to the CPU and run them in RAID configuration bypassing the chipset and in theory should eliminate some of the typical 4K speed degradation you see in chipset RAID. Testing on this is very hard to find and in practice the only good use is to avoid limited chipset lanes to be shared between multiple high end GFX cards and a multi-SSD RAID setup.

 

[BonzaiDuck]

I always thought m.2 slots blew all others out of the water?

Well, my personal knowledge acquisition is limited by an approach toward getting things to work in the least amount of time. I had to come on board with NVME and M.2 soon after initial construction of my 2016 Skylake project.

This is what I discovered -- right away. Use of the M.2 slot on my motherboard would share bandwidth with the remaining SATA ports of the onboard Intel disk controller. The choice of other hardware options of the motherboard (i.e., just for instance making PCIE slot #3 an x4-capable slot or enabling the onboard USB3 19-pin connector) would deprive me of two out of six SATA ports. So I simply chose to use a PCIE x4 adapter card for my NVME M.2 drive. I now have two NVME's configured that way -- one using PCIE lanes of the chipset, and one using the lanes available from the CPU.

I didn't bother experimenting with the "shared bandwidth" situation.

 

[nosirrahx]

Use of the M.2 slot on my motherboard would share bandwidth with the remaining SATA ports of the onboard Intel disk controller.

What that means is that they share a path and the M.2 port will disable the 2 SATA ports on the same lanes. Intel's actual problem is the total PCIe lanes it has to go around which potentially could limit the speed of hardware if multiple high bandwidth parts are trying to use the PCIe lanes at the same time.

the X299 motherboards paired with the 44 lane CPUs 'should' never have an issue with this, not that Z370 boards would under normal consumer usage scenarios.