Anyone have advice or ideas about the best way to connect 2 Intel Optane 905p drives, as individual drives (not RAID)?
Currently I have an ASUS Prime Z370-A motherboard, with a single GTX1060 video card in the main pcie16 slot, and 3 I/O cards in 3 of the 4 pcie1 slots. The other two pcie16 slots are empty.
The path of least resistance might look like just getting both Intel 905p's as pcie4 cards and putting them into the two open pcie16 slots. As nearly as I can tell from the motherboard manual, the two main pcie16 slots would then run as 8 lanes each from the CPU. Apparently the 3rd pcei16 slot is always only 4 lanes supplied from the Z370 chipset.
But that has 2 drawbacks. I'd like to leave that last pcie16(4 lane) slot open for a 10G ethernet card in a couple years. Maybe this 2nd thing is a non-issue, but I've seen some benchmarks saying the chipset adds about 2us latency. Normally for nand-based SSDs like Samsung 970 that's almost nothing, but for Optane with ~10 us latency.... or maybe those benchmarks aren't real and the drive would perform the same on chipset lanes as with direct cpu lanes?
If the chipset lanes are ok, then I could get one of the 905p in the U.2 form factor and use the U.2 to M.2 cable to plug it into the main M.2 connector. But every indication I've seen is both M.2 connectors are chipset lanes, and one of them shares 2 of its 4 lanes with some of the SATA ports (but I'm not planning to ever use those), so whether chipset lanes add latency is a big question?
Another option might be the Asus Hyper M.2 X16 card. The Prime Z370-A manual specifically says this card is supported, and there's options in the bios for it (which presumably turn on pcie bifurcation, but the manual doesn't use such technical terms). The manual specifically does say a maximum of 3 of the 4 drives can be used if the Hyper M.2 X16 is in the first pcie16 slot with the 2nd pcie16 slot empty, and a max of 2 of 4 usable if it's installed in the second pcie16 slot.
The big question... would those U.2 drives work, connected through those U.2 to M.2 cables (which Intel ships with the U.2 drive), then though that Hyper M.2 X16 card, and then through whatever lane multiplexing the motherboard does to split the 16 lane to 8 lanes on each slot? Maybe all that stuff connected in tandem is no big deal?
Or maybe the ultimate answer is to scrap the Prime Z370A board (which was cheap compared to even one of those 905p drives), and also the i7-8086K chip (not as cheap) and step up to a X299 motherboard? But to get a chip with 44 pcie lanes adds about $1000 more. I'm also not so excited about those big chips due to lower single threaded performance. Sadly, quite a lot of my typical workload isn't very well multithreaded.
Another option might involve ditching the GTX1060 card and just using the CPU's graphics. I never run any games, so 3D doesn't really matter. I use the card mostly because prior attempts at 4K didn't work on anything but the nVidia cards, mostly graphics drivers on Linux tend to suck - nVidia's close-source driver is the exception. I absolutely need a 4K desktop (but would upgrade to 5K or 8K in a heartbeat if large format monitors existed).
The workload by the way is mostly software development and engineering like PCB CAD. I run only Linux, with a few virtualbox machines for 2 legacy Window programs, but there's no native Windows partition at all. The main motivation for spending so much on storage is speeding up lengthy software compile times, where makefiles and scripts process many thousands of small files (mostly single threaded & lots of filesystem access) and then lots of code is compiled (scales well to many cores). Two drives, allows the /tmp folder (and OS+software) on 1 drive and my home directory is on the other. The compiler makes heavy use of temporary files, which is the main reason to use 2 drives.
But how to best connect them for optimum performance?
Currently I have an ASUS Prime Z370-A motherboard, with a single GTX1060 video card in the main pcie16 slot, and 3 I/O cards in 3 of the 4 pcie1 slots. The other two pcie16 slots are empty.
The path of least resistance might look like just getting both Intel 905p's as pcie4 cards and putting them into the two open pcie16 slots. As nearly as I can tell from the motherboard manual, the two main pcie16 slots would then run as 8 lanes each from the CPU. Apparently the 3rd pcei16 slot is always only 4 lanes supplied from the Z370 chipset.
But that has 2 drawbacks. I'd like to leave that last pcie16(4 lane) slot open for a 10G ethernet card in a couple years. Maybe this 2nd thing is a non-issue, but I've seen some benchmarks saying the chipset adds about 2us latency. Normally for nand-based SSDs like Samsung 970 that's almost nothing, but for Optane with ~10 us latency.... or maybe those benchmarks aren't real and the drive would perform the same on chipset lanes as with direct cpu lanes?
If the chipset lanes are ok, then I could get one of the 905p in the U.2 form factor and use the U.2 to M.2 cable to plug it into the main M.2 connector. But every indication I've seen is both M.2 connectors are chipset lanes, and one of them shares 2 of its 4 lanes with some of the SATA ports (but I'm not planning to ever use those), so whether chipset lanes add latency is a big question?
Another option might be the Asus Hyper M.2 X16 card. The Prime Z370-A manual specifically says this card is supported, and there's options in the bios for it (which presumably turn on pcie bifurcation, but the manual doesn't use such technical terms). The manual specifically does say a maximum of 3 of the 4 drives can be used if the Hyper M.2 X16 is in the first pcie16 slot with the 2nd pcie16 slot empty, and a max of 2 of 4 usable if it's installed in the second pcie16 slot.
The big question... would those U.2 drives work, connected through those U.2 to M.2 cables (which Intel ships with the U.2 drive), then though that Hyper M.2 X16 card, and then through whatever lane multiplexing the motherboard does to split the 16 lane to 8 lanes on each slot? Maybe all that stuff connected in tandem is no big deal?
Or maybe the ultimate answer is to scrap the Prime Z370A board (which was cheap compared to even one of those 905p drives), and also the i7-8086K chip (not as cheap) and step up to a X299 motherboard? But to get a chip with 44 pcie lanes adds about $1000 more. I'm also not so excited about those big chips due to lower single threaded performance. Sadly, quite a lot of my typical workload isn't very well multithreaded.
Another option might involve ditching the GTX1060 card and just using the CPU's graphics. I never run any games, so 3D doesn't really matter. I use the card mostly because prior attempts at 4K didn't work on anything but the nVidia cards, mostly graphics drivers on Linux tend to suck - nVidia's close-source driver is the exception. I absolutely need a 4K desktop (but would upgrade to 5K or 8K in a heartbeat if large format monitors existed).
The workload by the way is mostly software development and engineering like PCB CAD. I run only Linux, with a few virtualbox machines for 2 legacy Window programs, but there's no native Windows partition at all. The main motivation for spending so much on storage is speeding up lengthy software compile times, where makefiles and scripts process many thousands of small files (mostly single threaded & lots of filesystem access) and then lots of code is compiled (scales well to many cores). Two drives, allows the /tmp folder (and OS+software) on 1 drive and my home directory is on the other. The compiler makes heavy use of temporary files, which is the main reason to use 2 drives.
But how to best connect them for optimum performance?
Facebook
Twitter