NAND does not scale in the same way as CPU / GPU.
CPU and GPU get a little MHz and a bunch of extra transistors that can do more stuff. They keep the amount of silicon about the same and pack more onto it. NAND dies you want a specific amount of memory. They can't use the extra space for speed like a CPU or GPU can, it's purely shrinking the size of the features.
Look at DRAM as an example. 7 years ago PC3200 was the standard. 200 MHz and latency around 5 ns. Today we are at 800 MHz and latency around 5 ns. doubling of speed every 3 to 4 years, no real improvement in latency. That's the kind of improvement we should expect.
Also, what do you see when you compare a 2GB DIMM vs. a 4 GB DIMM...
4GB DIMMs, in general, are slower
It's easier to make fast low density memory than it is to make fast high density memory.
Unrealistic expectations are unrealistic. Initial SSD speed improvements were rapid because of immature firmware. Firmware has, for the most part, matured to a point of diminishing returns. Extra memory channels can buy some throughput, but that also costs money, expect that the SSD market will eventually be a little tiered in that regard, with product differentiation like the 320 vs. 510 coming to more vendors.
Aside from that, it's totally reasonable to expect speed improvements to plateau to about the same rate as it has been for DRAM. Expect little to no scaling on latency, and a slow, methodical speed improvement.