Do we need DDR4 when high speed DDR3 has such little effect on overall performance?
That is a tricky question to answer because what we can't determine, as outsiders, is how much dependence on the memory has been "engineered out" of the existing crop of CPUs by way of larger and larger on-die caches combined with better and better prefetchers and prediction algos.
You might be compelled to respond along the lines of "well if the cache and prefetchers are doing such a bang-up job nowadays in insulating the CPU from the horrid performance of DDR3, then why not continue to depend on CPU designers to keep giving us more of the same there too? Making DDR4 as irrelevant as DDR3?"
The pitfall of this line of thinking is that we, as outsiders, don't know what trade-offs the design engineers are making in their decision of allocating x-amount of diespace for a larger cache, or spending more engineering resources crafting beefier prefetchers instead of allocating that same die-space and engineering resources towards developing every better intrinsically higher IPC circuits and so forth.
Today's processors don't need DDR4 because the engineers spent time, effort, and money (at a loss to spending it on other things while developing the CPU) to better insulate the processor from the deleterious performance robbing aspects of DDR3.
DDR4 may provide those same engineers, who are currently developing tomorrow's processors, with a lowered imperative for making design tradeoffs at the expense of developing better cores (more IPC, etc) instead of focusing those resources on developing better uncore circuits.
To throw in the obligatory automobile-inspired analogy, imagine if today's roads did not exist...that we all still drove on wagon-wheel rutted dirt and gravel roads. In such a world, today's auto engineers would find themselves spending an outsized percentage of their engineering resources in the development of tires, shocks, and suspension systems to insulate and isolate the rest of the car (and its occupants) from the realities of the road-surface.
Your jaguar wouldn't be a jag not just because of the road situation, but the jag engineers wouldn't have the resources to develop the rest of the car to be the high-performance machine that it is in our world.
Today's CPU engineers are sent into battle with one hand tied behind their back in that they are expected to spend a considerable amount of time and money developing support systems to buffer their high-performance cores from the dirty realities of what lies beyond the socket.
Better memory is like a better road, today's cars won't necessarily take immediate advantage of a smoother road but tomorrow's cars can be optimized and tuned for it, freeing up development resources that can then be used to improve other aspects of the processor.