the problem i think is to determine what tasks are more "serial" and which are more "parallel". different programs are going to have different access profiles to main memory, the problem is who's going to profile these programs? and then once you've profiled these programs, you have to be sure you've captured the behavior of the memory accesses for that program in it's entirety in order to optimize memory accesses, if you had the dual memory approach that you are proposing. from a feasability standpoint, this requires a lot of overhead, not to mention the hardware complexity. there's also the factor that the profiling may not entirely be accurate because the accesses to main memory may be a function of the dataset the program is working with, something like a database for instance. queries doing multiple joins on huge sets will access main memory in a very parallel way if the memory is there. something like many small queries on small sets will have a more random, "serial" type access pattern. this is all for the same database software.
memory is more dependent on the actual hardware platform you're running, i.e. cpu, not necessarily the program you are running. the object of the memory hierarchy present in computers now is to hide as much of the hardware implementation from the program as possible. each type of ram is also just a storage medium. how the memory controller handles the interface between the cpu and the actual modules is what's going to determine the performance factor. dual channel ddr can certainly compete with the likes of rambus in terms of bandwidth, and does better with latency. rambus could compete in the latency department if enough channels are interleaved. even serverworks implemented a design using standard pc133 sdram that had enough bandwidth to compete with the likes of ddr.
as it stands right now, i think that if you make the memory damn fast, the performance you're going to get is going to be good enough not to warrant a dedicated channel for disk buffering and another for video streaming etc... as long as you take into account the worse case scenarios. with hammer coming out, the hardware complexity will be increased even more because you have the memory controller onboard now, and i see this as a trend for the future.