Is there a limit to the amount of cache?

[Byte]

I've read somewhere that not only price limits the amount of L1 and L2 cache one can have. Anyone know what it is? If money isn't an issue, can you have 1GB of L1 or L2 cache?

 

[RSMemphis]

It's a question how much you can get on a die, internal processor structure (the L1 is closely tied in to the main CPU parts) and such.
With on-die L2, L1 cache does not have to be super large. L2 is relatively important though.

As we go to smaller structures, it is easier to incorporate more cache. In principle, any size cache could be used, die size limits this to a few MB right now, which is more than plenty in most cases anyway. Still, that much cache is *VERY* expensive right now.

 

[Sohcan]

<< If money isn't an issue, can you have 1GB of L1 or L2 cache? >>

Size is by no means the most important feature in a cache....hit-rate, latency, and bandwidth are. Many of the aspects that increase hit-rate (higher set-associativity, larger size) penalize the latency; large caches have longer wire delays, and higher set-associativity increases wire delay and mux fan-in. There's also block size...too small and you don't exploit spatial locality (hit-rate suffers), too big and you don't exploit temporal locality (hit-rate suffers, bandwidth overhead increases). Increasing bandwidth by increasing word length and number of ports (through true-multiporting, virtual multiporting, multi-banking, and multiple cache copies) can have their own effects on latency and occupied area. There's also replacement algorithms, write-through vs. write-back...caches are very complex. Remember that caches work by exploiting spatial and temporal locality; if you access data at a certain address, chances are you'll access the same data again soon, as well as data nearby. You might be surprised to learn that even small 8KB L1 caches on x86 MPUs can have average hit-rates as high as 96%.

Given the ever-increasing MPU clock rates, the latency hit of accessing data far from the CPU, and the persistent high-latency of DRAM, large caches located far off the CPU aren't feasible for improving performance anymore, even with faster (yet expensive) SRAM. Aside from workstation/server MPUs that may contain a larger L3 cache on a CPU cartridge, we're likely only to see caches integrated onto the CPU die for desktop MPUs. In that case, you're definitely limited by the target market and cost of the MPU....one thing that makes McKinley so groundbreaking is that it is the first MPU to feature an integrated custom L3 cache, weighing in at 3MB.