Perhaps somebody could go into more detail on this?
Well I can try to give it a go in general terms, although there are many guys that are way more knowledgeable here that can explain in much more detail
Think of Cache (all levels) as very high speed memory that boosts the access to instructions/data when the CPU requests them. If we look at a modern CPU, it is much faster than main memory, and if it had no cache, every time it needs certain data it will have to go through the comparatively slow memory bus to fetch the data. A modern CPU fetches data as follows:
CPU -> L1 -> L2 -> L3 -> RAM -> SSD/HD/CD
The L1 cache is the smallest but also the fastest, so that is where a CPU looks first. If there is a "hit", meaning that the instruction/data is found there, then it can be executed, if there is a "miss", then the CPU scans the L2 cache and so on. Usually every level is bigger than the one before it, meaning that it will hold more data, but it also ends up being slower. Lets give an example about the speed difference between data found on L2 cache and main memory:
Lets say a CPU accesses L1 in 3 cycles, L2 in 15 cycles, and main memory in 70 cycles. First we got a "miss" on L1 so we lost 3 CPU cycles, then a "hit" on L2 (15 cycles) so in total we found the needed data after 18 cycles. If the same data needs to be accessed from RAM, then 70 cycles are wasted. 18 vs 70 is less than 1/3 of the time needed, so you can see how cache can boost performance tremendously.
The way cache works is that the data that is most used is kept there, increasing the chance of getting a hit. Another way to increase chances of getting a hit, is having larger cache sizes (more data). Generally speaking, larger caches improve performance even at the cost of having to take more CPU cycles to access. For example, lets say our L2 cache is 512KB in size, and after doubling that to 1MB the latency increased by 20%. Which would be preferable? In such a case the 1MB cache would have increased the performance because it can hold twice as much data (increasing hit rate), so it is preferable. But we have to keep in mind that cache takes a big chunk of the CPU die size, so it can't be increased indefinitely. Also depending on the application/software used there can be situations where a certain threshold will be met, meaning that increasing the size may not help much and could even affect performance negatively.
There are many cache design decisions that have pros and cons, such as size, associativity, inclusive and exclusive etc, and there is no one perfect design for all situations. That's why if we compare Intel and AMD processors, their cache subsystem may seem similar on the outside, but is actually very different. Bulldozer for example will reportedly have a 1 to 1 cache ratio between L2 and L3 cache, 4MB L2 and 4MB L3 or 8MB and 8MB. AMD usually uses "exclusive" caches, meaning that data won't be duplicated on all levels of cache, so the L3 cache would have data other than what the L2 cache contains. Intel uses "inclusive" caches, meaning that data is duplicated on all caches. This method works well when the next level of cache is much larger than the one before it. For example, L1 64K, L2 2MB, L3 8MB.
I know there was a lengthy discussion about how AMD is making a mistake with their L2 and L3 cache ratios for BD, and I hope if doesn't open up again here, but there are so many details that we have no idea about (and never will). So talking about size alone being a handicap is futile in my humble view. Also keep in mind when we look at BD's cache on a per-Module basis, a Module has 2MB L2 and 4MB L3 (or 8MB for 4-Modules), so the L3 is still larger when looking at it this way.
Finally I just want to add that I believe even though AMD was the first to have an integrated memory controller/NB, it's actually Intel that has an edge here in terms of cache/memory subsystem performance. This is one area where I see Bulldozer improving on, and at least having parity with current i7 CPUs. Sandybridge of course will be a another beast, but like I said before BD would most likely compete against it with higher frequency SKUs.
Facebook
Twitter