What exactly does cache do?

[CTho9305]

Well, every time a program accesses memory, the CPU first looks for it in the L1 cache. The L1 cache is small (<128KB), and programs can generate addresses anywhere in a 4 gigabyte range. If you're lucky, the address your program just asked for is currently stored in the cache - this is called a hit. If you're unlucky, it's not in the small set of data in the L1 cache and you have a "miss". You then have to check L2, and if you miss again, you go to main memory. With "virtual memory", there's another layer but I won't go into that here. It's like the game "battleship"... programs ask for locations, and if they're in cache you "hit", otherwise you "miss". The CPU does its best to guess which locations the program is likely to want, and keep those in the lower cache levels.

 

[harrkev]

Originally posted by: Googer

Originally posted by: sao123

L0 - on-chip registers
L1 - Close to the CPU - 8-64K or so
L2 - Slower than L1. Still on CPU. 128K to 1M or so.
L3 - 256MB to 1GB. Often called DDR DRAM. Made my Mushkin, PNY, Crucial, etc.
L4 - 100Gb. Made by Seagate, Western Digital (this actually happens when you use virtual memory)
L5 - Multiple Terrabytes. Service provided by EarthLink, RoadRunner, AOL, etc.

This is not necessarily a good pattern to follow...

Many Processors... including the P4EE, P4Xeon, Itanium 1 &amp; Itanium 2 Server Lines have on CPU L3 cache (1MB to 27MB)

What you are calling L3 is actually known as Main Memory. OR System RAM.

What you calling L4 is your Virtual Memory or Swap File. Not necessarily a cache, since it is slower than Main Memory.

What you are calling L5 is actually not even a form of cache. I'm not even sure where you are going with this. Web Cache on ISP DNS servers? This has nothing to do with the Software / Operating System processing &amp; Memory heirchy.

Agreed.

What I typed is true, if you look at it in a certain sense. If you are "too close to the problem," then you see hard drive space as being vastly different from RAM. On the other hand, if you look at it as just being ways to store bits which differ on the "performance vs. cost" scale, then what I typed makes sense. You *could* think of your page file as being your system memory, and your physical RAM as being a cache of your page file. It is strange, but valid if you stop and think about it.

 

[CTho9305]

Registers cannot be counted as cache with any reasonable definition - they're used in an *entirely* different way. To a userland program, all levels of cache are transparent - registers are not. There are other differences as well.

 

[sao123]

You *could* think of your page file as being your system memory, and your physical RAM as being a cache of your page file. It is strange, but valid if you stop and think about it.

Not really.
During the course of program execution, There will never be an instance of a Main Memory Miss, where the CPU will have to go to the Virtual Memory to find a hit. Programs are only placed into Virtual Memory when the CPU is no longer operating on them(paused state) and room needs to be created in Main Memory to load a new instance of a program execution. Upon restarting the program in the paused state, the entire program is reloaded into main memory and another is shelved to the virtual memory.
This is not really a form of cache.

Similarly, Web Server storage area has nothing to do with internal execution and system cache. Your cpu could never directly access a program instruction from an external wan storage medium. This is a chaching form for the internet yes... but not related to the execution of binary instructions in your cpu.

 

[CTho9305]

During the course of program execution, There will never be an instance of a Main Memory Miss, where the CPU will have to go to the Virtual Memory to find a hit.

That's not entirely true... a page fault is effectively a main memory miss. Calling the hard drive main memory, and DRAM a cache is not entirely unreasonable. It's possible to have a currently running program get swapped out - it's just going to result in terrible performance.

 

[Googer]

Originally posted by: harrkev

Originally posted by: Googer

Originally posted by: sao123

L0 - on-chip registers
L1 - Close to the CPU - 8-64K or so
L2 - Slower than L1. Still on CPU. 128K to 1M or so.
L3 - 256MB to 1GB. Often called DDR DRAM. Made my Mushkin, PNY, Crucial, etc.
L4 - 100Gb. Made by Seagate, Western Digital (this actually happens when you use virtual memory)
L5 - Multiple Terrabytes. Service provided by EarthLink, RoadRunner, AOL, etc.

This is not necessarily a good pattern to follow...

Many Processors... including the P4EE, P4Xeon, Itanium 1 &amp; Itanium 2 Server Lines have on CPU L3 cache (1MB to 27MB)

What you are calling L3 is actually known as Main Memory. OR System RAM.

What you calling L4 is your Virtual Memory or Swap File. Not necessarily a cache, since it is slower than Main Memory.

What you are calling L5 is actually not even a form of cache. I'm not even sure where you are going with this. Web Cache on ISP DNS servers? This has nothing to do with the Software / Operating System processing &amp; Memory heirchy.

Agreed.

What I typed is true, if you look at it in a certain sense. If you are "too close to the problem," then you see hard drive space as being vastly different from RAM. On the other hand, if you look at it as just being ways to store bits which differ on the "performance vs. cost" scale, then what I typed makes sense. You *could* think of your page file as being your system memory, and your physical RAM as being a cache of your page file. It is strange, but valid if you stop and think about it.

Some Early and Mid late AMD k6 processors (super socket 7) had An L3 cache that was solderd on to the motherboard so that rule is not always perfect. Where would that put ram? Generaly Cache is somthing that alows a processor to keep frequently used data nearby so it does not have to resort to using slower SYSTEM RAM. There can be many levels of cache Each one corresponding to a Speed level and the fastest always being L1 and the slowest being the last L(x) System Ram is not really considerd to be CPU Cache. Cpu CACHE is usualy embeded on to the CPU but someimes (really old motherboards) it can be found solderd to the motherboard right next to the CPU socket. On early Pentiums (66-133mhz) this was often reffered to L2 cache (massive 256 or 512k) and was not as fast as L1 on the CPU die. There was no L3 at that time for intel. The cool part was that if you had 256k of cache and the right motherboard you could upgrade it to 512k with a COAST module. System Ram never Gets an L designation and neither do hard or floppy drives.

512k! WoW! Most systems then only had 4 to 8 megabytes of ram! In proportion to todays systems that would be like haveing a processor with 128-256 megabytes of cache on board! AMAZING and disapointing at the same time. Where has the progress gone?

 

[Philippine Mango]

Originally posted by: sao123
I suggest you read this Article From CPU-Z ...contrasting the AMD K8 with the Pentium 4 line of CPU's. The section on caching scemes is excellently written.

Is the reason Intel does this is to improve cache coherency algorithms for multiprocessor designs?

No, the L1 &amp; L2 caches are both on the P4 cpu (unlike previously P!!! &amp; P2 were on the Motherboard.) So the cache is self contained, and not shared among the processors in a mulitple processor scheme.

Um, no that is not true at all! The Pentium II processor did not have L2 cache on the die but on the cartridge so it would run at the same speed as the processor. The L1 cache *I think it has always been on the processor die (probably not on much older processors like 286 or most likley 8088). The pentium boards had the L1 cache on the die and L2 cache on the motherboard, so you could actually upgrade your cache on your motherboard. The pentium III non cartridge had finally resolved the issue with the L2 cache and was able to incorperate the cache onto the die. There are Pentium III with cartridge and I'm not exactally sure why intel decided to stick with it for a long time but at least it gave more life into my PII system! thats my .02

 

[complacent]

Originally posted by: sao123

During the course of program execution, There will never be an instance of a Main Memory Miss, where the CPU will have to go to the Virtual Memory to find a hit.

What do you think happens the first time you open a program? That is surely within the course of program execution and I can guarantee that every time you turn on your computer and it is loading all of your applications you are getting tons of main memory misses. That is why Windows will allow you to start programs in the tray; so when you try to start them later on you don't get a main memory miss.

 

[sao123]

What do you think happens the first time you open a program?

The cpu does not execute a single instruction of that program until the program is loaded into main memory. Loading from the hard drive into memory is a function of the operating system and does not happen during the course of execution of a program. It happens before.

 

[complacent]

The cpu does not execute a single instruction of that program until the program is loaded into main memory. Loading from the hard drive into memory is a function of the operating system and does not happen during the course of execution of a program. It happens before.

Even then, no program is executed until the data is in a register. Strictly speaking, nothing gets executed until it is loaded into one of the CPU registers, so even data in L1 cache will not get executed until it is moved into the actual CPU.

We are all talking about different things here. We can look at this with a strict dichotomy; if the instruction is not in a register in the CPU, then it is not going to be executed. Otherwise, the further away from the CPU the data is the longer it will take to get the data.

Do you really think that when you are playing a video game (which is a program in the general term) that all of the textures are placed in memory? If we truly never had a main memory miss, we would never have to load things. Loading, by definition, is a main memory miss.

 

[Malak]

Originally posted by: Googer
Damn, This is the HIGHLY Technical forum. I was seriously hoping to find somthing other than a simple questen like this here.

This is not a simple question, and I'm having a hard time decyphering the answers.

 

[CTho9305]

Originally posted by: malak

Originally posted by: Googer
Damn, This is the HIGHLY Technical forum. I was seriously hoping to find somthing other than a simple questen like this here.

This is not a simple question, and I'm having a hard time decyphering the answers.

It is a fairly simple question, and has been answered - Ariste's explanation on 12/15 is excellent. I think questions like this are still a lot better than the tech support requests.

Many of the other replies are poorly written, and a lot of the later ones are arguing about trivial semantics: whether main memory can be considered a cache, whether you can have a "main memory miss", what defines "execution" for a CPU executing instructions, etc.

 

[Venix]

The cpu does not execute a single instruction of that program until the program is loaded into main memory. Loading from the hard drive into memory is a function of the operating system and does not happen during the course of execution of a program. It happens before.

Every modern OS I'm familiar with supports demand paging. The OS only loads a few pages of the executable and brings in the rest on page faults.

 

[Megamixman]

The point of cache is to make the most used data more quickly available. It has limitation because of the cost of increasing the cache. Look at how much more a P4 EE costs compared to a regular P4. That is because cache takes up a lot of transistors. Look at the die layouts of the P4 and A64, you will see that it takes up a lot of space and can even be more the half the die itself. So the bigger the die, the more area you have for defect, equaling less turn out, meaning the cost per chip increases. Intel could easily put a P4 core in the center of a wafer and use the rest of the die area for cache, but think how much each processor would cost and whether the profit made off those processors would justify the R&amp;D costs. Not in a million years. That is the reason you don?t see lots of cache, business.

 

[hoppa]

Originally posted by: silverpig
Laymans terms:

Hard drive = Library of Congress
RAM = Local library
L2 cache = file cabinet in your office
L1 cache = stack of papers on your desk

and i guess we'll have to say registers = brain

 

[hoppa]

Originally posted by: sao123

L0 - on-chip registers
L1 - Close to the CPU - 8-64K or so
L2 - Slower than L1. Still on CPU. 128K to 1M or so.
L3 - 256MB to 1GB. Often called DDR DRAM. Made my Mushkin, PNY, Crucial, etc.
L4 - 100Gb. Made by Seagate, Western Digital (this actually happens when you use virtual memory)
L5 - Multiple Terrabytes. Service provided by EarthLink, RoadRunner, AOL, etc.

This is not necessarily a good pattern to follow...

Many Processors... including the P4EE, P4Xeon, Itanium 1 &amp; Itanium 2 Server Lines have on CPU L3 cache (1MB to 27MB)

What you are calling L3 is actually known as Main Memory. OR System RAM.

What you calling L4 is your Virtual Memory or Swap File. Not necessarily a cache, since it is slower than Main Memory.

What you are calling L5 is actually not even a form of cache. I'm not even sure where you are going with this. Web Cache on ISP DNS servers? This has nothing to do with the Software / Operating System processing &amp; Memory heirchy.

what he said isnt really wrong, its just mislabeled. its the standard memory hierarchy (registers, cache, memory, disk, network), not cache levels. still, its a good thing to have in here.