In lay terms...
your cpu access is very fast, compared to main memory (ram), which is exceedingly slower.
Since most programming often involves loops and code reuse (aka functions etc), in order to speed things up a bit, instructions from RAM are copied into a smaller memory called the cache.
the reason for the different levels of cache are the following...
level 1 is very fast... almost as fast as the cpu itself. but it is very expensive.
if your ram was made out of the memory lvl 1 cache was, it would be 10x as expensive, and cost prohibitive. because it is so expensive, your cpu only have a few hundred kilobytes of it.
lvl 2 cache... somewhat slower, but cheaper
lvl 3 cache... even slower, and cheaper memory, same principal, but still faster than main memory (RAM)
So, when a cpu attempts to load an the next instruction from memory, it copies it into all 3 caches. when you get into a loop, the next time it goes to the top, it can fetch part or all of the loop from the faster memory (the cache), thus improving performance.
since lvl 1 is so small, it gets overwritten a lot, level 2 is larger and will get overwritten less, and lvl 3 is even larger and will get overwritten less than 2.
any cache hit is still faster than RAM access. If the next instruction is not in RAM, then it is slow.
If you understand the concept of the SWAP FILE (virtual RAM) its the same concept, except the reverse... instead of larger and slower, its faster and smaller.
in short... more cache = better performance for any program.