A little bit of color to add to the previous explanations. Why do they call it a stack? Because somewhere back in the dawn of computing some young researcher was trying to think of a metaphor for the data structure he had in mind, one which would allow him to save the state of each function call as program flow progressed from function to function. He saw a spring-loaded stack of plates in a cafeteria, and the term was coined. I used to know who it was and where it happened, but have forgotten, and couldn't find it. The analogy is now in very widespread use to describe the operation of a stack. It appears in Mark Chasin's
1984 book on Atari assembly language programming, but I'm sure even then he was just repeating it.
Stacks are devilishly interesting structures. They are architecturally simple in the extreme, but can be used to implement all sorts of state machines. To visualize one, you can think of the plate analogy, but to visualize what they are for, I like the bread crumb analogy.
Suppose you walk into an old mine, with tunnels branching every which way into the darkness. As you walk in you choose various branches, and are soon deep into what is essentially a n-ary tree. The stack is like a trail of bread crumbs that you drop on the floor as you walk in. When you want to leave you turn and follow them out. A program is also an n-ary tree of possible paths of control flow. A function A may call functions B or C depending on decision criteria, and then B may call D or F, and C may call E or G, and so on. At some point a function does its work and returns without calling any other function (this function is what we think of as a "primitive"), at which point the flow of control "unwinds" back through all of the previous functions. Each of them has a "frame" of processor state and local variables that was saved ("pushed") onto the stack. When the flow of control reenters that function the processor state and any local variables are "popped" off the stack.
It seems confusing to most people at first, but then you get it in a flash and you're done with it. It really is that simple at heart. Get it, and you'll understand programming
.