It's definitely not a dumb question at all... in fact it's very complex question and there are engineers and financial analysts who spend an awful lot of time trying to figure out the "right" size for a CPU. There's five problems with increasing CPU size: power, intra-die process variation, defect density, die-per-wafer, and most fundamentally the "reticle limit".
There's a really great discussion in this thread (from 7 years ago).
http://forums.anandtech.com/messageview.aspx?catid=50&threadid=1613541
CPU's are manufactured on a thing circular sheet of silicon called a wafer - which looks like
this. Each of the squares in that sheet is a CPU which people who work in the industry call a "die" (probably from machine shop jargon). Starting with "die per wafer" as the first problem, clearly if you make the individual chips bigger, then you will get fewer of them, but not only are they bigger but any die that touch the edge can't be used (because they are missing a chunk that falls off the side). The bigger each die is, the more space as a percentage that you can't use along the edge of the wafer. But also each individual CPU is bigger and you get fewer of them overall and both of these factors increase manufacturing cost.
Next is defect density. You can imagine defects and being something like a pinch of sand thrown over the wafer. If a grain of this hypothetical sand touches a CPU die, then that CPU won't work and you have to throw it away. So when you throw your pinch of sand on the wafer, you see it scatter randomly all over, and wherever it lands, the CPU won't work. So you can imagine that the bigger each CPU die is, the more of them that won't work. So if you made them all super huge so that there were four of them in a 12" wafer - hypothetically, as we will see, you can't do this - if there were 4 total... then none would work because your grains of sand would touch all 4... unless you got lucky and then you might get one that works. So, the smaller each die is, the more likely it is that it won't have a defect in it and that it will work... and the larger they are, the more you will have to throw away.
Then we start to get into the somewhat more confusing technical reasons. One problem is called intra-die process variation - and I know this one personally because I have worked on several of the largest CPUs that that have ever been made (the Itanium CPU's codenamed: Montecito and Tukwila) and I know very well the problems you have when the die gets enormous. One problem is that they are so big that the electrical characteristics of the transistors vary over the surface of the die. Basically, when you make a CPU die, it sometimes comes out fast and sometimes comes out slow. If it's a fast one, you mark it as being fast and sell it for more money, but if it's a slow one you mark it as slow and sell it for less money. All CPU manufacturing follows this rule... all CPU's come out slightly differently from each other when they are manufactured and some will be fast and some will be slow and some will burn more electricity and some use less and this is called "process variation". When a die gets really big, you can have both slow and fast sections in the same die and this makes the whole die effectively "slow" because you can only run it as fast as the slowest section.
Next you have the "reticle limit". This is a fundamental limit of the lens system that is used to focus the light to make the chip. There's this answer from the old thread mentioned above by Anandtech user Eskimo that explains it:
Lastly - and it's not one of the main reasons, but it's still important - is power... which goes up the larger a die is. You have to send the clock signal farther which burns more power, there are more transistors in there that use power. Power is higher in larger dies because they are bigger... So those are five good reasons why manufacturers keep CPU die sizes small - and in these days when customers are happy with "good enough" performance, and power is very important and the price becomes one of those most important factors, it's likely that CPU die will continue to shrink into the future.
Patrick Mahoney
Microprocessor Design Engineer
Intel Corp.
* not an Intel spokesperson... just a random engineer that works there *