4 cores aren't enough though. 6 cores is the current sweet spot for most gaming engines. Even Intel agrees with this now. The funny thing is they were saying quad cores were all we need right up until they released their first mainstream 6 core. Then that became what everyone needed. So long as it was Intel. No one buys Intel's marketing because they are following whatever AMD is doing. Nothing they have released in the last couple years has been proactive. Their entire lineup from consumer to enterprise has been a reactive exercise in trying to counter AMD.
Are you telling me a superior company lets an inferior company dictate where the market is going?
If it's about sales you can look at Mindfactory data showing AMD outselling Intel 2:1. You can talk about all the benchmarks, but in reality most consumers make a decision on price to performance for as much money as they want or can spend. No one really cares what the top CPU is when 98% of the people can't or won't afford it. The 9900k will not go down in history as the best CPU ever because it costs too much. It will never have the mindshare such as the 2500k garnered. It won't be as legendary the AMD Thunderbird. if you want to talk enterprise wins you can look at all of the supercomputers being built that are using AMD Epyc. I've seen 4 using Epyc and none with Intel.
Are you telling me they put inferior CPUs in supercomputers?
As far as servers we used to have Sun servers running our carrier grade VOIP cluster until they couldn't keep up with the traffic. Sun offered no product to get more performance per rack. We switched to IBM blades because they could deliver. 90-100% load down to 10% with the new servers. I don't buy the broadstroke "Sun makes the best servers" argument you are pushing.
4 (hyper-threaded) cores are still more than enough for gaming though. Last I checked, the 7700K was still beating every AMD CPU for gaming purposes, and in 99% of games is right on the heels of the 8700K/9900K. Intel didn't release the 8700K/9900K because they were behind in gaming and 'needed' 6 or 8 cores, they needed to release those chips so AMD doesn't double their core count on the desktop market. That would look bad PR wise.
Despite the great strides Zen made in terms of ST performance (relative to the construction cores), it was still lacking the sheer clockspeed and IPC of Intel, so AMD could only compete in one way - more cores. Nothing wrong with that strategy and it makes perfect sense given the situation. Unfortunately, not all software has 'caught up' and many can't take advantage of those extra cores yet.
Do most normal folk actually 'need' 6 or 8 cores? I'd argue probably not. Is it good to have? Of course, especially if you dabble in some CPU intensive tasks. It's really not a make or break situation on the desktop to have >4 cores. If your workloads revolved around heaviliy multi-threaded apps, chances are you would have invested into some of the older HEDT platforms already, or got Ryzen when it launched. For most 'normal' folk though? I bet if you surveyed a bunch of people on the street and asked them how many cores their PC / laptop / phone has, most of them wouldn't have a clue.
My daily machine is a 4C/4T i5 laptop and it does everything I need, including some casual gaming. Does it hold a candle to my 8700K desktop machine? Of course not, the 8700K benchmarks twice as fast, but its really not functionally twice as good in the majority of tasks that I do.
You are right that DIY buyers willl always favour price/performance, so AMD naturally sells better in this regard. I commend AMD on bringing affordable 6 core CPUs to the masses, and if that forced Intel to respond with Coffee Lake then that is great for everyone involved.
No one is expecting the 9900K to 'go down in history as the best CPU ever', it is what it is, a $500 flagship CPU.