Moderate CPU loads are the issue. Why power up a high amp core when you can get buy with a very low amp one?
Intel has other problems, they can never match the die size of the ARM cores due to their rather excessive baggage they lug around for legacy support. On desktop chips where neither die size nor power are that big of an issue, it doesn't matter. For mobile parts the 5 CPU core Tegra 4 devotes most of its die space to GPU cores and still comes in smaller then Intel's parts which have two functional CPU units and a dated and tiny GPU in comparison.
Intel's only hope is the one thing they are always better at then everyone else. Fabrication technology. Given enough of a lead in fab tech, the rest is just noise.
Legacy rubbish? If it is legacy then it has been die shrunk for at least 10 years and be a spec of dust by now. Why must you insist on spouting this absolute lie about the x86 front end? You have absolutely no empirical evidence of this. It is a complete myth.
As for size, CloverTrail comes in at about 60 mm^2 on 32nm and the GPU and CPU cores are actually a fraction of that space, around 30%. The GPU has DX 10.1 support which adds to the area of the GPU significantly.
Tegra 4 is rumored to be 80 mm^2 on 28nm (assuming the Nvidia presentation slide was not just some rendered graphic) That gives Intel a lot of space to add performance, and if they start leveraging their process advantage, which they will, even if they are slightly behind on architecture that will make up for it.
T4 doesn't support OpenGL ES 3.0 from what I remember reading a decision most likely made to save on area too, so T4 is no miracle chip in fact it could look out of date very quickly.