It'd be difficult to design something "better" than the PS4, while hitting the same price target.
The biggest bottleneck is the CPU, with the small Jaguar cores. They're very efficient little cores, though, and GPGPU should help alleviate the CPU bottleneck some.
A unified memory architecture with hUMA support, means that memory management is easier on the PS4 than any current or previous console... Or even against a PC.
The cooling solution is similar to what you'll see in some discreet GPUs, with an enclosed blower design. While it's slightly louder than the Xbox One's design, it's within 1dB from 3 feet, while keeping lower load temperatures while gaming. The more compact design allowed the PS4 to be substantially more aesthetically pleasing in it's case design, and also include the power supply unit in the case.
Where I feel improvements could be made were in the Controller. Going with the smaller battery wasn't a smart cost cutting measure IMHO. I also think that the surfacing on the analog sticks could have been slightly improved without adding cost. The light bar should have had the ability to dim or be turned off from the start. That said, it's still my favorite controller ever made.
Oh, yeah, the Wireless chip was another area that was cheaped out on a tad too much. I would have liked to see a bit higher end WiFi on a machine that's supposed to last as long as Sony intends. Dual Band, 11 AC, and 5GHz support, should have been included in the chip they sourced. And it would have likely only cost a few dollars extra. A slightly higher end Bluetooth adapter, would have also been smart for better headset support.
With my adjustments, I figure the PS4 would have cost an extra 20 dollars to produce... and would eliminate much of their common complaints about the system.