We need to be able to look for a relevant threshold when advising and comparing, especially reviewers and this is done by taking the benchmark, its bench result in actual figures, its applicability to real world and its total runtime and sizes in consideration. % statistics are useful to determine which is faster, for analysis, but don't tell the complete tale for an end user.
The best thing is to use benchmarks that measure the different aspects of the CPU in a concise and targeted manner, such as:
cache performance
integer performance
floating-point performance
AVX performance
AVX-2 performance
graphical performance (for integrated graphics)
performance per watt, maximum and minimum
performance per watt over time
A one-size-fits-all benchmark can have big drawbacks, like relying overly on a FP-heavy benchmark to characterize the difference between an 8370 and a 3770K. 8 integer cores and 4 floating point units is a design that is going to look particularly sub-par in a FP-heavy benchmark, unless those 4 floating point units are really powerful.
"Real-world" is tricky because applications can be coded in a manner to favor one architecture over another. Cinebench, for instance, could be updated to further favor Intel by leaning even more heavily on something like AVX-2. I assume Skylake and Kaby Lake are going to have stronger AVX-2 performance than Zen. So, all one needs to cook up a benchmark that proves how sad Zen is is something "real world" like Cinebench "12" that leans very heavily on a specific Intel advantage. The opposite is a benchmark that doesn't use AVX at all in a circumstance where it would provide additional performance.
If Intel were putting Broadwell C-style EDRAM in its chips a "real world" benchmark could also be designed to heavily favor a large victim cache. Since Zen is unlikely to have anything like that initially then that could be a big marketing edge. "Zen falls short of Kaby Lake by 60%!" (in benchmark that leans heavily on 128 MB of L4 cache). Of course, if Intel and the benchmark maker were to pursue this more obviously-exposed avenue it would have been wise to put the L4 on all Skylake as well as Broadwell E parts. Even more clever would have been to withhold the L4 until Skylake to claim that the big performance advantage is due to the newer Skylake cores.
Facebook
Twitter