Discussion SPEC06/17 IPC vs CB R15 IPC Comparison.

nicalandia

Golden Member
Jan 10, 2019
1,196
1,306
106
According to anandtech.com the SPEC suit is the "Go To" bench for IPC testing, but as you can see they have OG Zen beat Skylake IPC, which at the time OG Zen was on par with Broadwell-E and just behind Skylake, They also show Excavator only 44% behind Zen which is about 52% ahead of Excavator in reality.

Here is the SPEC06/17 Chart.

1637674599219.png





Here is the CB R15 just for reference.
1637674735698.png


Another Chart
1637675166913.png

Also there were no IPC improvement from Skylake to Comet Lake.
 
Last edited:

JoeRambo

Golden Member
Jun 13, 2013
1,534
1,611
136
Cinebenches are unique in being completely irrelevant to real world performance. I think the most extreme example were those Threadrippers with half of cores not having direct access to memory. They continued to scale almost perfectly in Cinebenches while regressing in performance in quite a few other tests.

Rendering is a niche, and rendering where said rendering is done with tiles that are optimized to mostly fit cores L2 cache and not touch memory has no relation with IPC of actual CPU elsewhere.
There are obviously finer points between Cinebench versions, 15 vs 23 etc that vary in how useless they are.

Anandtech IPC results are meant to be strictly compared to Anandtech benchmark results ONLY. Posting Anandtech IPC and something from other site that is running real world memory config does not reveal anything of value.
But obviously you can still compare "SPEC difference is 10%, but in Anandtech web rendering the difference is 20%"
 

nicalandia

Golden Member
Jan 10, 2019
1,196
1,306
106
Anandtech IPC results are meant to be strictly compared to Anandtech benchmark results ONLY.
How is Skylake 24% behind Comet Lake in IPC? If anandtech say stock vs stock well isn't IPC supposed to be benched at ISO Speed?

1637678340988.png
 
Last edited:

JoeRambo

Golden Member
Jun 13, 2013
1,534
1,611
136
How is Skylake 24% behind Comet Lake in IPC? If anandtech say stock vs stock well isn't IPC supposed to be benched at ISO Speed?
Memory speeds increased and so did cache. L3 went from 8MB to 20MB and memory speed increased from 2133 to 2933. With the way Anandtech tests at JEDEC timings, cache size and memory speed improvement have oversized impact.
Also compilers have changed, making comparisons of rather dubious value as well.
 

Doug S

Golden Member
Feb 8, 2020
1,110
1,656
106
Cinebenches are unique in being completely irrelevant to real world performance. I think the most extreme example were those Threadrippers with half of cores not having direct access to memory. They continued to scale almost perfectly in Cinebenches while regressing in performance in quite a few other tests.

Yep, it is the modern equivalent of Dhrystone - only useful for measuring how well stuff that almost never needs to touch RAM will run.
 
  • Like
Reactions: Nothingness

Hitman928

Diamond Member
Apr 15, 2012
4,057
4,768
136
it's the most recent anandtech article, let me try to find it

Looks to me like that chart is pure performance, not IPC.

Edit: Nevermind, after looking closer, it seems to be IPC even though it's not titled/mentioned as such. Without knowing exactly how they calculated the chart, it's hard to comment. Your Skylake over Excavator math is a little off though, it is 41% advantage to Skylake, not 27%. There are also a few tests in Spec that are heavily memory bound so using DDR5 Alderlake or increasing CPU cache between generations may have an exaggerated effect in those tests leading to a little bit of a skew in the overall IPC calculation.
 
Last edited:

nicalandia

Golden Member
Jan 10, 2019
1,196
1,306
106
Looks to me like that chart is pure performance, not IPC.

Edit: Nevermind, after looking closer, it seems to be IPC even though it's not titled/mentioned as such. Without knowing exactly how they calculated the chart, it's hard to comment. Your Skylake over Excavator math is a little off though, it is 41% advantage to Skylake, not 27%.
Really?

am I looking at something different? Excavator shows 66%, Skylake 94%, Gracemont 93%, am I doing the math wrong?

1637699875094.png
 

DrMrLordX

Lifer
Apr 27, 2000
19,165
7,924
136
@nicalandia

You should try comparing the SPEC 526.blender_r sub-results between two processors to the actual performance delta in release Blender builds between the same two processors, if you want to talk about SPEC. Humour me, it should be interesting.

(you aren't the only one to notice the discrepancies between SPEC performance deltas and other benchmarks, btw)
 

eek2121

Golden Member
Aug 2, 2005
1,800
1,986
136
According to anandtech.com the SPEC suit is the "Go To" bench for IPC testing, but as you can see they have OG Zen beat Skylake IPC, which at the time OG Zen was on par with Broadwell-E and just behind Skylake, They also show Excavator only 44% behind Zen which is about 52% ahead of Excavator in reality.

Here is the SPEC06/17 Chart.

View attachment 53277





Here is the CB R15 just for reference.
View attachment 53278


Another Chart
View attachment 53279

Also there were no IPC improvement from Skylake to Comet Lake.
Full disclaimer, I did not read all the comments in the post because I have been unwell, but I came here to say that there is not a single benchmark out there that can cover all performance scenarios. That includes SPEC and Cinebench.

As a general rule of thumb, a CPU that is a leader in both these benchmarks will likely be a performance leader, but that is absolutely NOT always the case.
 

tamz_msc

Diamond Member
Jan 5, 2017
3,123
2,878
136
Looks to me like that chart is pure performance, not IPC.

Edit: Nevermind, after looking closer, it seems to be IPC even though it's not titled/mentioned as such. Without knowing exactly how they calculated the chart, it's hard to comment. Your Skylake over Excavator math is a little off though, it is 41% advantage to Skylake, not 27%. There are also a few tests in Spec that are heavily memory bound so using DDR5 Alderlake or increasing CPU cache between generations may have an exaggerated effect in those tests leading to a little bit of a skew in the overall IPC calculation.
It's IPC for the AMD parts at least, and core performance for the Intel parts. The math checks out.

Excavator -> Zen: 66 -> 100: 51.5% uplift (AMD claim 52%)
Zen+ -> Zen 2: 103 -> 118: 14.5% uplift (AMD claim 15%)
Zen 2 -> Zen 3: 118 -> 141: 19.4% uplift (AMD claim 19%)
 
  • Like
Reactions: lightmanek

Abwx

Diamond Member
Apr 2, 2011
9,485
1,529
126
@nicalandia

You should try comparing the SPEC 526.blender_r sub-results between two processors to the actual performance delta in release Blender builds between the same two processors, if you want to talk about SPEC. Humour me, it should be interesting.

(you aren't the only one to notice the discrepancies between SPEC performance deltas and other benchmarks, btw)
Besides Spec, wich has a FP and INT benches, IPC can be grossly characterised with Cinebench for FP and something like 7 ZIP for INT, the latter is somewhat sensitive to RAM latency and speed contrary to the former.
 

Thala

Golden Member
Nov 12, 2014
1,309
612
136
Besides Spec, wich has a FP and INT benches, IPC can be grossly characterised with Cinebench for FP and something like 7 ZIP for INT, the latter is somewhat sensitive to RAM latency and speed contrary to the former.
I disagree with Cinebench, it contains too much hand written processor specific code in order to be used as reference.
 
  • Like
Reactions: deasd and scineram

JoeRambo

Golden Member
Jun 13, 2013
1,534
1,611
136
It also depends on CB version, for example CB15 was characterized in this classic piece here:


Basically a mix of SSE scalar and memory instructions. Does not make much use of vector units. Fits Zen architecture like a glove.

Overall what i hate about rendering benchmarks - is that they are irrelevant for 99.9% of people. At least with something like x264 or x265 encodes there are more people who do the encodes of their home collections etc, while with rendering - you are either in that sphere or not. And frankly with workload characteristics like that it is easy to see why rendering is moving to GPUs big time.
 
  • Like
Reactions: tamz_msc

Gideon

Golden Member
Nov 27, 2007
1,481
3,015
136
Overall what i hate about rendering benchmarks - is that they are irrelevant for 99.9% of people. At least with something like x264 or x265 encodes there are more people who do the encodes of their home collections etc, while with rendering - you are either in that sphere or not. And frankly with workload characteristics like that it is easy to see why rendering is moving to GPUs big time.
Yeah. Blender Cycles X is now available for RDNA1/2 GPUs (and soon on M1X with v 3.1) for free. Raytracing support (OptiX alternative) will probably be added in the near-future as well. It's often an order of magnitude faster on GPUs. It makes less and less sense to use it as the be-all-end-all multicore CPU benchmark.
 
Thread starter Similar threads Forum Replies Date
I CPUs and Overclocking 0

ASK THE COMMUNITY