Benchmarking Turbo capable CPUs

Eeqmcsq

Senior member
Jan 6, 2009
407
1
0
Not too long ago, I surfed into some comments posted by an actual AMD engineer from their server division at amdzone.com. He was talking about Turbo mode, and said that since Turbo is a variable boost, it's more useful in a desktop environment than a server environment. He said server customer prefer more consistent, predictable performance. They don't want to start asking questions about why a particular task runs faster on a light server load, but slows down on a heavier server load. I'm paraphrasing from memory, but that's along the lines of what he said. [EDIT: Having found one of his original threads, I think I remembered wrong. He was talking more about power draw than performance, so never mind this part. :( ]

I don't really follow server CPUs all that much, so I simply shrugged and thought that he could be right, or he could be wrong, and I simply filed it away in the back of my head.

Yesterday, the Lynnfield NDA was lifted, and all the benchmarks from various sites were unleashed to us. I read or skimmed through most of them, and I saw how the Lynnfield processors excelled in nearly every test. In some cases, the addition of Turbo mode really helped out, such as in single threaded tests.

But then the stuff that the AMD engineer said started coming back to me and made me think: Sure these single threaded benchmarks look great, but if you (or the OS) is busy multitasking heavily in the background, you're NOT going to get the same level of performance. The problem is that readers who read these benchmarks don't naturally think that way. They see the single threaded benchmarks and will mistakingly think that Lynnfield will ALWAYS run that fast, instead of thinking that it CAN run that fast.

Though this issue only applies to Intel right now, since only they have Turbo processors, AMD might implement their version in a future CPU too, and once again, we'll have readers that could misinterpret CAPABLE performance for GUARANTEED performance.

So do you think tech sites should benchmark Turbo capable CPUs both with and without Turbo enabled?

My personal opinion is that they should. Benching without Turbo would give the reader the "guaranteed minimum" a CPU is capable of. Anything extra is a free bonus.
 

Idontcare

Elite Member
Oct 10, 1999
21,118
58
91
Since turbo is controlled by power-consumption, and power-consumption is controlled by active thread count and instruction mix, it is all very much application specific, as is the benchmark application and result itself.

Meaning the performance boost of turbo-mode as seen in benches of specific applications is exactly what you see, it boosts the performance in those applications and the reader would realize that same performance boost should they use their i5 rig to run that application too.

Change the application and you change performance anyways, benchmarked or not, for more reasons than just the fact that turbo-mode itself might be different running that different program.

IPC, instructions per clock, averaged over a myriad of instructions used to process code and data will vary from application to application (naturally).

In fact it is this fact alone that makes the server-guy's comment from AMDzone a bit of a puzzle, regardless whether the cpu's frequency is moving up or down the IPC of the software he is running on the server already is moving up and down from app to app...so he's already dealing with variable performance to begin with.

So you've got varying applications, each with their own characteristic instruction mix and thus its own effective/average IPC, combined with a variable clockspeed at the cpu resulting in a specific performance number for benchmarking. Who cares?

The only case where people would care about non-turbo numbers is for folks who are trying to dissect the performance results down to another level of architectural rational for making comparisons at a more fundamental level. Doing that isn't fodder for determining what rig to buy, you do that to make conversations about how/why the performance you see with turbo enabled is what it is, etc. I.e. the same reason people like to see clock-to-clock comparisons, it makes architectural analysis all the easier.

Given how many performance features are buried in an architecture below the level of its clockspeed, I find it humorous that anyone at an IT level would care about turbo. Do they get persnickety about cache hit rates? Because the branchiness of their applications will have variable speeds on one architecture versus another as branch prediction fails (or succeeds) and as cache hits miss (or hit) and yet I doubt these guys would turn off cache or prediction tables (if they could) just to eliminate this variability in performance of their servers under load.
 

Soleron

Senior member
May 10, 2009
337
0
71
He also said similar about SMT. It can give +30% performance... but, depending on what other apps you are running in the background and whether those threads are running on a different core or the same one but on the other thread, it can reduce performance by 5% too.

So again it can show performance you won't get in the real world where you're not running on a clean install. AMD will never use SMT according to that server guy for that reason; they will use a different multithreading technoogy called CMT that gives consistent performance.


Also with Turbo mode, if Intel gives good samples to review sites so they don't reach near TDP at normal clocks and can therefore turbo more often, it could again misrepresent what retail CPUs can achieve.
--

That server guy on amdzone is really good at explaining what AMD is doing and why they made the choices they did. For example, AMD's TDP is 100% of theoretical max power while Intel's is some fraction of that that used to be 70% for P4s but has been falling to even 40% with recent CPUs - with no technological reason. AMD's ACP is the figure that is comparable to Inte's TDP.
 

Keysplayr

Elite Member
Jan 16, 2003
21,209
50
91
Originally posted by: Eeqmcsq
Not too long ago, I surfed into some comments posted by an actual AMD engineer from their server division at amdzone.com. He was talking about Turbo mode, and said that since Turbo is a variable boost, it's more useful in a desktop environment than a server environment. He said server customer prefer more consistent, predictable performance. They don't want to start asking questions about why a particular task runs faster on a light server load, but slows down on a heavier server load. I'm paraphrasing from memory, but that's along the lines of what he said.

Keys - At worst, the CPU's that employ Turbo Mode will only revert to it's advertised clock rate. So if you have a Lynnfield CPU running in a server at 2.66GHz, That is the "slowest" each core can get under multithreaded load. What you heard from the AMD engineer was true in some aspects. But there is no bite in his comment about a server customer preferring more consisten, predictable performance when that is exactly what a Lynnfield CPU would give. Worst case scenario is the CPU will run at rated speed, which still blows away MUCH higer clocked PhenomII's. No matter how you look at it, Turbo Mode is a bonus above and beyond what the CPU is rated for.

I don't really follow server CPUs all that much, so I simply shrugged and thought that he could be right, or he could be wrong, and I simply filed it away in the back of my head.

Yesterday, the Lynnfield NDA was lifted, and all the benchmarks from various sites were unleashed to us. I read or skimmed through most of them, and I saw how the Lynnfield processors excelled in nearly every test. In some cases, the addition of Turbo mode really helped out, such as in single threaded tests.

But then the stuff that the AMD engineer said started coming back to me and made me think: Sure these single threaded benchmarks look great, but if you (or the OS) is busy multitasking heavily in the background, you're NOT going to get the same level of performance. The problem is that readers who read these benchmarks don't naturally think that way. They see the single threaded benchmarks and will mistakingly think that Lynnfield will ALWAYS run that fast, instead of thinking that it CAN run that fast.

Keys - Not likely. In the reviews we have seen, this is usually WELL explained and made clear to readers what exactly Turbo Mode is.

Though this issue only applies to Intel right now, since only they have Turbo processors, AMD might implement their version in a future CPU too, and once again, we'll have readers that could misinterpret CAPABLE performance for GUARANTEED performance.

So do you think tech sites should benchmark Turbo capable CPUs both with and without Turbo enabled?

Keys - Sure do. Turbo mode set to "OFF" would be the baseline and an indication of the lowest possible performance in a given app. Turbo Mode set to "ON" would offer different degrees of performance gains depending on the app. Besides, I don't think anyone who owns these CPUs would turn off Turbo Mode.

My personal opinion is that they should. Benching without Turbo would give the reader the "guaranteed minimum" a CPU is capable of. Anything extra is a free bonus.

Keys - Totally agree.

 

Eeqmcsq

Senior member
Jan 6, 2009
407
1
0
Did some searching to see if I could find the original threads, in case my memory is wrong. Here's one I found that I do remember reading.

AMDZone

The guy's screen name is JF-AMD.
 

Eeqmcsq

Senior member
Jan 6, 2009
407
1
0
Sorry guys, I guess I remembered wrong on some parts about what JF-AMD said. The question about benchmarking without Turbo as a "baseline" remains valid, though.
 

exar333

Diamond Member
Feb 7, 2004
8,518
8
91
Originally posted by: Soleron
He also said similar about SMT. It can give +30% performance... but, depending on what other apps you are running in the background and whether those threads are running on a different core or the same one but on the other thread, it can reduce performance by 5% too.

So again it can show performance you won't get in the real world where you're not running on a clean install. AMD will never use SMT according to that server guy for that reason; they will use a different multithreading technoogy called CMT that gives consistent performance.


Also with Turbo mode, if Intel gives good samples to review sites so they don't reach near TDP at normal clocks and can therefore turbo more often, it could again misrepresent what retail CPUs can achieve.
--

That server guy on amdzone is really good at explaining what AMD is doing and why they made the choices they did. For example, AMD's TDP is 100% of theoretical max power while Intel's is some fraction of that that used to be 70% for P4s but has been falling to even 40% with recent CPUs - with no technological reason. AMD's ACP is the figure that is comparable to Inte's TDP.

Nice fanboy answer there. SMT and Turbo were features added to make the processor faster and more efficient, of course these features should be represented in a review. Intel has clearly stated the (impressive) turbo stages of the i5/i7, and "cherry-picked" samples is a bunch of FUD.

AMD's ACP <> Intel's TDP.

The i5/i7 is more efficient per watt than AMD's offerings, plain and simple. Did you even read the review article on AT?
 

VirtualLarry

No Lifer
Aug 25, 2001
56,230
9,990
126
Originally posted by: Majic 7
Here ya go. http://www.pcper.com/article.p...776&type=expert&pid=14 AMD doesn't win this way either.

Interesting, but they chose a Q8400 to represent the C2Q category, which was unfortunate. The Nehalems have 8MB of L3 cache, right? The Q8400 only has 4MB of L2 cache. I would have preferred to see them use a downclocked Q9550 or something with the full 12MB of L2 cache. The scoring differences between the C2Q and the Nehalem could have been down to cache differences, not architecture.
 

Majic 7

Senior member
Mar 27, 2008
668
0
0
I don't think the OP was interested in the differences between Intel Cpus.
 

Keysplayr

Elite Member
Jan 16, 2003
21,209
50
91
I am quite surprised at how well the PhenomII does compared to C2D. Not bad AMD.
 

Soleron

Senior member
May 10, 2009
337
0
71
Originally posted by: ExarKun333
Nice fanboy answer there. SMT and Turbo were features added to make the processor faster and more efficient, of course these features should be represented in a review. Intel has clearly stated the (impressive) turbo stages of the i5/i7, and "cherry-picked" samples is a bunch of FUD.

AMD's ACP <> Intel's TDP.

The i5/i7 is more efficient per watt than AMD's offerings, plain and simple. Did you even read the review article on AT?

i5 certainly outperforms any Phenom, I'm not denying the obvious. And it uses less power. But review performance is best-case with regards to SMT and Turbo, YMMV, and that should be made clear in the article (it is in Anandtech's) and also show benchmarks without SMT or Turbo for at least one chip so we can see the difference (Anand does it once but not in each review graph).

AMD TDP <> Intel TDP too. But Intel's TDP is worthless, as the random fraction of max power they give with it has no physical basis like AMD TDP or ACP does. Intel's actual max power is something around 170W for a 55xx Xeon. So no one, not Anandtech or customers, should be comparing the TDPs together. Only real-world thermal tests should be used (and Lynnfield is cooler than Phenom in this).

The samples don't have to be 'cherry-picked' (we'll never know if they are), but your retail chip could be better or worse than the review sample.
 

Idontcare

Elite Member
Oct 10, 1999
21,118
58
91
Originally posted by: Soleron
AMD TDP <> Intel TDP too. But Intel's TDP is worthless, as the random fraction of max power they give with it has no physical basis like AMD TDP or ACP does. Intel's actual max power is something around 170W for a 55xx Xeon. So no one, not Anandtech or customers, should be comparing the TDPs together. Only real-world thermal tests should be used (and Lynnfield is cooler than Phenom in this).

The samples don't have to be 'cherry-picked' (we'll never know if they are), but your retail chip could be better or worse than the review sample.

For nehalem-based cpus, Intel's TDP really is the max power-consumption the chip can dissipate unless you intentionally turn off the current limits thru the bios (if your mobo/bios even give you the option).

Intel has added an Overspeed Protection to its Core i7 processors, keeping them from exceeding 130W or 100A.

http://www.tomshardware.com/re...king-core-i7,2063.html

How does this make Intel's TDP worthless?
 

dmens

Platinum Member
Mar 18, 2005
2,271
917
136
AMD TDP <> Intel TDP too. But Intel's TDP is worthless, as the random fraction of max power they give with it has no physical basis like AMD TDP or ACP does.

ill say it again, both intel TDP and amd TDP don't mean "max power". i realize the superfudged prescott TDP numbers gave the conspiracy people a bunch of ammo to work with, but i can guarantee the actual max power for both intel and amd procs is much higher than the TDP.

Intel's actual max power is something around 170W for a 55xx Xeon.

depending on the sample, if you run a custom power virus, that actually *might* be true, perhaps a tad on the high end. but that is no different from the actual max power for an opteron, which is also higher than the stated TDP, and far higher than the stated ACP.
 

Phynaz

Lifer
Mar 13, 2006
10,140
819
126
Originally posted by: Eeqmcsq
Did some searching to see if I could find the original threads, in case my memory is wrong. Here's one I found that I do remember reading.

AMDZone

The guy's screen name is JF-AMD.

JF-AMD = AMD marketing droid.
AKA paid mouthpiece. His job is to make the competition look bad.
 

Avalon

Diamond Member
Jul 16, 2001
7,565
150
106
I've actually been curious to see what the differences were on benches with turbo disabled and enabled. Thanks Majic!