AMD's Gigahertz Equivalency: Inexperienced Buyers Accept Bad Science - Aberdeen Group white paper

[lookin4dlz]

I know this topic has been previously discussed in a long thread, but I don't know how much fact was in that thread. This research article (you have to register at the site before you can view the document) explores the topic more fully.

What is really kind of funny is that all of us on this board, who think of ourselves as experts compared to the average buyer, were duped as badly as the average buyer was. Even more interesting is the fact that many people continue to stand by AMD despite this fact. (disclosure statement: I was on the original AMD bandwagon as an Intel alternative when AMD first became competitive with Intel. A lot of processor errors & a chip failure later I moved to Intel)

Arberdeen Group research article

This White Paper focuses on Advanced Micro Devices (AMD) and details how the company is taking a step down a slippery slope of bad science. Specifically, AMD is naming its Athlon XP line of microprocessor models using clock-speed gigahertz ratings equivalent to Intel's competing Pentium 4 - based on a set of application benchmarks. Aberdeen's research suggests that AMD's processor equivalency methodology is seriously flawed: It assumes a specific application usage model that does not apply to many users; it is platform-specific, ignoring critical differences such as memory type; and it is inconsistent between mobile and desktop processors. Moreover, the methodology uses system-level benchmarks including Input/Output, an approach that is not used by the industry to measure processors alone.

====================
The following article isn't specificallly about the AMD situation, but does reference it as the quotes indicate.

Fast PCs: Maxed-Out Architectures?

Of course, AMD claims much more, providing performance charts at its site that show the Athlon XP clobbering Intel's CPUs. We have yet to see such a pummeling in our labs.

...the performance is not scaling as AMD claims it should.

 

[lookin4dlz]

Even more funny - AMD used Arthur Andersen (apparently birds of a feather) to audit it's results Read the Arberdeen analyst's quote below:

Arthur Andersen ought to explain the discrepancies between its audited results and what is reported at BAPCo.

 

[DivideBYZero]

*Yawn*

A lot of processor errors & a chip failure

Like Intel don't have processor errata? LMAO

What processor errors did you get?

How did you hose your chip?

 

[omar888]

Nobody's being duped but you, bub.
Why bring this up again? A thinly veiled agenda have you?

Since Intel funded the Aberdeen Group research article you mention, none of their conclusions can be considered valid.

Perhaps you should conduct some of your own real world tests, or find some unbiased test results from the various tech sites on the net.

 

[Soulkeeper]

the model numbers are a success as far as i am concerned. They may not be a perfect representation of a particular Athlon's performance compared to a P4 at the correcponding clock speed, but they are close enough and professional enough to not officially be declared as such. Think about it if you saw a 1.67ghz athlon on a self and a 2.0ghz P4 and you didn't know anything about computer cpu's then you would prob get the 2ghz P4 now if you saw 2000+ and 2.0ghz you might think a little and possibly go with the cheaper one.
it was a fair and neccesarry step for amd to take in order to stay competitive.
one of the reasons people argue that the modeling system is inaccurate is because the Northwood doubled the P4's cache and uped the FSB to 533 which changes everything but nowhere does AMD state that the numbers are supposed to match the P4 and if they did then they would have to change the name of all their cpu's everytime Intel released a new core which isn't practicle

hope this clears things up
--Soul_keeper

 

[AgentSmith]

it's funny how a few years back and amd were pushing the k6-2s (yes those slow ass crappy chips) noone gave a hoot about those dodgy performance ratings.

 

[Duvie]

I pretty much would discount an intel founded research as well as bapco which many know also has ties to intel....haha...the games they play...

Also, the article is flawed as once again they make the mistake saying it is compared to p4 chips...AMD themselves and we need to take the word from them and their site that the ratings are based as per the tbird line of chips....


Besides it is no secret from fine eviewing sites such as anandtech that 2000+ is better then a 2.0ghz willamette, period....I own a p4 and yet I still can see clearly the truth. Wake up it doesn't appeared to be flawed at all...

 

[DivideBYZero]

I applaud Duvies objectivity.

 

[Swanny]

Originally posted by: lookin4dlz
I know this topic has been previously discussed in a long thread, but I don't know how much fact was in that thread. This research article (you have to register at the site before you can view the document) explores the topic more fully.

What is really kind of funny is that all of us on this board, who think of ourselves as experts compared to the average buyer, were duped as badly as the average buyer was. Even more interesting is the fact that many people continue to stand by AMD despite this fact. (disclosure statement: I was on the original AMD bandwagon as an Intel alternative when AMD first became competitive with Intel. A lot of processor errors & a chip failure later I moved to Intel)

Arberdeen Group research article

This White Paper focuses on Advanced Micro Devices (AMD) and details how the company is taking a step down a slippery slope of bad science. Specifically, AMD is naming its Athlon XP line of microprocessor models using clock-speed gigahertz ratings equivalent to Intel's competing Pentium 4 - based on a set of application benchmarks. Aberdeen's research suggests that AMD's processor equivalency methodology is seriously flawed: It assumes a specific application usage model that does not apply to many users; it is platform-specific, ignoring critical differences such as memory type; and it is inconsistent between mobile and desktop processors. Moreover, the methodology uses system-level benchmarks including Input/Output, an approach that is not used by the industry to measure processors alone.

====================
The following article isn't specificallly about the AMD situation, but does reference it as the quotes indicate.

Fast PCs: Maxed-Out Architectures?

Of course, AMD claims much more, providing performance charts at its site that show the Athlon XP clobbering Intel's CPUs. We have yet to see such a pummeling in our labs.

...the performance is not scaling as AMD claims it should.

Specifically, AMD is naming its Athlon XP line of microprocessor models using clock-speed gigahertz ratings equivalent to Intel's competing Pentium 4 - based on a set of application benchmarks.

Gee, they know what they're talking about. It's based on T-Bird performance, not P4.

 

[lookin4dlz]

DivideBYZero
- Too many errors to remember, lol. The chip just slowly faded away - I wasn't even an overclocker at that time & I re-used the components with my next system, so they weren't the cause.

omar888
- Sorry you don't have anything more substantial to add to the conversation, try again later.
- No agenda, just trying to get the facts because I don't remember there being anything factual in the previous discussion.
- See above as to why I brought this up. I just happened to see & read the Aberdeen white paper last night.
- There's no mention that Intel funded this research. Do you have a link to back that up? I emailed the author of the paper asking for disclosure as to whether the Intel did sponsor the research. I'll post his reply.

Soulkeeper
- I also think that this was a great marketing move by AMD. As I mentioned, I have no agenda here the best processor at the lowest cost is all that I cae about. However, I do like to get past marketing hype to the real story.

Duvie
- Thanks for pointing out that Intel is associated with BapCo (Business Applications Performance Corporation). I wonder why AMD didn't join? That way, proper benchmarks that are fair to both companies could have been developed.
- Actually, the article did not make a mistake. AMD clearly has positioned the XP against the Northwood processor You can go look through all of the press releases, pdf's etc. on AMD's site (I just did) and they are using the PR rating to pit their processor against Intel's. If you have a quote or a link by AMD that says otherwise (i.e. that PR is , it just proves the point that AMD hasn't been honest with its customers.

Here's the BapCo disclosure:

A non-profit consortium, BAPCo's charter is to develop and distribute a set of objective performance benchmarks based on popular/personal computer applications and industry standard operating systems. BAPCo?s current members include: Adaptec (www.adaptec.com); Amdahl Corporation (www.amdahl.com); Compaq (www.compaq.com) ; Dell (www.dell.com); Federal Computer Week (www.fcw.com); Hewlett-Packard (www.hp.com); IBM (www.ibm.com); InfoWorld (www.infoworld.com); Intel (www.intel.com); Microsoft (www.microsoft.com) ; NEC (www.nec.com); and VNU Business Publications Limited (UK) (www.vnu.co.uk). BAPCo is currently headquartered in Santa Clara, CA. For more information, please visit www.bapco.com.

 

[DivideBYZero]

Lookin4Dlz:

One word: FUD

You state your chip had 'more errors than you can remember', can you remember what CPU it was so we can check this reliability record out?

You also state it 'just faded away', nah, soz m8, that just doesn't happen, explain.

If you really don't think the XP is a competitive CPU to the PIV, then why are we even having this discussion? Everywhere you read nothing but PIV vs AXP, the final battle, the next level, etc. I'd like to see how many Bas A$$ CPU's Intel bring your way if AMD go down. Not many, I should coco.

You seem to think that the new ratings somehow relate to the old PR system. Well if that was the case then my Tbird scores some mad 2300 or more PR rating as stated in Sandra. Still think we are talking PR ratings?

This is the plain and simple truth, THE RATINGS ARE BASED ON THE SPEED REQUIRED BY A TBIRD TO ACHIVE THE EQUIVELANT PERFORMANCE OF AN XP. So, an XP2000+ could only be matched by a mythical TBird running at 2000MHz.

If you want a link, look up Quantispeed Architecture on the AMD web site, as this describes the above.

End of.

 

[UpliftMofo]

just check out this site and you'll see what truly matters among the processors . . . real world data

http://www.tomshardware.com/cpu/01q4/011031/index.html

 

[lookin4dlz]

I received a reply from the author of the paper, Peter Kastner. He was very nice, despite the insinuation that he was biased because of a covert association with Intel. Peter stated that he was completely objective & independent in his evaluation. He also said that there is additional research that backs up his findings.

Here's a link to an article where he publicly states there was no Intel influence:
Aberdeen analyst stands by AMD report: Says AMD cannot refute findings

"Intel, AMD and all the major chip makers are clients of ours," he said. "If that prohibits us from writing what we believe, it would prevent us from writing at all. The fact is that Intel had nothing to do with the report and didn't add one word."

He said one of the criticisms AMD had made earlier in the week, that AMD based its numbers on its Thunderbird 1.4 processor, was contradicted by the fact that in its own white paper it then went on to immediately compare their processors with Intel's.

Examples of misleading ads like this have already appeared in shops and serve to confuse consumers. "AMD says they're policing it but they're not policing it enough, he (Kastner) said"

 

[DivideBYZero]

Here is Mofo's link again:

Real World Comparision P4 vs AMD XP

Now tell me this, would an average Joe Shmoe really be that pissed if his 1800+ spanks his friends P4 2Ghz, but really runs at 1.53Ghz, nah, he'd just laugh at the mug who bought a 2Ghz PC that gets wipped by a 1.53Ghz PC.

Click

 

[lookin4dlz]

Dividebyzero
- I believe (now don't jump on me because I can't remember exactly, I'd have to wait until I got home to see the receipt) it was a K6-2, and yes it did just slowly die. It started with the errors while under warranty, but died after the warranty period.
- ooops, you made a mistake and posted a comparison of the XP to the Willamette P4...

Everyone
- quit trying to turn this into an AMD vs. Intel thread! I don't really care who makes the processor. The only reason I pointed out that I previously owned an AMD chip that failed was to be fair. If you'd like there are lots of other threads on that topic. This is more of an AMD vs the consumer thread.

 

[Priit]

lookin4dlz

I can't exactly understand what's your problem is. Sure, AXP's performance rating isn't (and never can be) very accurate, but it's more than good enough for average buyer, even when considering the latest P4/NW processors. When using similar components (same video card, memory etc), AXP's PR performance is about in pair with NW. Sure, you can choose tests to show that NW is faster, but it's also possible to show that 1Ghz T-bird is faster than 2Ghz Williamette when selecting benchmarks wisely.

About your dying K6-2: it's hard to me belive that processor just slowly "dies" (that's a feature of some MS OS'es . Processor's death is pretty fast process and in 99.9% of cases related to bad cooling. Sure, computer can be unstable when processor overheats, but it shouldn't be very hard for person with a little knowledge to find out the cause. BTW, I "fix" computers for living and I have a handful of dead Pentiums, one burned K6, melted P2/300, P3/550 and some other killed processors...

 

[omar888]

lookin4dlz, to say that you have no agenda is absurd. Insulting. It's right up there with, "I did not have sexual relations with that woman"

Here are some of your recent thread titles:

"AMD's Gigahertz Equivalency: Inexperienced Buyers Accept Bad Science - Aberdeen Group white paper"

"Intel 845G chipset motherboards are out!"

"Lots of 845E/845G based motherboard reviews - post your comments in here!"

"P4 2.53GHz out this quarter!"

Then there's this thread, "Celeron 1.7GHz $88 newegg - how high of an overclock?" lookin4dlz then starts in with the sales pitch:

"When teamed with an 845G motherboard, this is going to make a heck of an inexpensive and pretty powerful machine! "

AND

"...with the Celeron 1.7GHz oc'd to 2.26GHz and with using the new 845G chipset (built-in graphics) that reviews say is the fastest of the current DDR chipsets you can get a really nice system for hardly anything This would get your foot in the door with a system that could easily be upgraded. "

-----------------------------------------------

Here's lookin4dlz's comments in a thread entitled, "AMD .13ì Problems?" (suprise, suprise, he showed up in that thread)


"I just don't see that this new chip will work out, with the exception that the .13 process will lower costs (more per wafer than .25). Perhaps for the general consumer, computers based on AMD chips using the .13 core will cost less for a given clock speed (or PR or whatever). These buyers won't know/care about the heat, they'll just find that their machines are more noisy and maybe that will be enough to drive some sales away from AMD-based computers. However, for the enthusiast that overclocks it looks like Intel will be our future.

One thing that I've always wondered is whether AMD "overclocks" their chips vis-a-vis Intel chips as standard practice. What I mean is that the AMD chip, with the exception of a point in time when Intel was transitioning from high GHz PIII chips to PIV chips, doesn't seem to get as high of an overclock as easily as Intel. Additionally, the v-core for AMD's chips is higher which is an overclocking trick. This comment isn't to accuse AMD of anything, but probably the opposite - that is Intel probably sells their chips at a lower clock speed than they could to ensure there are no problems with the chips. "


How many shares of Intel do you own?

 

[Swanny]

One thing that I've always wondered is whether AMD "overclocks" their chips vis-a-vis Intel chips as standard practice. What I mean is that the AMD chip, with the exception of a point in time when Intel was transitioning from high GHz PIII chips to PIV chips, doesn't seem to get as high of an overclock as easily as Intel. Additionally, the v-core for AMD's chips is higher which is an overclocking trick.

Then explain how a 1Ghz T-Bird can get to 1.4 easily. Or even 1.6 if you got a good one? How do you explain the T-bred already going to 2.7Ghz?

 

[imgod2u]

This is the plain and simple truth, THE RATINGS ARE BASED ON THE SPEED REQUIRED BY A TBIRD TO ACHIVE THE EQUIVELANT PERFORMANCE OF AN XP. So, an XP2000+ could only be matched by a mythical TBird running at 2000MHz.

If you want a link, look up Quantispeed Architecture on the AMD web site, as this describes the above.

End of.

Ummm, can I say BS? A 1.67 GHz Athlon is no where near a 2 GHz t-bird. You've gotta be pretty diluded to think so. Let's see, a 2.0 GHz Northwood still has less IPC than a T-bird. So a 2.0 GHz t-bird would be significantly faster than a 2.0 GHz Northwood wouldn't it? Wow, I wonder why an AthlonXP 1.67 GHz isn't so significantly faster than a 2.0 Ghz Northwood then. *Gasp* I found a hole in the PR statement!

As for the original post, the PR system is merely a "rough estimate" for performance measurements. It is implied to be comparable roughly, and I will stress roughly, to a P4 (I suspect AMD meant willamette but also had enough forsight to see Northwood bringing about slightly better per clock performance). It is not meant as an exact comparison, it isn't even officially meant to be an exact comparison to the t-bird (although we all know that's bs). So to say there is a slight variation really isn't saying much. The bottom line is, it's marketing and really isn't that big of a deal. It helps sales and that's really all there is to it. Most knowledgable people know this and don't expect the PR 2000+ to perform exactly like a Willamette or Northwood 2.0 GHz. Although the more diluded among us will bite whatever AMD marketing (and in other cases, Intel marketing) throw out as truth despite how obviously false PR statements are (you'd be surprised how many people believe macs are truely twice as fast as any x86 machine).

 

[lookin4dlz]

Priit
- The point is that the PR system is not good enough for the average user if it's used to take advantage of them. The PR system is not on par with anything. It doesn't scale and even AMD has admitted its inaccuracies at higher processor speeds. Now they're coming out with a new rating system. Why not just join up with BapCo and get a fair system? I have absolutely no problem with AMD using the PR rating for comparisons with their own chips. It's a mis-representation to do so with other chips.
- I don't know how long the average chip takes to die. I've heard that some chips when overclocked experience "electron migration" (or something like that, but I'm no engineer) that causes a slow death, however mine wasn't overclocked...

omar888
- you need to go back a bit further in your research to see when, as an AMD fan, I was saying the same thing about Intel.
- let's see six of the eight posts you mentioned deal exclusively with Intel stuff because I'm looking to upgrade my system and am trying to make the best decision. And you have a problem with that - why? What point are you trying to make?
- this thread, as I've stated, is not an AMD vs Intel thread though all of you seem intent on making it one (have no idea why). I don't recall the last thread on this subject having factual references and so I posted one for discussion.
- in the other thread about AMD's move to .13 micron production did I say anything bad about AMD? Exactly what's your problem here?

Swanny
- All I know is that the overclocking databases I've seen show that AMD doesn't oc as well. Just for kicks, I pulled data from an overclocking database (about 1300 total entries) which shows the weighted average oc for AMD to be 23% vs 35% for Intel.

 

[Duvie]

It is obvious you haven't been around here long...or maybe in the computer market!!!! AMD's prior to the northwood debut were the undisputed kings of ocing. while tbird and duron owners were getting 300+ mhz ocs you would have been hard pressed to find anything like that in the intel area. P4 willamettes have been poor ocers while axia's debuted and many were taking 1ghz chips to 1.4 and above...recently new stepping had ppl buying 1600+ xp's and ocing to 1800+-1900+ at default voltage....

Those numbers are clearly based on the northwood frenzy of the recent past....not a clear picture....

 

[DivideBYZero]

*YAWN* (Again)

IamGod2U blubbed:

Ummm, can I say BS? A 1.67 GHz Athlon is no where near a 2 GHz t-bird. You've gotta be pretty diluded to think so. Let's see, a 2.0 GHz Northwood still has less IPC than a T-bird. So a 2.0 GHz t-bird would be significantly faster than a 2.0 GHz Northwood wouldn't it? Wow, I wonder why an AthlonXP 1.67 GHz isn't so significantly faster than a 2.0 Ghz Northwood then. *Gasp* I found a hole in the PR statement!

Seen a Tbird @ 2Ghz with standard 266FSB?
Got Benches of it vs an AXP?
I didn't think so...

Lookin4dlz fluffed:

ooops, you made a mistake and posted a comparison of the XP to the Willamette P4...

How so? Is this a discussion on the Northwood core? Did I post the link or simply repackage it for you as you seemed not to be able to comprehend the original link by 'mofo? Did it worry you that the willamette was edged out by an AMP XP? Could you not, just once, see beyond the Blue men, astro-bunnies and 5 note bing-bongs to see a world that many appreciate. I have been an Intel owner in the past, they are very nice processors. My preference for a system right now is AMD. This thread relates to the Aberdeen Group FUD, which is all most learned users will know it to be, not any core.

NW core is very good, it has given the PIV longer legs by keeping the long pipeline full after brach mispredictions, etc., but don't count out the AMD line as not being competitive. They are more than competitive and your argument is more down to taste and preference than actual fact.

AND pullleez stop going on about PR, that stood for (P)entium (R)ating AFAIK and IS totaly irrelevant in so much as it does not apply to XP CPU's. See my previous post

 

[DivideBYZero]

IG2U:

Most knowledgable people know this and don't expect the PR 2000+ to perform exactly like a Willamette or Northwood 2.0 GHz

ROTFLMAO

Did YOU click the earlier link I reposted to Toms Hardware? See how the Willamette stacks up then come back to eat the above words...

It doesn't perform 'like' a 2Ghz CPU it OUTperforms it.

 

[imgod2u]

IamGod2U blubbed:

Ummm, can I say BS? A 1.67 GHz Athlon is no where near a 2 GHz t-bird. You've gotta be pretty diluded to think so. Let's see, a 2.0 GHz Northwood still has less IPC than a T-bird. So a 2.0 GHz t-bird would be significantly faster than a 2.0 GHz Northwood wouldn't it? Wow, I wonder why an AthlonXP 1.67 GHz isn't so significantly faster than a 2.0 Ghz Northwood then. *Gasp* I found a hole in the PR statement!

Seen a Tbird @ 2Ghz with standard 266FSB?
Got Benches of it vs an AXP?
I didn't think so...

Ummm, do I need to? If processor A is significantly faster than processor B, and processor B is around the same as processor C, the obviously processor A is definitely faster than processor C. It would seem obvious but apparantly no. However, for the sake of arguement, I will pull some numbers:

Anand's benchmarks

Of 14 benchmarks, the 1.4 GHz t-bird is 8.36% slower than a 1.4 GHz Palomino. That's 8.36% improvement in average IPC among those benchmarks and that includes a 21.47% increase in Content Creation (SSE anyone?) while the others were:

10.64% increase in mpeg4 encoding
10.16% increase in video2000
6.15% increase in LAME mp3 encoding
4.60% increase in Q3A
3.36% increase in RTCW atdemo6
4.60% increase in RTCW atdemo8
2.25% increase in Max Payne
5.60% increase in Serious Sam
2.77% increase in SPECviewperf 6.1.2 - DRV-07
34.09% increase in SPECviewperf 6.1.2 - DX-06 (even the 1.5 GHz Willamette was 19.26% faster on this one and only this one)
and a 1.21% increase in SPECviewperf 6.1.2 - Light-04

Now, if we follow this trend, a 1.67 GHz Palomino should perform similar to a 1.81 GHz T-Bird. However, that's only IF these benchmarks were all CPU-dependent, which they are not. The actual increase would be much smaller (if you increase both CPU speeds, the performance gap lessens due to limitations of the other components). However, for the sake of arguement, let's assume that there are no other limitations. 1.67 would be similar to a 1.81 GHz t-bird, that's a far cry from a 2.0 GHz t-bird.

ROTFLMAO

Did YOU click the earlier link I reposted to Toms Hardware? See how the Willamette stacks up then come back to eat the above words...

It doesn't perform 'like' a 2Ghz CPU it OUTperforms it.

My point is, it's not suppose to be an exact representation. So saying the PR system is misleading based on the fact that an Athlon "2000+" performs slightly better OR worse than a P4 Willamette OR Northwood at 2.0 GHz is a failed arguement.

 

[DivideBYZero]

IG2U:

I was PM'd regarding this point we have been discussing, and you may be surprised to learn that I do agree that is is not exact, but to say it is misleading and 'Bad Science' does not take into account the fact that performance CANNOT be measured by MHz alone. Here is my reply to 'Wingnut PEZ'

yes, A 2000Mhz Tbird would be a pretty fast ship able to run out any AXP, but this statement:


Quote
(IMG2U)
--------------------------------------------------------------------------------
Most knowledgable people know this and don't expect the PR 2000+ to perform exactly like a Willamette or Northwood 2.0 GHz.
--------------------------------------------------------------------------------



...Is a bag of horse pucky and was the main target of my post. I reposted a link to the XP/Willamette comparison to rebuf this. I am sure you know yourself that the vastly differing solutions provided by AMD and Intel provide similar power output at different MHz ratings. I agree with IG2U that they are not 'acurate', but then if they were we would be looking at ugly numbers like 1948+ and 2077+. Marketing will always be skewed for the purpose of sales, none of us should ever believe otherwise.

For this forum and most of those who post in it, it is irrelevent as we all research our puchases to the N'th degree, studying benches and reviews till our eye's bleed. However, for regular joe's who see a computer as a commodity item, along side HiFi and DVD, they need to have an idea of power. Due to a discrepency in the MHz/Processing power that has emerged in AMD's architecture.

The issue here is that the processor markings ARE a good designator of performance for average Joe, and indeed more savvy users. Intel's lead in the MHz race has served them well, but don't be surprised when AMD come back with their own marketing. As I state above, Marketing is all it is and at no time do AMD not state the actual MHz speed of each processor incarnation. Also, if MHz are so important, then theaverage joe will ask about it anyway, and as long as the shop assistant is not a total Gimp, they should be told the real values.

Put the same average joe's in front of 5 rigs, 1GHz, 1.5Ghz 2.0Ghz and 2.5Ghz and I would be very surprised if, when he is surfing the web or writing a complaint letter to the Electricity company he could tell a jot of difference.

So tell me, where does the average Joe miss out with the speed marking system? Why is it so bad for AJ?