SERIOUS FLAWS in THGs Turion 64 Inside Story pt2

[StriderGT]

I post this message because there seems to be GIGANTIC confusion regarding the real performance and battery usage levels of the AMD Turion vs the Centrino platform.
In THGs article the MSI M635 model (Turion MT34 1.8Ghz 1MB Cache, ATI RX480M/SB400 m/b chipset, ATI X700 128MB) scores ONLY 43FPS in Doom3 which is more than 100% below the Centrino based Gigabyte W511A notebook with the same graphics subsytem! 1MB more cache CAN never yield such differences!
I run Doom3 timedemo1 at my ACER 5024 (Turion ML34 1.8Ghz 1MB Cache, ATI RS480/SB400 m/b chipset) with the same ATI X700 128MB which scored 67FPS@640x480 HQ and 75FPS@MED Qual. (Vsync OFF) nearly 35% more!!!

It is more than obvious that THGs comparison is NOT VALID since both the performance and obviously power usage will vary tremendously (+-100%!!!) between different notebook implementations -if no serious benchmarking fault is involved in this case- not giving us a clue about the real potential of the Turion or the Centrino platform...

Further proof of this disturbing trend of inconsistent conclusions can be gained by these two comparisons:

Clash of the Titans: Dothan vs Turion

MSI S270 AMD Turion 64 Mobile Technology (link courtesy of Accord99)

that reach completely opposite conclusions and even vastly different than THGs.

So this is an open invitation to all hardware sites and anandtech to do a real platform comparison of Turion and Centrino notebooks including the best or even better a statistically significant portion of the notebook implementetions of these two platforms.

PS My setup
Acer 5024
Default frequencies ATI X700 128MB (347/330Mhz)
Driver version: 6.14.10.6525 (factory default)

 

[Markfw]

I thought the same thing when I read their article, but didn;t notice the big benchmark variation, I just knew something wasn't right.

 

[Duvie]

Not a surprise from THG....they usually see what they want to see...and have a nasty habit of neutering one system....I think after the debacle of the X2 versus P4 XE stress testing review most ppl dont even mention that name aournd here..especially anywhere around words like credible, respectable review sites.....

I think the top sites are

TechReport
Anandtech

 

[Bona Fide]

I posted the Laptop Logic article a while back...they seem pretty reputable.

@Duvie: I would put Guru3D up there also.

 

[Duvie]

Actually it look qute celar like he F'd it up as usal...How do you take DDr266 and beat ddr400 at same speed...Bandwidth will make a huge difference in these games....It is clear there is an external bottle neck here...the sad thing is he is too stupid to see it...

 

[StriderGT]

I agree that the X2 versus P4 XE stress test conclusions were completeley misguiding and much of that test methodology -especially the usage scenario modelling- was very flawed.
But the Centrino vs Turion comparison seems to be ill covered by all hardware sites so far...

 

[MercenaryForHire]

So in conclusion, Tom is full of bullsh!t.

Wanna tell us something we didn't already know?

- M4H

 

[Duvie]

The problem is finding similar enough platforms to rule out other factors...If we are doing a laptop review, fine...then you could use that sh^tty one up there from the MSI 270 review...

BUT....If we are doing a cpu review bent on looking at the performance of the cpu and its unique architecture then you need apls and apples on the rest of the platform....

 

[StriderGT]

Originally posted by: Duvie
The problem is finding similar enough platforms to rule out other factors...If we are doing a laptop review, fine...then you could use that sh^tty one up there from the MSI 270 review...

BUT....If we are doing a cpu review bent on looking at the performance of the cpu and its unique architecture then you need apls and apples on the rest of the platform....

I agree that is why I insist there should be a comparison between the best -or even better with a lot- of similar configurations of turion and centrino.

THG tried for example to level the GPU freqs which is correct, the problem is they should have brought many -not just one- Turion laptops from Acer,Compaq,DELL, etc with similar configs (=same graphics, memory, screen) but even different chipsests ATI, SIS, etc since you are comparing platforms not procs and then see the general trend of how they perform. It is obvious that you cannot reach desktop level apples to apples comparison but it would be close enough...

 

[Aenslead]

So what's new?

Nothing is. We all knew THG wasn't liable at all... not a single friggin bit.

Old news. Its been more than 6 months since I last visited them.

 

[KoolDrew]

Not a surprise from THG

QFT.

 

[StriderGT]

My intention was not to bash THG, but to show that there is no "real world" thorough performance comparative analysis between Turion and Centrino.

There are other articles over at THG that are quite informative, so there is no point to level everything.

EG The fact that I do not agree with the need of high grade memory in the A64 platform does not make anandtech, who suggests the opposite, a bad site. But I will agree that lately THG seems to falter a bit in the unbiased conclusion making process

 

[bersl2]

Originally posted by: StriderGT

Originally posted by: Duvie
The problem is finding similar enough platforms to rule out other factors...If we are doing a laptop review, fine...then you could use that sh^tty one up there from the MSI 270 review...

BUT....If we are doing a cpu review bent on looking at the performance of the cpu and its unique architecture then you need apls and apples on the rest of the platform....

I agree that is why I insist there should be a comparison between the best -or even better with a lot- of similar configurations of turion and centrino.

THG tried for example to level the GPU freqs which is correct, the problem is they should have brought many -not just one- Turion laptops from Acer,Compaq,DELL, etc with similar configs (=same graphics, memory, screen) but even different chipsests ATI, SIS, etc since you are comparing platforms not procs and then see the general trend of how they perform. It is obvious that you cannot reach desktop level apples to apples comparison but it would be close enough...

Part of the problem with benchmarks is that to do them scientifically, you need many of every part, all from different batches, then average the results out. Unfortunately, this is expensive to do. Which is why all benchmarks still need varying amounts of salt taken with them.

 

[Shenkoa]

I think THG works for Intel imo.

 

[StriderGT]

Originally posted by: bersl2

Part of the problem with benchmarks is that to do them scientifically, you need many of every part, all from different batches, then average the results out. Unfortunately, this is expensive to do. Which is why all benchmarks still need varying amounts of salt taken with them.

The issue here is not about achieving 100% statistically correct benchmarking results, including different samples, batches etc. since the cost, the time, the effort and even the significance of the improvement will not be worthwhile.
But to investigate the performance of a platform eg Turion with ONLY ONE representative -the msi notebook- is simply wrong.

 

[imported_NinjaGnat]

Anybody else notice the HUGE graphics memory descrepancies between the two systems? The Turion they had was running at 297 DDR3 while the Pentium-M used 344.25? Whaaa? Still, this most likely doesn't explain both why Doom 3 ignored memory speeds, OR why the Doom chart is at 640x480 while they said previously in the article that it was being tested at 1024x768. Or them jumping to the conclusion that the M's chache is magically providing a near linear increase over the turion. Something is clearly wrong here...

 

[StriderGT]

Originally posted by: NinjaGnat
Anybody else notice the HUGE graphics memory descrepancies between the two systems? The Turion they had was running at 297 DDR3 while the Pentium-M used 344.25? Whaaa? Still, this most likely doesn't explain both why Doom 3 ignored memory speeds, OR why the Doom chart is at 640x480 while they said previously in the article that it was being tested at 1024x768. Or them jumping to the conclusion that the M's chache is magically providing a near linear increase over the turion. Something is clearly wrong here...

I think they leveled both the gpu/mem freqs by ocing the msi notebook gpu.

I am also amazed how can such an obvious mistake/inconsistency go unnoticed all this time. I e-mailed THG with my findings but no response came from them. I also e-mailed anand with the whole issue asking for a more thorough Turion vs Centrino platform comparison along with my suggestions. I did not see any reaction from any other hardware site either -If someone did find something please correct me-. Its seems that either nobody bothers looking at graphs or everybody takes their results for granted!?

 

[Alaa]

here is the respond from anandtech in the last paragraph under Playing with Flash Drives september 6th : http://www.anandtech.com/weblog/default.aspx#245

 

[Gamingphreek]

Tom himself is actually very smart. However, it is well known that he can be easily "bought" to write reviews slanted towards one company. Additionally, while Tom himself is smart (May not have the common sense to know that biased reviews are wrong) the people under him are complete morons most of the time.

I wouldn't pay his review anymind. THe only think that is semi-worthwhile from that site are the Video Card roundups.

-Kevin

 

[StriderGT]

Originally posted by: Alaa
here is the respond from anandtech in the last paragraph under Playing with Flash Drives september 6th : http://www.anandtech.com/weblog/default.aspx#245

thx Alaa. Seems that we must wait for a wider availiability of Turion platforms, but THG like articles could damage the reputation of an otherwise excellent product... I am very happy with my ACER 5024 (except maybe with the TFT LCD coating method: too many reflections and narrow angles for my taste)

 

[T2k]

Frankly, who cares? Does anybody still check, let alone reading, Tom's Garbage?

I thought only young padawans, newbies and outsiders...

 

[StriderGT]

Originally posted by: T2k
Frankly, who cares? Does anybody still check, let alone reading, Tom's Garbage?

I thought only young padawans, newbies and outsiders...

Well I did, that's how I found out the descrepancies...
I've been following both anand and thg among many other hardware sites for more than 5 years now and even longer (but I am starting to forget) so I guess I am no newbie and as I've found hidden treasures buried here and there you should keep an open mind; read, compare then judge and even then keep a doubt, it will come handy later... My 2 cents and may the force be with u.

 

[carlosd]

I Knew there were something wrong about the Doom3 test in the turion laptops, I have seen already the laptop logic review before tom's. The I reviewed the LL article again and I confirned my supects. THG sucks.

 

[StriderGT]

Still no answer ftom THG. How does someone know that they even got/read the e-mail?
Actually it was a contact form i filled in and sent them my findings, I don't know perhaps I should have adressed the form to the author of the article.

 

[StriderGT]

I've addressed the contact form message to one of the authors of the article, hoping that someone will answer this time. Further proof of something seriously wrong with the Doom3 benchmarks is that my machine scored 40FPS in 1024x768 HQ nearly the same as the MSI almost identically configured notebook scored in 640x480 HQ!!! Could it be that they made the very simple mistake of benching the Centrino and Turion laptop in different resolutions (640 vs 1024 as NinjaGnat also hinted)? But even if so -ok such mistakes can happen-, they went on to draw conclusions on such bizarre numbers?!?!

PS Irony: the article is in the hot article list of THGs. Ok but all these people who read it don't they get the impression that something is wrong over there???