Whatever happened to OfficeBench in AnandTech's testing suites? Edit: Anand "investigating Sysmark issues"?

[ssanches]

The PDF is very interesting...and personally I find that Sysmark 2002 is definitely skewed in Intel's favor when compared to Sysmark 2001. I hope BAPCo has some good reasons for changing the weightage of programs in the net score....

I also find the way in which they have changed the filters used pretty disturbing. They have not only removed all of the filters that favored the Athlon and used only the ones where the P4 excelled, but they've also ran those filters multiple times to exaggerate the scores by a larger extent....

This issue with Sysmark 2002 is definitely another reason for me to take benchmarks with a grain of salt.

 

[Burmese]

Anybody see Tom's editorial? Seems he thinks that the data is too inconclusive to make a story about (and, of course, he knew about it before any other hardware sites and talked AMD out of going public with it). Also unloads on Van in a big way.

 

[DocM]

Toms been such an Intel lackey of late is that really a suprise?

Personally; I think Tom's the one who is losing his credibility.

DocM

 

[Double Trouble]

These are some interesting developments indeed.

Tom (of THW) has no credibility to me (he's shown himself to be a liar and cheat many times over the years), so I don't attach any value to his self-glorifying 'editorial'.

Van has always been an AMD zealot, so you usually have to take his writings with a grain of salt as well. It's interesting though that Van and Kyle are now going at it. Ah, as the world turns

I've always been very leary of the Sysmark benches because they always seem to put the Intel processors way further ahead than what my (admittedly non-scientific) personal usage experience would suggest. I guess with the AMD PDF it's become clear why that is.

AMD has been trying for a long time to enter BAPCo, and they've just now finally gotten in..... but I doubt they'll be able to do much since they are just one vote among the other companies, each with it's own agenda.

Sysmark should definitely be removed as a 'trusted' benchmark -- I for one do not trust it as a performance indicator.

Van's "reply" to Tom's editorial is pretty interesting as well. He's right on the money on Tom as a person! (as Harvey can attest to).

 

[MrMaster]

We have a war a brewing. It would be nice to see some sites go under after this bullshit. Everyone getting all personal and such.

Neither one deserves any respect or any hits on their sites for personally and publicaly attacking one another. They lost their professionalism. All of them.

 

[RegisPhilbin]

Here's one that seemingly cuts through the BS.

Overclockers.com Editorial

 

[keyeye]

I think he accurately discriminates between the two questions, but then decides he really doesn't want to deal with the second. The philosophy part I think is directed at the mudslinging....?
Q1. He clearly agrees that Sysmark is not a good benchmark for its state purpose. I don't think (almost) anyone who has read what is currently available would disagree (see Anand's comment above, etc).
Q2. Why is a bad benchmark? To me that is the only question left. Is it stupidity/bad benchmarking skills (I doubt that) or is it another case of corporate greed.
The evidence is compelling in AMDs presentation. Bapco does not provide clear methodology for its benchmarks (just abstractions). I can't see why the members of bapco, as far as I can tell, would be interested in the quality or fairness of the benchmarks. The only logical conclusion is that Bapco is really an extension of Intel PR, hidden behind a jumble of legal deniability.

This argument is made from the available data (which can change). I don't have the legal expertise to know whether this is sufficient to win a case. But look what happened when they dug through microsoft's file....

In this era of Microsoft, Enron, Worldcom, Qwest, you name it..... I for one, would like to see this resolved legally.
If it was the Intel/Bapco Sysmark, I'm ok with that = clear disclosure.
Fixing supposedly "independent" benchmarks to dupe the consumer is illegal, IMHO

 

[rck01]

To the forum faithful:

Just thought you would all be interested to know that we've made OfficeBench 3.01 and several other elements of our Benchmark Studio Framework available for FREE at our web site.

Simply surf over to www.csaresearch.com and sign-up for our FREE Performance Portal service. You'll gain access to a full-blown version of the OfficeBench 3.01 test script (the "Productivity Portal"), as well as our ADO Stress client/server database, MAPI Stress client/server workflow and WMP Stress multimedia workload simulators (the "Database," "Workflow" and "Multimedia" Portals, respectively).

It's our way of helping to fill the vacuum generated by all of the SysMark controversy, etc.

Enjoy!

Randall C. Kennedy