Nvidia/AMD/VIA leave Sysmark company Bapco

[Soleron]

http://www.marketwire.com/press-rel...e-sysmark-2012-benchmark-nyse-amd-1529559.htm
http://semiaccurate.com/2011/06/20/nvidia-amd-and-via-quit-bapco-over-sysmark-2012/


This would be fine if Sysmark scores weren't used as minimum requirements in a lot of government contracts, sometimes as the only performance requirement.

 

[ikachu]

http://www.marketwire.com/press-rel...e-sysmark-2012-benchmark-nyse-amd-1529559.htm

 

[AdamK47]

Has Intel payed off BAPCo?

 

[Idontcare]

Has Intel payed off BAPCo?

Wha!? Never...Intel is smarter than that.

Maybe they left some money on the nightstand...and if BAPCo just so happened to have picked it up on their way out of the hotel room then who is to say what was a favor among friends and what was a gift for nothing related to any services rendered...

 

[Phynaz]

AMD will only endorse benchmarks based on real-world computing models and software applications, and which provide useful and relevant information.

And then they will go right ahead and quote Spec numbers.

 

[Idontcare]

And then they will go right ahead and quote Spec numbers.

I thought SPEC was composed of actual apps and datasets. Sure the composite scoring system is synthetic, but the test itself is not. (as I understood it)

That said, there's only one reason why these companies are pulling out of Sysmark and that's because it fails to flatter their own product lineups.

There's no mystery here. Expect bulldozer/trinity to suck balls in 2012 sysmark benching. This is all so very dejavu...didn't AMD go through this cycle once before?

 

[Topweasel]

That said, there's only one reason why these companies are pulling out of Sysmark and that's because it fails to flatter their own product lineups.

True. But you wouldn't have every major micro and graphics manufacturer outside Intel leave it if it was just that simple. My guess is that they feel that the tests run are no longer representative of actual use and system performance. Along those lines sure it probably means that they don't fair as well as they feel their equipment should in a market that has shifted. But I don't think that right off the bat that this is a take my ball and go home ploy, its the start of a shift to find a better way to measure performance then these synthetic best/worst case scenario tests typically do.

For example with BD a lot of the conjecture on BD, is that its design lowers the theoretical IPC of each core but does a much better job then any other CPU in keeping its pipes filled and IPC near max. Which means that real world performance might be closer to the the synthetic performance, but all the typical synthetic benchmarks will shine on CPU's with higher theoretical maxes.

 

[Vesku]

Makes sense that NVIDIA and AMD would leave if Sysmark is actually taking a step backwards and barely including GPU performance or excluding it entirely. May explain VIA too, they've been making some leaps in their IGP performance. But could also stem around compiler choice or perhaps nothing in Sysmark takes advantage of their on chip encryption acceleration?

 

[Idontcare]

A non-profit consortium, BAPCo's charter is to develop and distribute a set of objective performance benchmarks based on popular computer applications and industry standard operating systems.

Q5. How does a company join BAPCo?
A5. Companies interested in joining the consortium are required to fill out an application, which will be reviewed by BAPCo's board of directors and pay an annual Full membership fee of $8,250.00 and a $10,000 initiation fee ($18,250.00 first year). For an Associate Membership the payable amount is $8,000.00 ($5,000.00 initiation fee and $3,000.00 first year dues).
Q6. Why would a company want to join the consortium?
A6. BAPCo was founded to foster the creation of meaningful benchmarks in an applications environment. BAPCo members receive released products at a cost rate. Also, BAPCo members have access to beta pre-releases before outside companies can purchase the released product. The BAPCo organization always promotes member companies in its literature, WWW site, and convention (exposure) show booths.

http://www.bapco.com/about/

The challenge here is that the losers of a benchmark are naturally going to gravitate to using red herring arguments to explain away the lackluster performance of their products.

Maybe AMD products really do just suck at office productivity stuff. Naturally they would disagree with that statement, whether the statement had merit or not.

So AMD logging their complaint over Bapco and sysmark is not proof of anything in and of itself other than that we ought not be surprised to see AMD chips not doing so well in sysmark next year (for legitimate reasons or not).

It would help their (AMD and the other protesters) case gain a little bit more credibility if they documented their grievances in a way that was verifiable by third-parties.

Otherwise this is just your usual damage-control and spinmeisters that comes with any product that falls short of being competitive.

Everyone who loses an election swears voter-fraud was involved. Its human nature to be sore losers and never take responsibility for our own downfalls.

Businesses are ran by people who have to justify their decisions as a means of self-preservation and job security. Naturally they are going to look for someone/something else to blame when the outcome of those decisions (bulldozer design makes a hard job for the marketing team) is something short of more dancing in the aisles.

 

[Nemesis 1]

My personnal opinion is this is more about compilers and what paths are choozen for other brands than intel . Why NV pumped into this is anyones guess . I seen this coming the moment Intel reached agreement with FTC on the compiler Issue . Intel got what they wanted AMD and the others got bit-- slapped for cring about intels software.

 

[piesquared]

The challenge here is that the losers of a benchmark are naturally going to gravitate to using red herring arguments to explain away the lackluster performance of their products.

Maybe AMD products really do just suck at office productivity stuff. Naturally they would disagree with that statement, whether the statement had merit or not.

So AMD logging their complaint over Bapco and sysmark is not proof of anything in and of itself other than that we ought not be surprised to see AMD chips not doing so well in sysmark next year (for legitimate reasons or not).

It would help their (AMD and the other protesters) case gain a little bit more credibility if they documented their grievances in a way that was verifiable by third-parties.

Otherwise this is just your usual damage-control and spinmeisters that comes with any product that falls short of being competitive.

Everyone who loses an election swears voter-fraud was involved. Its human nature to be sore losers and never take responsibility for our own downfalls.

Businesses are ran by people who have to justify their decisions as a means of self-preservation and job security. Naturally they are going to look for someone/something else to blame when the outcome of those decisions (bulldozer design makes a hard job for the marketing team) is something short of more dancing in the aisles.

And as a counter point, ^this is exactly the kind of damage-control and spinmeister statements that one would expect to come from the side of the biased benchmark in question.

Here's some more good reading for you:

http://blogs.amd.com/nigel-dessau/2011/06/21/1006/comment-page-1/#comment-603

and

http://www.newsweek.com/2009/06/18/hurry-up-and-type.html

I thought this part was particularily interesting:

AMD also points out that the president of BAPCo happens to be the head of performance benchmarking at Intel.

 

[Madcatatlas]

This really DOES render all Sysmark2012 benches as pure garbage.

Lets see if some joint venture from AMD/Nvidia/Via etc can come up with a separate and more belivable benchmark tool.

Im also thinking Sysmark (or Intel if they do own the bench) will stretch out a hand to amd/nvidia/via to try and regain some integrity, time will show.

@ IDC, whats with the witty first post and then subsequent "damage control"? :\

 

[Idontcare]

And as a counter point, ^this is exactly the kind of damage-control and spinmeister statements that one would expect to come from the side of the biased benchmark in question.

huh? How is pointing out the obvious - that Bapco collects fees from its members for a reason and that losers in elections decry voter fraud as a matter of human nature - are tantamount to damage-control and "spinning" of anything?

Your counter-point is to claim that anyone who acknowledges that people claim voter fraud is themselves doing so as a means to spin? This is bizarre.

 

[Idontcare]

@ IDC, whats with the witty first post and then subsequent "damage control"? :\

I thought it was obvious, I'm clearly on Intel's payrolls.

Except for that other 50% of the time when I'm posting the "witty" stuff - then I'm clearly on AMD's payrolls.

 

[Madcatatlas]

huh? How is pointing out the obvious - that Bapco collects fees from its members for a reason and that losers in elections decry voter fraud as a matter of human nature - are tantamount to damage-control and "spinning" of anything?

Your counter-point is to claim that anyone who acknowledges that people claim voter fraud is themselves doing so as a means to spin? This is bizarre.

I guess the point being made is that your post is almost entirely focused around the "amd/nvidia/via are "bad" losers who cant acknowledge that intel is the best".
To me it seems as though the "losers" as you put it, aren't being acknowledged at all and are making their point by resigning.

When you make 3 competitors of intel out to be whiny losers for leaving what is presumably an intel sponsored/owned group, especially when you consider the competition between those whiny losers themselves, your comments just don't ring right.

Why is Nvidia leaving? why is Via leaving? these are competitors of AMD themselves. Im dont quite understand how you can focus on such a "bad" piece of argument when it to me seems clear that the conflict lies not in "losing the fight" but in "rules for the fight".

Rule number 1, "there are no rules", obviously does not apply to what i believe has been a joint agreement between these companies.


edit: Edited out alot of off topic stuff. Sorry about that

 

[Mopetar]

The only problem I really have with the SYSmark benches is that their scores are disproportionate to the actual results which makes them misleading at worst, and utterly pointless for comparison in any case.

For example, if two CPUs complete the benchmark with times of 917 seconds and 1023 seconds, but the SYSmark scores will be something like 250 and 130, which makes the first CPU seem almost twice as powerful as the second, despite the fact that it is only around 10% faster.

 

[Vesku]

Reading what it takes to join BAPCo reminds me a bit of the ISO structure and how it was abused by Microsoft for getting their document structure approved. Since the purpose of Sysmark seems to primarily be about a metric that can be included in tenders from governments and other very large organizations I don't think Dell, HP and Lenovo have any reason to oppose a company structure and decision tree that ensure their primary CPU vendor scores quite well at it.

I've always wondered about how their excel measurements became primarily focused around sorting huge datasets. I can understand including that metric but not why it appears to be weighted so heavily other than to lend credence to AMD claims that it's weighting is based on it being one of the tasks Intel especially clobbers them at. Anyone know if they actually analyze work tasks performed by large organizations in an empirical manner?

 

[Topweasel]

huh? How is pointing out the obvious - that Bapco collects fees from its members for a reason and that losers in elections decry voter fraud as a matter of human nature - are tantamount to damage-control and "spinning" of anything?

Your counter-point is to claim that anyone who acknowledges that people claim voter fraud is themselves doing so as a means to spin? This is bizarre.

No you are right. But that is why there are always two sides to every story.

Sure AMD isn't going to be happy that their Products won't look as great in Sysmark.

Sure they want a benchmark that tests for the things that their CPU/Video cards do better.

But I guess the question is can we automatically assume that this is an attempt at crying Voter Fruad. Why would someone continue to partner with a company, pay money to invest in its manufacturing of the product and allow their company logo's and name be used to add legitimacy and promotion of the test if they don't feel its representative of what the benchmark is supposed to accomplish? Also what if they are right?

Its easy to wrap AMD into this complaint and we don't even know if this choice is because of the CPU end or the Video end. But they are not the only company making this choice. If it was just AMD that would be one thing. But three manufacturers left not just one. The one thing that usually separates the people yelling Voting Fraud have one thing in common they are the minority, that's why they lost. That doesn't seem to be the case, 3 of their biggest contributors say that they didn't have any influence in this partnership. Maybe its true this time and the minority stacked the deck. Now all that's left is Intel, and a bunch of system builders or media (reviewers). Strange that all of the non-Intel processing unit companies left. Now they have 1 component manufacturer outside Intel, in a hard drive manufacturer, well maybe two HDD guys if you count Samsung.

 

[atari030]

This really DOES render all Sysmark2012 benches as pure garbage.

Lets see if some joint venture from AMD/Nvidia/Via etc can come up with a separate and more belivable benchmark tool.

An agnostic tool like this already exists. It's called Linux.

 

[Phynaz]

Its easy to wrap AMD into this complaint and we don't even know if this choice is because of the CPU end or the Video end

In particular, SM2012 scores do not take into account GPU-accelerated applications that are widely used in today’s business environments.

I have no idea what GPU accelerated applications are widely used in business today.

Maybe AMD uses a different definition of the word "wide" than the rest of us, and they are applying the same definition as the "wide variety of workloads" that Barcelona was supposed to trounce Clovertown in.

 

[Vesku]

What about all those web videos you know the government employees must watch on their "lunch breaks"?

 

[Idontcare]

No you are right. But that is why there are always two sides to every story.

Sure AMD isn't going to be happy that their Products won't look as great in Sysmark.

Sure they want a benchmark that tests for the things that their CPU/Video cards do better.

But I guess the question is can we automatically assume that this is an attempt at crying Voter Fruad.

Actually I thought I had made it obvious that this was the point of my post - that there are two sides, two possible reasons for this one outcome, and the problem is that we can't make any conclusions in terms of deciding the "why".

Is Intel biasing Sysmark to favor Intel's products (voter fraud) or is Intel simply fielding the superior products and the other's are naturally going to lose out in the next benchmarking cycle?

Didn't I address both of these possible progenitors in my post?

We can't take the "loser" at their word on this because they have a duty to themselves to always cloud the issue when the outcome is not to their favor, ergo we need (and so to does AMD/Via/NV) some kind of third-party auditing done to verify the claims.

I would have thought this much was just stating the obvious. Who knew that stating the obvious could generate any controversy whatsoever LOL

At any rate, hasn't AMD already been through this cycle once before, crying foul over optimized benchmarks? You'd think they'd learn already, get ahead of the 8-ball and formulate their own variant of Bapco sysmark.

Hopefully this time we'll see them take that step.

 

[Vesku]

If AMD starts having some real financial success with it's heterogeneous strategy it is imperative they plow money to strengthen their in house software developers and external developer support. They've started to adopt NVIDIA marketing strategies against Intel so we may see their software side strengthen as well ala NVIDIA's TWIMTBP and similar. Not sure if they will be ready for Intel's pushback mid-late next year.

 

[Mopetar]

I have no idea what GPU accelerated applications are widely used in business today.

Web browsers are the most likely candidate, but I would not consider the applications that many companies run in a browser terribly taxing on system resources to begin with, so the point may be moot. Additionally several of these applications have left people tied to earlier versions of Internet Explorer.

However, as hardware becomes more capable of providing acceleration for certain tasks and software developers gain familiarity with the tools and frameworks to take advantage of that hardware, eventually GPU acceleration will come to business applications.

It may not be terribly important for most businesses at this time, but it will eventually move in that direction. For the time being, it will definitely be more useful in the consumer market as it allows end users to obtain levels of computational power that were previously beyond their price range.

 

[Gigantopithecus]

Good riddance. Tasks that computers run are increasingly diverse and rely on different components' respective abilities. I haven't even looked at SYSmark in years. I look at the benchmarks that more accurately reflect the user's specific needs. If they don't have specific needs (i.e. they are a general non-intensive internet/office user), I try to pitch an SSD and/or discrete GPU because the overall system usage experience is so much nicer than with a mechanical disk or integrated graphics.

CPUs just aren't as important as they used to be, at least not to the typical user.