Truth or Myth?: Is SYSmark a Reliable Benchmark?

[sontin]

I absolutely agree, but AMD did offer up evidence to prove their claims. Though, since we can't see it in detail, you can claim it's rather weak. Still though, I don't see any counter arguments here.

Is SYSmark indicative of real world performance?
Is SYSmark only optimized for Intel?

There is no proof in the video. AMD just used another benchmark.

Different software will produce different results.

 

[Lil'John]

I absolutely agree, but AMD did offer up evidence to prove their claims. Though, since we can't see it in detail, you can claim it's rather weak. Still though, I don't see any counter arguments here.

Is SYSmark indicative of real world performance?
Is SYSmark only optimized for Intel?

Thank you for non-ranting retort... kind of refreshing

I don't know what AMD has offered up as evidence to support their claim.

For the first, in looking at the quote, is it supposed to be "real world use"? Obviously, the benchmark exists in the real world and is providing something showing performance. Whether someone really is going to do the task(s) that were benchmarked is another story

For the second, again, I don't know the proof being provided.

In my opinion, the person who is making claims is the person on the hook to provide the bulk of the proof. Otherwise, everyone could run around saying "I won 1 million dollars. Prove I didn't."

 

[nvgpu]

http://vrworld.com/2011/06/24/amd-i...-is-an-excuse-for-poor-bulldozer-performance/

AMD tried to discredit SYSmark before during Bulldozer years and is trying the same tactic again, what a sad, pathetic, uncompetitive company.

It's pretty obvious Zen is Bulldozer 2.0 when it comes to CPU performance.

 

[Ma_Deuce]

It's pretty obvious Zen is Bulldozer 2.0 when it comes to CPU performance.

What makes it obvious?

 

[mrmt]

What makes it obvious?

AMD starting to beat this dead horse again?

 

[Ma_Deuce]

AMD starting to beat this dead horse again?

So if I got this straight. It's far fetched that a benchmark might not be a true indicator of real world performance, but the fact that AMD releases a video is an obvious indicator or future CPU performance?

You guys are too funny sometimes.

This i3 I am typing this on should be fine for general office use if I went by the benchmarks, unfortunately real world scenario it's painfully sluggish. Now if it was my responsibility to run benchmarks all day though, I would be set!

 

[Phynaz]

AMD starting to beat this dead horse again?

Yep.

God, AMD needs a competent marketing team. This is what happens when you promote secretaries to marketing.

 

[TheELF]

So if I got this straight. It's far fetched that a benchmark might not be a true indicator of real world performance, but the fact that AMD releases a video is an obvious indicator or future CPU performance?

You guys are too funny sometimes.

This i3 I am typing this on should be fine for general office use if I went by the benchmarks, unfortunately real world scenario it's painfully sluggish. Now if it was my responsibility to run benchmarks all day though, I would be set!

In deed, there is no benchmark that can factor in the incapabilities of a user,every benchmark assumes a taken care of system and an user actually capable of properly using a computer.

 

[SimianR]

I feel like AMD needs to stop worrying about synthetic benchmarks and to me it's a bad strategy to highlight your weaknesses from the get go - they should be focusing on their strengths and not worrying about Sysmark. "We perform poorly in synthetic A, so it's not valid but we perform well in synthetic B - you can trust that one."

 

[mrmt]

This i3 I am typing this on should be fine for general office use if I went by the benchmarks, unfortunately real world scenario it's painfully sluggish. Now if it was my responsibility to run benchmarks all day though, I would be set!

So you think a Bulldozer APU would solve your problems?

 

[mrmt]

I feel like AMD needs to stop worrying about synthetic benchmarks and to me it's a bad strategy to highlight your weaknesses from the get go - they should be focusing on their strengths and not worrying about Sysmark. "We perform poorly in synthetic A, so it's not valid but we perform well in synthetic B - you can trust that one."

Synthetic benchmarks have their place, they stress a given scenario in order to find glass jaws in a given product. They may or may not be analogous to some specific real life scenario. For example, the kind of advantage AMD is claiming that is unfair compared to Intel processors would be very real if ST applications were preponderant on the scenario but would not if there was a selection of MT applications.

What I find interesting is that AMD does not address why their chips perform so poorly in the SYSmark scenario or how it should be improved in order to better reflect real workloads, but instead is calling the benchmark biased and unfair without making a case. It's more like a smear campaign than a complain.

 

[Sweepr]

This i3 I am typing this on should be fine for general office use if I went by the benchmarks, unfortunately real world scenario it's painfully sluggish. Now if it was my responsibility to run benchmarks all day though, I would be set!

Can you provide specific details about your system, now I'm curious.
I used Core i3s in the past and even Celeron/Pentium and none of them felt painfully sluggish. Actually 24/7 browsing and light office use is 99% what my Haswell-E delivers. All of them had an SSD though.

 

[The Stilt]

Office Productivity:

Adobe® Acrobat Pro XI
Google® Chrome® 32.0
Microsoft® Excel® 2013
Microsoft® OneNote® 2013
Microsoft® Outlook® 2013
Microsoft® PowerPoint® 2013
Microsoft® Word® 2013
WinZip Pro 17.5

Media Creation:

Adobe® Photoshop® CS6 Extended
Adobe® Premiere® Pro CS6
Trimble® SketchUp™ Pro 2013

Data / Financial Analysis

Microsoft® Excel® 2013
WinZip Pro 17.5

These do not represent real world workloads? :hmm:

 

[zlatan]

Making a benchmark is really hard, because it needs to be fair. But what's the fair practice? There should be an optimized codepath for every hardware, or there should be one codepath? Hard question. Even that a company try to be fair, it isn't easy. There are a lot of question flying around now, that is it fair to write a standard multi-engine code for D3D12, when it won't run well on Maxwell/Kepler/Gen9? Or there should be a standard path for AMD, and an optional path for the others? Or the code should use a single queue? Hard questions. For a program there should be two or three codepaths, but for a benchmark ... Fairness wants the same path, but that should be a standard path or a path that won't kill the performance on the mostly used hardwares?

 

[ShintaiDK]

Office Productivity:

Adobe® Acrobat Pro XI
Google® Chrome® 32.0
Microsoft® Excel® 2013
Microsoft® OneNote® 2013
Microsoft® Outlook® 2013
Microsoft® PowerPoint® 2013
Microsoft® Word® 2013
WinZip Pro 17.5

Media Creation:

Adobe® Photoshop® CS6 Extended
Adobe® Premiere® Pro CS6
Trimble® SketchUp™ Pro 2013

Data / Financial Analysis

Microsoft® Excel® 2013
WinZip Pro 17.5

These do not represent real world workloads? :hmm:

Everyone uses the Lotus Notes package

I had to lookup Trimble, but else the rest fits perfectly. Chrome could be Firefox/IE11/Edge etc as well.

 

[sontin]

Office Productivity:

Adobe® Acrobat Pro XI
Google® Chrome® 32.0
Microsoft® Excel® 2013
Microsoft® OneNote® 2013
Microsoft® Outlook® 2013
Microsoft® PowerPoint® 2013
Microsoft® Word® 2013
WinZip Pro 17.5

Media Creation:

Adobe® Photoshop® CS6 Extended
Adobe® Premiere® Pro CS6
Trimble® SketchUp™ Pro 2013

Data / Financial Analysis

Microsoft® Excel® 2013
WinZip Pro 17.5

These do not represent real world workloads? :hmm:

Here is the whitepaper: https://bapco.com/wp-content/uploads/2015/09/SYSmark2014Whitepaper_1.0.pdf

So, Sysmark is just a benchmark suite for business user. Makes AMD video just worse...

 

[deasd]

Old news......the same as below.
[SA] Nvidia, AMD, and VIA quit BAPCO over SYSmark 2012

http://www.overclock.net/t/1046815/sa-nvidia-amd-and-via-quit-bapco-over-sysmark-2012

BAPCO has turned into a bad joke, so bad that Nvidia (NASDAQ:NVDA), AMD (NYSE:AMD), and VIA (2388:Taiwan) just quit. Yes, that leaves Intel (NASDAQ:INTC) as the only semiconductor maker still in the consortium, and there is a good reason for this.

The short story is this, BAPCO makes the SYSmark series of benchmarks, and several others. SYSmark in particular has been long seen by everyone but Intel as a joke, it is so skewed that it is laughable, it really doesn’t measure anything but Intel or non-Intel CPUs. With the release of SYSmark 2012, things got even worse.

 

[ShintaiDK]

Nvidia quit because they dont have anything in it after they stopped with chipsets.

BAPCo members include: Acer, ARCIntuition, ChinaByte, CNET, Compal, Dell, Hewlett-Packard, Hitachi, Intel, LC Future Center, Lenovo, Microsoft, Western Digital, Wistron, Quanta Computer, Samsung, Sony, Toshiba, Zol and others.

AMD quit with the release of their Bulldozer junk and their first APU flop.

AMD's complaints seem tied to the introduction of its new Llano platform which is admittedly slower in x86 than comparable Intel chips, but far faster in graphics performance.

 

[guskline]

I really hope Zen is a first class cpu and puts AMD back into the CPU race. However the timing of this video sure makes me wonder whether or not that is going to happen.

If Zen is really a big jump, why aren't we seeing videos about the future of Zen instead of attack videos?

 

[Burpo]

Old news......the same as below.
[SA] Nvidia, AMD, and VIA quit BAPCO over SYSmark 2012

"Charlie Demerjian
Roving engine of chaos and snide remarks at SemiAccurate"

 

[zlatan]

Office Productivity:

Adobe® Acrobat Pro XI
Google® Chrome® 32.0
Microsoft® Excel® 2013
Microsoft® OneNote® 2013
Microsoft® Outlook® 2013
Microsoft® PowerPoint® 2013
Microsoft® Word® 2013
WinZip Pro 17.5

Media Creation:

Adobe® Photoshop® CS6 Extended
Adobe® Premiere® Pro CS6
Trimble® SketchUp™ Pro 2013

Data / Financial Analysis

Microsoft® Excel® 2013
WinZip Pro 17.5

These do not represent real world workloads? :hmm:

From a technical standpoint these apps are old. And because of this the performance is not as good as the newer versions or the competition.
For exapmle I have an xls file for my stock trading business, and it uses a lot of data. I use it to see short-term fluctuations and highlight longer-term trends. In Excel 2013 the calculation takes ~40 seconds, but in Libreoffice Calc 5 the same job is done under ~30 seconds on the same CPU or ~4 seconds if I accelerate it with a mid-range GPU.
The same applies to Photoshop and Premiere Pro. The newest versions are much faster on the same CPU or with GPU acceleration. These GPGPU options also runs well on Intel iGPUs.

 

[mrmt]

From a technical standpoint these apps are old. And because of this the performance is not as good as the newer versions or the competition.
For exapmle I have an xls file for my stock trading business, and it uses a lot of data. I use it to see short-term fluctuations and highlight longer-term trends. In Excel 2013 the calculation takes ~40 seconds, but in Libreoffice Calc 5 the same job is done under ~30 seconds on the same CPU or ~4 seconds if I accelerate it with a mid-range GPU.
The same applies to Photoshop and Premiere Pro. The newest versions are much faster on the same CPU or with GPU acceleration. These GPGPU options also runs well on Intel iGPUs.

Nobody cares about libreoffice in corporate environments.

 

[Vesku]

Wonder if this is a premptive move to get Sysmark removed before Zen comes out and possibly flops on it too.

Tin foil hatting, why would AMD care this far into Bulldozer to even try this angle?

I don't know if it's still the case but US government equipment contracts often referenced Sysmark, I'd guess as minimum requirements or to compare bids. It was mentioned at the time AMD withdrew from BAPCO.

 

[USER8000]

HMM:

http://www.xtremesystems.org/forums...y-in-Tablets&p=5150663&viewfull=1#post5150663

The new Intel multimedia benchmark used by AnandTech: TouchXPRT 2013 is another brainchild
of dr Who's boss Shervin Kheradpir, General Manager of Intel's Performance Benchmarking and Analysis
Group and founding President of Bapco (via HDXPRT/Principled Technologies)

I think I will just keep to just getting a general consensus for performance reading reviews,and not benchmarks which have involvement from hardware companies. Its why I wouldn't buy a graphics card based on 3DMark scores for example!

 

[ShintaiDK]

HMM:

http://www.xtremesystems.org/forums...y-in-Tablets&p=5150663&viewfull=1#post5150663



I think I will just keep to just getting a general consensus for performance reading reviews,and not benchmarks which have involvement from hardware companies. Its why I wouldn't buy a graphics card based on 3DMark scores for example!

Good to see you find impartial forum posters.

/sarcasm