Quantifying price/performance

[TerryMathews]

That isn't what I was getting at, but my wording lends itself to easy misinterpretation so the fault is mine.

It is the price/performance comparisons that will be skewed, and numerical analyses of those comparisons that will become silly. That was all I was trying to speak to.

Fair enough. So what sort of TCO should we use as a scalar? $400?

There is also the purpose for which you built your system and thus why you picked the parts you picked.


I know I didn't build my system to run Handbrake more than a few times a month.

In my OP I asked what benchmark we should use. Suggest one.

To all of you hung up on some of the things I suggested in OP, if you reread I asked for other opinions on all of it. Suggest something. I suggested Handbrake because we can grab results for 20+ systems from there and price them via Newegg etc.

If we can all come together on a fair measurement for performance there are some things I know we can do that will be cool with the data.

 

[crashtech]

In my OP I asked what benchmark we should use. Suggest one.

To all of you hung up on some of the things I suggested in OP, if you reread I asked for other opinions on all of it. Suggest something. I suggested Handbrake because we can grab results for 20+ systems from there and price them via Newegg etc.

If we can all come together on a fair measurement for performance there are some things I know we can do that will be cool with the data.

Well, there should be a single thread component, but which bench to use and how to incorporate it into the results I can't say right now. If there was an easy and consistent way to run Handbrake on one core, that would be an option, but I don't think everyone would have that capability, nor perhaps the knowledge to execute such a test accurately.

 

[Yuriman]

Well, there should be a single thread component, but which bench to use and how to incorporate it into the results I can't say right now. If there was an easy and consistent way to run Handbrake on one core, that would be an option, but I don't think everyone would have that capability, nor perhaps the knowledge to execute such a test accurately.

Would not setting affinity work? Perhaps we could have a Handbrake 2 threads and a Handbrake all threads priceerformance.

 

[TerryMathews]

Well, there should be a single thread component, but which bench to use and how to incorporate it into the results I can't say right now. If there was an easy and consistent way to run Handbrake on one core, that would be an option, but I don't think everyone would have that capability, nor perhaps the knowledge to execute such a test accurately.

Handbrake has an option for number of threads. It normally auto detects but you can explicitly set it.

 

[crashtech]

Well there you go.

 

[Yuriman]

I loaded "Android" profile, and under the Video tab, checked "Use Advanced Tab instead". Here I added ":threads=1" to the end of x264 Encoder Options.

4.6ghz, 1600mhz DDR3 @ 9-9-9-24

1 thread: 77.032906 fps
4 threads: 231.179199 fps

At current market value, my Ivy returns:

0.30 FPS / $ single threaded performance
0.91 FPS / $ multithreaded performance

Together they average to be ~61 FPS / $

Taking just my CPU,

0.47 FPS / $ single threaded performance
1.40 FPS / $ multithreaded performance

Together they average to be 0.94 FPS / $

 

[crashtech]

Is that the cost of the CPU only? The OP was including CPU+mobo. I'll run the single thread version when I get home tonight.

 

[BallaTheFeared]

I think we should leave the mobo out, because some of us could have gotten $80 boards and achieved the exact same CPU results.

 

[crashtech]

I agree, now what does the OP say?

 

[BallaTheFeared]

i5-4670k (4.8GHz) @ $200

work: average encoding speed for job is 104.940605 fps

work: average encoding speed for job is 323.048004 fps

104.94/200

323.04/200

0.525 fps per $ single thread

1.615 fps per $ quad thread

 

[crashtech]

Current lowest BIN price on ebay for 4670K: $230.00 Newegg: $239.99

 

[BallaTheFeared]

Isn't newegg doing $60 off with select boards/ram and $10 off the proc with a code?

Or would it be cheating to say it was $170? >.<

 

[crashtech]

My take is this should not be personal, so taking out subjective elements like smart shopping might be indicated, finding a fair market value of the CPU instead

A more favorable and perhaps more accurate metric might be the avg. price of the last 10 that sold on ebay: $208

 

[ElFenix]

I'd really prefer we use 2 pass for benching. My 2 cents.

 

[TerryMathews]

I think we should leave the mobo out, because some of us could have gotten $80 boards and achieved the exact same CPU results.

I agree, now what does the OP say?

My only thought against that is that certain processors are available cheaply but require an expensive motherboard to extract the full performance potential.

We were concerned with skewing results. I think leaving the motherboard out will do that in cases where it is more expensive than the processor itself.

 

[crashtech]

My only thought against that is that certain processors are available cheaply but require an expensive motherboard to extract the full performance potential.

We were concerned with skewing results. I think leaving the motherboard out will do that in cases where it is more expensive than the processor itself.

OK, so what do you think of the 2 pass (Cinebench?) suggestion, and/or adding a single core Handbrake result? Obviously, this erases some of the advantage your Xeon setup might enjoy, but may be more representative of actual use scenarios.

 

[TerryMathews]

OK, so what do you think of the 2 pass (Cinebench?) suggestion, and/or adding a single core Handbrake result? Obviously, this erases some of the advantage your Xeon setup might enjoy, but may be more representative of actual use scenarios.

Absolutely. I was never hung up on my specific suggestion. I only threw it out there because we could mine the Handbrake results thread.

What would be fairer to everyone Cinebench or running Handbrake single threaded? And if we do Handbrake, should we average the results or run a separate chart? My gut is second chart.

When this is done, I want to plot the price performance ratio on an excel graph and highlight the outliers.

 

[Idontcare]

Fair enough. So what sort of TCO should we use as a scalar? $400?

I think that would be right. From there the efficiency comparisons will be all the more realistic in terms of accentuating those processors which enable your entire computer investment to deliver superior ROI versus those processors which do little to add value to your total computing investment.

 

[crashtech]

Absolutely. I was never hung up on my specific suggestion. I only threw it out there because we could mine the Handbrake results thread.

What would be fairer to everyone Cinebench or running Handbrake single threaded? And if we do Handbrake, should we average the results or run a separate chart? My gut is second chart.

When this is done, I want to plot the price performance ratio on an excel graph and highlight the outliers.

I'm agnostic on which bench to use, but would like to see some kind of combined result, whether or not scores are also charted individually, because combined load types represent most of our real-world experiences.

 

[TerryMathews]

I'm agnostic on which bench to use, but would like to see some kind of combined result, whether or not scores are also charted individually, because combined load types represent most of our real-world experiences.

How should we weight it? 50/50?

 

[crashtech]

How should we weight it? 50/50?

I would, but it's your baby!

 

[TerryMathews]

I would, but it's your baby!

Ok, so tomorrow I'll run single threaded on mine.

We'll run three charts: ST, MT and simple average of those two.

We are going to price based on FMV of processor and a reasonable board to capture the performance level indicated plus $400 to account for TCO.

Did I miss anything?

 

[BallaTheFeared]

Shouldn't that be left to the user/reader to decide?

For me, performance past 4 cores at this point in time is a moot point. My only concern is what my cpu can get done with two, and to a lesser extent the performance of four (since two lends to four it's not the major factor).

To someone else, it could be totally different. They might not care what one or four cores can do but what the CPU as a whole is capable of.

To each their own!

 

[TerryMathews]

Shouldn't that be left to the user/reader to decide?

For me, performance past 4 cores at this point in time is a moot point. My only concern is what my cpu can get done with two, and to a lesser extent the performance of four (since two lends to four it's not the major factor).

To someone else, it could be totally different. They might not care what one or four cores can do but what the CPU as a whole is capable of.

To each their own!

Of course it is your choice. No one is forcing you to participate...

If you have a different idea I'm open to it. Your complaint though seems to be that performance isn't quantifiable as different users have different needs. While true, I think capturing ST and MT performance will encapsulate the needs of 95% of users and that other 5% with special needs should already know how to evaluate the data provided to estimate results for their scenario.

I don't think it's realistic to ask our participants to run the benchmark in every possible permutation. For the hex core users, that would be 12 times.

 

[BallaTheFeared]

I already participated, twice in fact! And it's not really a complaint, more like a suggestion.

My suggestion was that raw data should be presented, not weighted based on a scale and given a "total score". But it's not my thread, not my idea, I'm only giving suggestions as you requested.

I will participate regardless