Quantifying price/performance

[TerryMathews]

I have this theory that those of us who got in on the L5639 chips and cheapish X58 motherboards broke the price performance curve pretty significantly.

I want to quantify it though, and do it in a way the community agrees with. I also want to use this thread as a sort of compilation, where people can submit their prices and performance and we can keep a running tab of the best values.

First off, I think we need to measure only CPU-bound tasks and it should be a test that is comparable and fair between AMD and Intel. We have that Handbrake benchmark thread going already. Do we all agree that is a good test?


If so, then I'm at:
264 FPS / ($108.99+$85) = 1.37 FPS/$

If everyone agrees this is a rational metric, please contribute. Mobo and proc prices only please, let's treat RAM, HSFs, etc as fixed and separate costs. The reason for that is this benchmark is only measuring CPU performance. Other factors such as SSD/GPU wouldn't play into Handbrake performance in any significant manner. If we wanted to quantify total system performance over cost we would need a more comprehensive test like 3dmark or Crysis3 or BF3.

I've also heard at least one person ask for single threaded performance. We can't grab that from the Handbrake thread directly but I know it can be done single threaded.

Regarding pricing, I would think current FMV would be more important than the price you paid for the kit if it's old.

 

[TerryMathews]

Rough draft of chart and raw data

Please feel free to add in your own entries. This is using the big buck bunny Handbrake benchmark, once single threaded and once multi threaded. "Component cost" is FMV your CPU + motherboard.

9/17/13: Added charts for pure single-threaded and pure multi-threaded performance. The data is there, might as well graph it. Plus, now I can troll the Handbrake thread for results...

 

[sefsefsefsef]

Mobo and proc prices only please, let's treat RAM, HSFs, etc as fixed and separate costs.

I don't understand the rationale for doing this. These are significant contributions to the overall cost of ownership, and ignoring them will greatly reduce the value of what you're trying to do.

 

[KingFatty]

Better also establish how you are arriving at the prices?

Part of this surely would depend on when you buy your equipment, and how far along the price depreciation curve you are when you actually make your purchase? I mean someone buying the same mobo and CPU may pay an entirely different price so their calculation would be way off.

If anything, you should determine if you want to do this with standardized prices somehow, or just illustrate who got killer price deals when they purchased their equipment?

 

[DominionSeraph]

My X4 945 and AM2 board were free.
I win.

 

[Yuriman]

I'm not sure what kind of meaning can be extracted from this.

Are we taking current market value, or what we paid when we got our chip? My Athlon64 back in 2003 would have a terrible priceerformance today, but I bought it 10 years ago.

Anywho, approx. $210 + $130 = $330 1.2 years ago

234FPS / $330 = 0.71 FPS/$

Or if we're talking what I could get my CPU/board off ebay right now, we're looking at approximately $255, which gives 0.92 FPS/$.

My total time is 89% of yours with 4 threads vs 12, so with all threads loaded I'm at approximately 2.67x your performance per thread. That's certainly not a fair comparison since 6 of your threads are virtual, but .89 / (4/6) = 1.34x an Ivy Bridge single core vs a hyperthreaded Nehalem core. Borrowing from this post, we can guestimate that hyperthreading will give approximately 25% more FPS when enabled so 1.35 * 1.25 = 1.69x single threaded performance. So, in applications that can effectively use 4 threads or less, an Ivy Bridge combo off ebay would have (1.69 x 0.92 = 1.55, 1.55 / 1.37 = 1.14) so approximately 14% better single threaded performance per dollar and provide close to 70% more performance.

So really, it depends on what you're trying to do as to what processors have the best value. I'd definitely take your Xeon if I intended to do lots of encoding, but it doesn't have the same value if you're looking to play games.

 

[crashtech]

Maybe there should be one fully threaded and one single threaded metric. Otherwise, OP, you will not escape the shadow of confirmation bias.

UPDATED to accepted version:

2700K @ 4.9GHz + Z77X-UD3H = $380

Handbrake 0.9.9.530 x64 (Big Buck Bunny/Android):

fully threaded: 267.397949 FPS

single threaded: 74.182213 FPS

Full log files.

Single core score was with Hyperthreading still enabled, and no affinity settings. Only mod was per Yuriman's suggestion, here.

Added decimal points, I'll leave the rounding to the OP. This time ran both tests at the same speed for consistency. The single threaded test will run at nearly 5GHz on my machine, but the multi will not. I feel my 2700K is not the best example of the breed, but it will have to do.

I bought the 2700K used on ebay for $240, and the Z77-UD3H on Newegg for $139.00. I think there are cheaper boards that could have gotten this job done, but I'll leave the additional calculations to the OP.

Happy to participate and assist if desired.

 

[BallaTheFeared]

322 / $401 = 0.80 FPS/$

23% faster than an x6 = priceless

 

[Idontcare]

I don't understand the rationale for doing this. These are significant contributions to the overall cost of ownership, and ignoring them will greatly reduce the value of what you're trying to do.

Yeah I agree here as well. While it likely won't alter the pecking order itself, it will certainly serve to flatten the spectrum (and rightly so) such that you don't end up with inappropriately wielded statistics.

 

[BrightCandle]

Would be useful for you to at least publish the details of the benchmark you are using in this thread if you want more people to contribute.

The problem with this measure is partly time based as others have described, there is no adjustment for current value involved. More importantly the TOC should probably be part of such an analysis and not just the initial cost. Even with all this the slower processors tend to be a lot cheaper for their performance so such a study ultimately results in telling us what we already know. If Performance per dollar is the goal then buying the cheapest motherboard along with the cheapest CPU does very well. Performance/price always favours the lower binned processors that are sold in greater volumes. But it ignores the fact it might not be performing very well at all, it ignores for example the very real cost of having to wait considerably longer for a process that on another CPU may well take a lot less time.

IMO its a measure that is only useful in comparison with others, like when comparing similarly performing parts.

 

[TerryMathews]

Yeah I agree here as well. While it likely won't alter the pecking order itself, it will certainly serve to flatten the spectrum (and rightly so) such that you don't end up with inappropriately wielded statistics.

It's a CPU benchmark. What difference would it make if someone paid $50 for a HD6450 or $700 for a Titan? Both would have identical performance running the Handbrake benchmark.

First post edited with a link to the Handbrake benchmark thread. Sorry, my phone wasn't cooperating with grabbing the URL today.

Would be useful for you to at least publish the details of the benchmark you are using in this thread if you want more people to contribute.

The problem with this measure is partly time based as others have described, there is no adjustment for current value involved. More importantly the TOC should probably be part of such an analysis and not just the initial cost. Even with all this the slower processors tend to be a lot cheaper for their performance so such a study ultimately results in telling us what we already know. If Performance per dollar is the goal then buying the cheapest motherboard along with the cheapest CPU does very well. Performance/price always favours the lower binned processors that are sold in greater volumes. But it ignores the fact it might not be performing very well at all, it ignores for example the very real cost of having to wait considerably longer for a process that on another CPU may well take a lot less time.

IMO its a measure that is only useful in comparison with others, like when comparing similarly performing parts.

This echos the point made by several posters, and I'm certainly not blind to it. Maybe the correct answer is multiple performance tiers, sorted by price? Like <100FPS and then additional tiers each of +50FPS or whatever people think is a significant performance increase?

 

[T_Yamamoto]

My X4 945 and AM2 board were free.
I win.

My computer was free.

I win

 

[crashtech]

My computer was free.

I win

No, your rig is undefined by our metric. You have to lie and say it cost a penny in order to win.

 

[BallaTheFeared]

I don't work so all my PC cost me was my soul.

322 / soul = ???

 

[Kallogan]

Welcome on dull thread

 

[TerryMathews]

I don't work so all my PC cost me was my soul.

322 / soul = ???

Well your soul is either worthless which would drive your value towards infinity, or priceless which would drive your value towards 0.

 

[BallaTheFeared]

Only time can answer this question...

 

[YBS1]

It's certainly a nice performance metric, but like mentioned above it's going to tell us what we already know. The cheapest processors are always going to win at this. 39x0k's simply dominated this benchmark, but fail hard in price/performance. Conversely, how would you quantify the performance of say my w3520 in that chart? Based on the price I paid for it, it would fair very poorly, however it's within spitting distance of all but the tip top performers on that chart, and I've had that level of performance for four years. Same goes for the Gulftowns on there, to a more exaggerated degree in both directions.

 

[Enigmoid]

Well your soul is either worthless which would drive your value towards infinity, or priceless which would drive your value towards 0.

 

[TerryMathews]

Would it be better if it were restructured as "what can I buy it for today"?

 

[crashtech]

I presume this is all mostly for fun, but there should be a perceived level of fairness for it to stay that way. Staying close to the current going rate, and averaging single and fully threaded performance might put this little game into a place where more posters feel like participating.

 

[YBS1]

might put this little game into a place where more posters feel like participating.

I thought one of the more interesting little "benchmark" threads that have popped up here as of late was the how fast can you go on 1.0V thread. I have been busy the past week or so though so I didn't get involved in it, and it didn't really seem to grab people's attentions and got buried off the front page pretty quick.

Since Anandtech seems to have picked up using that 3D Particle Movement benchmark in it's cpu tests lately it would be amusing for us to run that just to see how long it would be before a stock clocked retail processor would match the top numbers.

 

[Zucker2k]

Say you give 5 people a blank check to build the least cost to performance ratio system running the handbrake benchmark. This simply means what you can BUY TODAY.

 

[Idontcare]

It's a CPU benchmark. What difference would it make if someone paid $50 for a HD6450 or $700 for a Titan? Both would have identical performance running the Handbrake benchmark.

That isn't what I was getting at, but my wording lends itself to easy misinterpretation so the fault is mine.

It is the price/performance comparisons that will be skewed, and numerical analyses of those comparisons that will become silly. That was all I was trying to speak to.

 

[BallaTheFeared]

There is also the purpose for which you built your system and thus why you picked the parts you picked.


I know I didn't build my system to run Handbrake more than a few times a month.