Car Comparisons: What can reviewers do better?

[fstime]

I remember reading a car magazine (Road & Track, Automobile, Car and Driver, etc) and in the reader comments section there was an interesting question:

Why aren't car comparisons done with all cars running the same brand and model of tire (obviously size is to spec)?

As i'm sure many of you know if you have tried many different types of tires is that they can make a HUGE difference, especially if your car's OEM tires are shit like they often are (runflats, etc).

The response from the magazine was totally off IMO: "Do you think we should put the same engines in the cars too?"

This response was totally off, tires are a wear/tear item which have to be replaced sometimes within 12,000 miles in terms of summer performance tires while an engine is meant to last a very long time and is much more expensive and inconvenient to replace.

 

[slashbinslashbash]

The car magazines test the cars as they come from the factory to show what a new owner can expect from a new car off the lot. If they mod the cars in any way, then that ideal is negated. There is a whole spectrum of things that can be changed to make a car "perform better" in some way. Tires and air intakes are pretty "easy"/minor. So is a "chip tune". Exhaust systems are a little more complex, but usually a pretty simple and quick swap. A turbocharger is another level up. Switching camshafts is another. And so on until you get to full engine replacement. Where do you draw the line?

The OEM tires are OEM tires for a reason. If a car manufacturer wants their car to do better in reviews, then they should put better OEM tires on it.

The answer from the magazine is exactly what is needed. Pithy and to the point. I spelled it out a bit lengthier here, but it conveys the exact same sentiment.

 

[CraigRT]

Cars should be tested as they come from the factory, as already stated. It's nice knowing the potential of a car, testing it on a different set of tires, but most people want to know how it performs right out of the box. Enthusiasts don't care as much, but they are a much smaller number of the general public.

 

[jlee]

The car magazines test the cars as they come from the factory to show what a new owner can expect from a new car off the lot. If they mod the cars in any way, then that ideal is negated. There is a whole spectrum of things that can be changed to make a car "perform better" in some way. Tires and air intakes are pretty "easy"/minor. So is a "chip tune". Exhaust systems are a little more complex, but usually a pretty simple and quick swap. A turbocharger is another level up. Switching camshafts is another. And so on until you get to full engine replacement. Where do you draw the line?

The OEM tires are OEM tires for a reason. If a car manufacturer wants their car to do better in reviews, then they should put better OEM tires on it.

The answer from the magazine is exactly what is needed. Pithy and to the point. I spelled it out a bit lengthier here, but it conveys the exact same sentiment.

Tires are a wear item. Nothing else you listed is, so I can see the argument. For regular cars, I wouldn't see it as meaning much - but when looking for the "Best enthusiast sports coupe", I think it'd be a good idea to run the same tires since those people are probably replacing the stock tires sooner than later anyway.

 

[Viper GTS]

Tires are a wear item. Nothing else you listed is, so I can see the argument. For regular cars, I wouldn't see it as meaning much - but when looking for the "Best enthusiast sports coupe", I think it'd be a good idea to run the same tires since those people are probably replacing the stock tires sooner than later anyway.

I'm not sure the majority of the car buying population fully understands the difference tires can make, or is willing to shop/pay for up-market replacements. Sadly I would include even people buying high end cars in that.

There are a frightening number of Vipers on the market, for instance, with the original 10+ year old Pilot Sports still installed. You would think that anybody owning a car like that would replace tires after a few years ("worn out" or not), but I have a set of spare wheels with new looking tires that say otherwise.

The non-enthusiast's tire evaluation is something like this:

What is this going to cost me, and don't you have anything cheaper?

Viper GTS

 

[jlee]

I'm not sure the majority of the car buying population fully understands the difference tires can make, or is willing to shop/pay for up-market replacements. Sadly I would include even people buying high end cars in that.

There are a frightening number of Vipers on the market, for instance, with the original 10+ year old Pilot Sports still installed. You would think that anybody owning a car like that would replace tires after a few years ("worn out" or not), but I have a set of spare wheels with new looking tires that say otherwise.

The non-enthusiast's tire evaluation is something like this:

What is this going to cost me, and don't you have anything cheaper?

Viper GTS

Yeah.

 

[exar333]

Tires are a wear item. Nothing else you listed is, so I can see the argument. For regular cars, I wouldn't see it as meaning much - but when looking for the "Best enthusiast sports coupe", I think it'd be a good idea to run the same tires since those people are probably replacing the stock tires sooner than later anyway.

This makes a lot of sense. If you are comparing performance-oriented cars, it makes sense to have equivalent tires. If a car costs $5000 less, and comes with cheap tires, but has similar performance with better tires, it could be a great buy. Even if you dropped the OEM tires immediately and bought a new pair it still would come out cheaper.

 

[Ferzerp]

This is a tech site, so let me use a computer analogy.

It would be like reviewing pre-built PCs, but swapping out the included HD for an SSD (hey, it's a cheap upgrade that can make a huge difference!), or replacing the video card, etc.

They review the product you can buy.

 

[jlee]

This is a tech site, so let me use a computer analogy.

It would be like reviewing pre-built PCs, but swapping out the included HD for an SSD (hey, it's a cheap upgrade that can make a huge difference!), or replacing the video card, etc.

They review the product you can buy.

But when benchmarking video cards, don't they run them on the same system?

For your typical appliance-mobile, it makes perfect sense to run it like it came off the lot. However, that's not what I was referring to. Tires are a wear item. They're going to get replaced at some point - and it's not going to void any part of your warranty like engine or suspension modifications could.

 

[Ferzerp]

But when benchmarking video cards, don't they run them on the same system?

That is not analogous at all. That's a single component test. The car version of that would be testing various brake systems on the same car to see which brakes are best, not comparing two cars to one another.

 

[T2urtle]

the current articles are like best car under $25,000 and etc. Or best $30k sedan.

To spin off the OP's idea i would like best $30k new, stock or modded. I'll make up numbers completely by say, Audi S4 is $30k and you compare it to a WRX which is $25k and you get $5k in mods that you MUST buy off the shelf. Say SPT suspension and a Cobb Stage 1 and downpipe. Now you compare that. Its a whole lot of varibles like where to get your parts and tune, but that could open up the market for companies like TRD, Nismo, SPT,honda racing to step up their game in making a tune chip like cobb's accessport or swaybar upgrades. I know BMW 335's have factory tunes now or something close to it. This way dealerships can make more money after intital sale of car. They could be conservative and offer warranty. TRD offered superchargers for the camry/solara installed at a dealership. Given it was like a $10k option but still now that is a comparable bases if another company did the same.

The currently comparsion is good because stock for stock is what 90% of the people drive and care about. Most dont care about what they MAKE the car do but what the can already do.

 

[AMCRambler]

I stopped reading car reviews after a buddy of mine at work showed me this one.

http://www.caranddriver.com/compari...-ford-mustang-gt-comparison-tests?redirect=no

They disregarded the fact that the gto far outperformed the mustang in every category and claimed the mustang won for it's looks. Give me a break. They also basically admit they said as much so they could validate their top ten list of sports coupes which the gto didn't make. Car and driver lost all of my respect at that point. Picking a car based on looks is something my wide would do for chrissakes. Retards all of them. The comments to the article say it all.

 

[Insomniator]

I just read car reviews for fun, does anyone actually base their purchasing decisions on them?

I don't the mini cooper I just bought placed high in any comparo tests I've seen...

 

[Pariah]

Common sense should dictate that the comparison criteria be adjusted depending on what type of vehicles are being compared and who the target audience for them is. Unfortunately that common sense is conspicuously absent from these publications.

I don't remember which magazine it was, but someone here linked to a comparison for minivans here a while back. In order to determine who the winner was, they used the same scoring sheet they would use for a comparison of supercars. So things like passenger comfort, ammenities, storage capacity and driver comfort were worth less combined than either slalom performance or quarter mile times. That's completely asinine, and really makes you wonder what total morons work for these publications and don't realize the idiocy of something like that.

 

[exar333]

I stopped reading car reviews after a buddy of mine at work showed me this one.

http://www.caranddriver.com/compari...-ford-mustang-gt-comparison-tests?redirect=no

They disregarded the fact that the gto far outperformed the mustang in every category and claimed the mustang won for it's looks. Give me a break. They also basically admit they said as much so they could validate their top ten list of sports coupes which the gto didn't make. Car and driver lost all of my respect at that point. Picking a car based on looks is something my wide would do for chrissakes. Retards all of them. The comments to the article say it all.

What are talking about? The 'X factor' if perfectly acceptable to use in an objective comparison.

 

[Zap]

But when benchmarking video cards, don't they run them on the same system?

Yes, but what if one card runs hotter than the other? Shouldn't they then normalize the heatsink/fan on the card so that they are just testing the card and not the cooling? What about normalizing the clock speeds?

The problem with car reviewers normalizing the tires is that you then buy a car off the lot and it performs differently than what you read in a review.

Hmmm, I guess that's why people in the Memory and Storage forum obsessively/compulsively benchmark their SSD and complain all the time about not getting the exact numbers that they expect. :hmm:

 

[Sulaco]

Fire the entire staff of Car & Driver, for a start...

 

[jlee]

For regular cars, I wouldn't see it as meaning much - but when looking for the "Best enthusiast sports coupe", I think it'd be a good idea to run the same tires since those people are probably replacing the stock tires sooner than later anyway.

Yes, but what if one card runs hotter than the other? Shouldn't they then normalize the heatsink/fan on the card so that they are just testing the card and not the cooling? What about normalizing the clock speeds?

The problem with car reviewers normalizing the tires is that you then buy a car off the lot and it performs differently than what you read in a review.

Hmmm, I guess that's why people in the Memory and Storage forum obsessively/compulsively benchmark their SSD and complain all the time about not getting the exact numbers that they expect. :hmm:

For enthusiast reviews, don't they do stuff like that? Overclocking/etc?

 

[Zap]

For enthusiast reviews, don't they do stuff like that? Overclocking/etc?

Depends. If it is a bare CPU, then yes reviewers might normalize cooling. If it is a graphics card that has cooling attached at the factory, I have yet to see a review that replaced it (unless the review was on the cooler)

I have another thought about putting the same tires on cars.

It won't work. Different cars come with different sized tires/wheels. Even if you use the same brand/model, different width or aspect ratios will make them perform different.

 

[jlee]

Depends. If it is a bare CPU, then yes reviewers might normalize cooling. If it is a graphics card that has cooling attached at the factory, I have yet to see a review that replaced it (unless the review was on the cooler)

I have another thought about putting the same tires on cars.

It won't work. Different cars come with different sized tires/wheels. Even if you use the same brand/model, different width or aspect ratios will make them perform different.

Not exactly, sure...but if one car ships with all seasons and another with ultra high performance summer tires, it'd at least make for a reasonable comparison.

 

[BassBomb]

even if you did choose same tires, oem tire sizes don't always match up... so then what?

I would say test it as it comes from the factory, and then if you want to look at potential, test all cars with the same tire (using oem size) but better tire