Originally posted by: dullard
Originally posted by: desy
What people used to just accept now is called a 'serious problem'
Something CR never defines, so its subjective, in fact what earns you a black circle in CR may just be a 'feeling' of a reviewer.
Serious problem has always been defined. I suspect you simply don't like their definition. I've filled out the survey forms for years. It says that normal maintanence or normal wear-and-tear are not serious problems. Then it says that a serious problem is
considered serious because of cost, failure, safety, or downtime. If a problem (not regular wear-and-tear) costs a lot of money, completely failed, caused actual safety problems, or required a lot of time to fix, it is a serious problem. You could argue that a Lexus owner's definition of "costs a lot of money" differs from a Kia's owner. But you can't argue that it isn't defined at all. Then the problems are weighted so that important problems (for example engine failure) are given more weight than unimportant problems (for example the automatic windows don't work). You could argue that the weighting is subjective, but you didn't.
As for the black circles, Consumer Reports bases the ratings on objective tests. They use repeatable and measureable tests (Gas mileage, crash test results, turning radius, acceleration, braking distance, usable space, etc). Subjective things like a car's "look" or "styling" is not considered at all. The result is an ugly horibly styled vehicle that performs well gets good ratings in CR. This is the part that many people hate. They want beautiful, well-shaped vehicles to score well instead.
The subjective part is how each of these measurables are weighted. Sure, opinion is discussed in the text, but Consumer Reports frequently discusses opinions in the text that aren't included in the ratings. But it all really boils down to the numbers. Compare JD Powers to Consumer Reports. The results are quite similar. I'll post 2003 model JD Powers 3-year study numbers and 2001 model Consumer Reports 5-year study numbers on all 30 makes that CR includes in their test. I use these because I have them available, but no they aren't the exact same comparison. I'm sorry I don't have the exact same comparison easilly available.
I hand typed these in from my best interpolation of the picture data. I may be off by a very slight bit (my eyes may have been poor on the CR graph). The data is scaled by dividing the problems reported by the average problems of all models. This helps put JD and CR on the same basis. These are termed "Scaled" in the table below. Then to show differences, I sorted each and gave them ranks. Due to Excel sorting, if vehicle brands had ties in the report, I may have inavertently swapped them, but they are still tied. Finally, I took the CR Scaled number and divided by the JD Scaled number to give an overall difference in the reports. For example, both gave Lexus the 1st place marks. But CR had only 55.8% of the serious problems compared to JD's total problems.
Make ( JD Place , JD Scaled ) ( CR Place , CR Scaled ) CR/JD
Acura ( 6 , 81.1% ) ( 3 , 52.9% ) 65.3%
Audi ( 26 , 122.9% ) ( 16 , 110.1% ) 89.5%
BMW ( 8 , 93.4% ) ( 18 , 111.4% ) 119.3%
Buick ( 3 , 67.4% ) ( 10 , 85.0% ) 126.1%
Cadillac ( 4 , 71.8% ) ( 29 , 147.7% ) 205.6%
Chevrolet ( 18 , 106.2% ) ( 22 , 115.6% ) 108.9%
Chrysler ( 14 , 102.2% ) ( 21 , 115.6% ) 113.1%
Dodge ( 22 , 113.7% ) ( 24 , 116.3% ) 102.3%
Ford ( 11 , 98.7% ) ( 11 , 97.5% ) 98.8%
GMC ( 16 , 105.3% ) ( 23 , 116.3% ) 110.5%
Honda ( 7 , 85.5% ) ( 4 , 55.7% ) 65.2%
Hyundai ( 21 , 111.5% ) ( 17 , 110.7% ) 99.4%
Infiniti ( 9 , 94.7% ) ( 5 , 61.3% ) 64.7%
Isuzu ( 27 , 124.7% ) ( 12 , 100.3% ) 80.5%
Jeep ( 24 , 116.3% ) ( 20 , 114.2% ) 98.2%
Lexus ( 1 , 59.9% ) ( 1 , 33.4% ) 55.8%
Lincoln ( 10 , 96.9% ) ( 15 , 105.9% ) 109.2%
Mazda ( 20 , 107.0% ) ( 7 , 72.4% ) 67.7%
Mercedes-Benz ( 17 , 105.7% ) ( 30 , 149.1% ) 141.0%
Mercury ( 2 , 66.5% ) ( 14 , 103.1% ) 155.0%
Mitsubishi ( 23 , 114.5% ) ( 8 , 80.8% ) 70.5%
Nissan ( 19 , 106.6% ) ( 9 , 82.2% ) 77.1%
Oldsmobile ( 12 , 98.7% ) ( 27 , 139.3% ) 141.2%
Pontiac ( 15 , 102.2% ) ( 26 , 135.1% ) 132.2%
Saab ( 30 , 143.6% ) ( 19 , 111.4% ) 77.6%
Saturn ( 28 , 127.3% ) ( 13 , 101.7% ) 79.9%
Subaru ( 13 , 102.2% ) ( 6 , 65.5% ) 64.1%
Toyota ( 5 , 78.9% ) ( 2 , 43.2% ) 54.8%
Volkswagen ( 29 , 131.7% ) ( 28 , 140.7% ) 106.8%
Volvo ( 25 , 119.8% ) ( 25 , 125.4% ) 104.6%
The important number is in the last column. Note how similar the ratio is. If the ratio is 100%, then both studies had the exact same reliability. Half of all makes were within 20% of each other. Half of all makes were placed within just a few spots in both reports. There are notable differences.
[*]Cadillac, Mercedes, Mercury, Oldsmobile, and Pontiac scored much worse in CR than JD. They scored worse in % and in Placement.
[*]Lexus, Toyota, Acura, and Honda scored better in % in CR but were placed about the same as JD.
[*]Isuzu, Subaru, and Mazda scored better in % in CR and were placed better in CR.
But overall, those numbers are quite close. Look at how many of the CR/JD ratio numbers are near 100% meaning they had the exact same reliablity scores in both tests. That result similarity is quite remarkable if one is real and the other is "biased" and wrong.