• We’re currently investigating an issue related to the forum theme and styling that is impacting page layout and visual formatting. The problem has been identified, and we are actively working on a resolution. There is no impact to user data or functionality, this is strictly a front-end display issue. We’ll post an update once the fix has been deployed. Thanks for your patience while we get this sorted.

The most ridiculous review... ever? You decide?

Ilmater

Diamond Member
Alright, I wrote in one other time, and I believed that a certain review that I read was ridiculous. I realize that, while it was ridiculous, it was mostly a subjective feeling, rather than a fact.

However, I read a review today that I believe to not only be horribly incompetent, but flat out misleading. The review is from MBReview.com, and the specific review I'm speaking about is this one.

Now, this review is truly incredible. First is some general information about DDR and how it transmits data on the top and bottom of each clock cycle, which is why a stick running at say 200MHz is running at and effective rate of 400MHz DDR. We'll refer back to "true" MHz versus "effective" MHz in a minute.

The next thing he mentions is that he couldn't get any stick to above 410 MHz DDR. This will also be important in a minute.

Now we have some Sandra benchmarks, and obviously enough, all the sticks perform exactly the same within a negligible statistical variance. He states that, "at the 166MHz test speed, the XtremeDDR PC-3500+ comes out the winner for integer performance, while the Corsair RAM takes the floating-point performance." What in the hell is he smoking? The scores for the two tests and the two sticks are 2040 vs. 2037 and 1944 vs. 1950, respectively. How in the hell can he claim a winner in that battle?

The next two pages worth of tests are far better (it's so hard to type sarcasm). If you don't see this for yourself, you won't believe it. The first page is a couple of Quake 3 benchmarks that have frames per second out to the tenth of a frame! Oh, but that's not all. Now we go to the next page with UT2003 benches. These are taken out to the thousandth of a frame! The largest variation of any one stick to another on this page is half of a frame, and most don't vary by much more than 2/10 of a frame! Seriously, I had to make sure that this wasn't a joke more than once. Just read what he says based on less than half a frame's difference:
This benchmark has the ability to show even minor differences in overall PC performance by a drop in frame rate. If any test can show us whether or not XtremeDDR is ?The Gamer?s Memory,? then this has to be it. And guess what? That?s exactly what it shows.
Based on half a frame!!!!!!

Just in case you thought the madness stopped there, you're wrong. The next page is full of results from overclocking. First, he says that Corsair XMS gets to "204MHz DDR... @ 2.5CL." Then he says that he got the Samsung to "203MHz DDR stably," but does not list the CL rating. I would assume that if it was true Samsung, it usually O/C's the same or better than respective Corsair XMS memory. So since the Samsung was PC2700 and the Corsair was PC3000, I would say that the Samsung was probably at CL2.5 as well, but maybe even CL2. Now, at this point, he has labeled both sticks at their "true" MHz rating, rather than their DDR rating. He's already says that he knows the difference, so this cannot simply be ignorance.

Now, we get to the RAM being reviewed: XtremeDDR PC-3500+. This "show-stopper of the night" as he calls it gets to "a cool 410MHz DDR @ CL3." Woah... there are so many problems with that statement, I have to take a second to get all of my complaints in order. First off, why in the hell did he switch to using the DDR rating of the RAM? If my mom read this (and she knows a little about how computers work), she would think that this RAM was able to run over twice as fast as the other two sticks. However, unless they were able to reach a SCREAMING 820MHz DDR!!!!, then this is just misleading. In fact, we all know that that's just not possible. Next, look at the CL rating. 3? I'll tell you this: if you drop the CL rating of a stick of RAM, you had better expect more than 1 MHz increase in clock speed. Then he goes on to say that this Editor's-Choice-worthy stick can't even get to it's rated speed! But hey, this is the board's fault. Remember, he states here the same thing that he said before: the board won't go faster than 205MHz. (In my head, I'm yelling this part) Well if the board is limiting your review, then how can you give an overclocking award to any of these sticks of RAM?!?!

So here are my gripes laid out:
- He talks like he knows something about RAM, so he can't claim pure ignorance
- He uses ridiculous benchmarks that show microscopic differences in scores and calls one stick the winner over the other.
- He incorrectly labels the overclock speeds of the first two sticks of RAM (should be just MHz, not MHz DDR)
- He compares the overclock of one stick of RAM to another at different CL ratings
- He says it's OK to not hit the speed the RAM is rated for, blaming it on the board (another gripe)

Please, all AT readers, only go to this site long enough to laugh your arse off at this ridiculous review, and then never return. If we can decrease the number of hits that this site gets, hopefully it will go the way of the Dodo. While this site definitely pisses me off, it also makes me bust up laughing. Enjoy... only once.
 
plz STOP makeing fun opf my INTARNET SIIT!!@!@!%354312

NO ONE IS MASKIGN YUPO GO TO IT!!@#!235

SO STPOP BEF4 I GET IN TOUCH WITH INTARNIC!!!@@@@@@@@@@@@@@@@@@@@@@@@@@@
 
These are taken out to the thousandth of a frame

That's because when you run the benchmark program for ut2k3... it shows them in thousandths.



Please, all AT readers, only go to this site long enough to laugh your arse off at this ridiculous review, and then never return

I have to wonder if you are a competitive website and you are trying to win hits.
 
In order of your comments...

1. Waste of time answering...
2. Well, when there are going to be small differences in performance, logic would dictate to use benchmarks that would show this would it not?
3 & 4. These were typos that I didn't catch when I originally went through the review and have since been changed. My fault on that one.
5. I never blamed it on any specific board. I tested all the sticks on 4 DDR400 and above capable boards and each time the overclock would fluctuate, sometimes higher and sometimes lower. Hence, I took the highest stable overclock for each stick and reported that.

If you don't like it, very well, don't visit the site again. I do these this as what some like to call a hobby. I, as everyone else, make genuine mistakes and may miss something when proofing a review. When you do things like that for a hobby, things like oh...college and work...can tend to dictate the amount of time you have to make sure everything is 110% perfect. That is all.
 
Ok very interesting how you try to make things seem a lot more than they really are as this shows from your review:
<quote> We were able to achieve an amazing 204MHz with our Corsair XMS3000 @ 2.5CL. This is up from our previous high of 200MHz from when we tested it back in June...But the show-stopper of the night has to be XtremeDDR?s PC-3500+. With this memory, we were able to hit a cool 410MHz @ CL2.5.
</quote>

You failed to mention that the 410MHz is actually 205, which is 1 (ONE) whole MHz greater than the Corsair! The average joe that fails to convert 410 to 205 would miss that, so I"m not sure what you're trying to do here. But clearly being able to overclock one more MHz doesn't matter.
 
it's really not that bad of a review.. if you don't like it.. well you are entitled to an opinion.. but let's not share it. :]
 
I wonder how many times the tests were repeated...
It seems completely pointless to test 3 sticks of RAM at the same clock speed and then declare winners based on differences of less than 0.5%
It also seems pointless to declare a winner in overclockability stakes when a) The difference in performance is again less than 0.5% b) The "Gamers choice RAM is effectively being underclocked - The name suggesting that a clock of about 222MHz is attainable.
 
It would seem like the normal thing to do would be to test the 3500+ memory at the same CL as the other sticks.
Drop the CL for the 3500+ stick to 2.5 and see how high it can get, Im betting it won't even reach 204, like the Samsung module did.

If I had to chose between 204 MHz at CL 2.5 and 205 MHz at CL 3, I'd take the 204 stick in a second, it'll probably outperform the 205 modules, if even only by a slight bit.
 
Originally posted by: Woodchuck2000
I wonder how many times the tests were repeated...
It seems completely pointless to test 3 sticks of RAM at the same clock speed and then declare winners based on differences of less than 0.5%
It also seems pointless to declare a winner in overclockability stakes when a) The difference in performance is again less than 0.5% b) The "Gamers choice RAM is effectively being underclocked - The name suggesting that a clock of about 222MHz is attainable.

I agree...I would say that none of the differences here account for more than just basic % error.
 
Originally posted by: wizdum
it's really not that bad of a review.. if you don't like it.. well you are entitled to an opinion.. but let's not share it. :]
You shared why can't everyone else? Did I miss some internet wide memo that stated you were the only person online who is allowed to have an opinion?

Thorin
 
what a waste of time! they run like 5 games benchmarks with all different memory, to see the 1 fps difference, but don't mention max speed of any memory stick with many comments on memory settings. After reading the review, I think i'm going to go out and pay $100 for one of those xtreme ddr sticks so it can run the exact same speed and timings my ram can, but get an extra fps in UT2K3
 
I agree everyone is entitled to their opinion. In all reality the opinions given here are only going to help me in future reviews to remember what not to do, what to add, etc. I never said "everyone go out and buy this RAM" either, I said I'd recommend it. And somewhere along the line someone wondered how many times tests were repeated I think. QIIIA - five times / res averaged. UT2K3 - same.
 
He states that, "at the 166MHz test speed, the XtremeDDR PC-3500+ comes out the winner for integer performance, while the Corsair RAM takes the floating-point performance."

Two modules running at the same speed and the same CAS rating will produce equal benchmarks. Reporting otherwise will simply confuse people.

Would you benchmark a 2.4/533 P4 vs a 1.8/400 OC'd to 2.4/533 & declare a winner?

The assumption that there would be a difference leads to interpretation of meaningless differences as a legitimate performance advantage. It was flawed from the beginning, let alone the interpretation of the results.

Viper GTS
 
Originally posted by: wizdum
That's because when you run the benchmark program for ut2k3... it shows them in thousandths.
My beef here isn't the fact that the differences are being shown, my problem is that you cannot possibly claim a winner in a roundup that has no more variation than half a frame per second.
Originally posted by: wizdum
I have to wonder if you are a competitive website and you are trying to win hits.
How many review sites are there? 20? 30? 50? There are a ton! Even if I did work for a review site (which I don't), I couldn't possibly believe that by trying to get people to avoid one of them would somehow garner me enough hits to be worth the effort. Look at my past posts on here. Sites I visit more than others include THG (though I dislike them, they do put out a lot of reviews that are pretty good), Anandtech, Storage Review, and Systemcooling. If anything you say is true, then surely I'd be smart enough to at least try to plug those sites. Anyone from AT can tell you I don't review for them, I wouldn't work for THG if they paid me, and... I'd love to work for Storage Review or Systemcooling, but I don't, and you are more than welcome to prove me wrong.

Now, compare these next two quotes:
Originally posted by: DaveSimmons
LOL, on the internet anyone can declare themselves a hardware authority, regardless of skill level.
Originally posted by: MBReview
When you do things like that for a hobby, things like oh...college and work...can tend to dictate the amount of time you have to make sure everything is 110% perfect.
Hey, I've got a job too. I understand that you may not have enough time to proofread everything you put out. I don't mind that. But the problems here aren't with typos (not all of them anyway), they're with skill level, as Dave puts it. Everyone in their right minds knows that two sticks of RAM, when set to the same clocks and timings, will perform identically, within some very slight margin of error. When reviewing a stick of RAM, all you really need are a few things:
  • One Sandra benchmark that shows how fast the RAM runs at the stock timings, just to give a general idea of how fast DDR RAM is supposed to run compared to an RDRAM stick or DDR stick at different settings. That's it. Unless there is soemthing seriously wrong with that particular stick of RAM, it should run almost exactly the same as any other stick at the same timings.
  • An overclocking result that shows how high it was able to reach versus other sticks of RAM. If you could squeeze 5 more MHz out of this stick of RAM than other sticks, then that is worth writing about. 1 MHz is just luck of the draw! Nobody should ever claim that one stick of RAM is better than another because of one extra cycle. That's just under a .5% increase. This is even more important when you yourself say that these sticks are ALL board limited. If they are all board limited, then you can't possibly say that one O/C's better than another. You see, that's what board limited means. It means that you don't know how high any of them will O/C because the board wasn't able to push them as far as they could go. Thus, what you should have said is, "Since there is no more difference than 2 MHz, and my board is limiting all of these sticks, I can't give an overclocking winner until I get a board that can run them to their limits."
  • Then, if you do have sticks of RAM that are O/Cing to their limits, you can run Sandra again and maybe a few other benchmarks (Q3 and UT2003 are appropriate) to show how much of a performance gain the overclock will give you. That way, when people are looking for the best overclocking stick of RAM and they find your review, they can see for themselves how much performance will be gained from spending $20 more for the stick you are reviewing versus the sticks that you compare it to that top out a few MHz behind. (The $20 number is just arbitrary, I don't know how much those sticks cost.)
My point is, he should have simply said that it performs as it should for a stick clocked at 166MHz and 200MHz with the given timings, and that he can't possibly claim a winner because of microscopic differences in performance and his inability to overclock it to its limits. You could have said that the RAM would be fine for anyone wanting to run it at the speeds you tested it at, just not that it performs better than the other two sticks. Nothing in your review would give you even the slightest reason to claim that one stick of RAM was performing better than the other. A good review of RAM is alll about overclocking. This is not because every stick of RAM needs to be overclocked; they don't. The reason is reliability. Let's say you ahve two sticks of RAM, A and B. RAM A overclocks to 205MHz @ CL2.5, while RAM B only overclocks to 185 MHz @ CL2.5. Will they both run the same at 166 MHz @ CL2? Yes, probably. So what does this review tell me about RAM A vs. RAM B? It tells me that, not only will I be assured that I can buy RAM A and run it at 166 MHz @ CL2, but that I won't be straining it as much as RAM B. RAM B might run at those speeds for quite some time - maybe for as long as I would want it to - but it probably wouldn't be as stable, and I can't be as positive about how long it would last as would have been about RAM A. When I pick RAM out for my systems, I try to look at at least 3 or 4 review of the RAM to make sure that everyone gets about the same results when overclocking it. This tells me quite a bit about the quality of the stick. Plus, if I do want to overclock, I would then know that these sticks are consistent, and I know what to expect from them.

When I read this review, it reminded me of this spoof of Tom's Hardware Guide. I seriously thought it was another spoof of a hardware review site.

DISCLAIMER - This is just my opinion, and I am entitled to it. If you don't share it, that's fine too.
 
I agree that the review author is suffering from conceptual problems more than typos.

From the data presented, the correct conclusion to draw is that there is no significant difference between the alternatives being compared.

Which is what most well-informed people would have expected, so it's hardly worth reporting.

Having said that, a lot of hardware reviews out there aren't exactly scientific in their approach: this one is just particularly amusing.
 
Originally posted by: MBReview
I never said "everyone go out and buy this RAM" either, I said I'd recommend it.
No, you didn't... not exactly. You gave it an "Editor's Choice Award." If that's not the highest award your site gives to a piece of hardware, then I have other issues with the naming of your awards. You also said:
If you?re looking to pick up some of this PC-3500+ to go on a FSB overclocking binge, I highly recommend you head on over and pick up a stick from him.
An overclocking binge? And with what do you back up that statement? Oh, right... with 1 MHz.
 
His biggest problem... he didnt even test the overclockability of the ram... they all could hit the same speed... Whats that tell you? Your FSB is limited not the RAM! Thats about the ONLY thing that really needs to be tested on ram... how fast it can go stably.... memory is all the same crap as long as it runs the speed... run generic or Corsair at 166 and the same timings and theyll score the same (Give or take 1% due to normal benchmarking error, benchmarks never get the exact same score over and over).

- He says it's OK to not hit the speed the RAM is rated for, blaming it on the board (another gripe)

Actually Im glad he ATLEAST said that... it didnt hit its rated speed because his board was the limiting factor, the fact that he had a gripe at all about not hitting its rated speed is laughable because its QUITE obvious he hit his motherboards limit since all 3 sticks hit the same speed give or take 2Mhz.

The review overall isnt badly written (A few mistakes here and there), its just a lack of knowledge. If you do a review on memory its quite obvious you should know as you can, go look up info. There shouldnt have been benchmarks, all memory will perform the same at the same speed and timings. The most for benchmarks you should have done was sandra for each one just to confirm every one of thoose memory sticks was running correctly and they scored similar, but there is no winner in memory bencharking between memory sticks except for overclocking. You deffinetely need to point out all 3 sticks overclocked about the same and that it seems your motherboard was the limiting factor so you were unable to test them to the fullest, which kinda negates the whole point of a memory review, but if you dont have a motherboard that will go higher, then there isnt a whole lot you can do.

Im basically rehashing what Ilmater said... he has it exactly right.
 
>So here are my gripes laid out:...

Pardon if I critique the critique.

I sometimes get teed off at reviewers too. These sorts of things you mention are not that uncommon.

Some of those things are typos. Yes in a technically oriented exposition they are confusing, especially if the technology is not that familiar.

Some of this is just journalism. Journalists like to punch up their writing with colorful, dramatic language to make it more appealing. They do this even when -especially when- there is nothing much to say due to the content not being noteworthy. It is part of the journalist art to create a story even when there is none, and to deliver X amount of words regardless if the story needs less or more. You are not considered a professional if you cannot do this. When you see translations of this type of thing from another language (for instance at tomshardware) it can be extremely puzzling and/or irritating.

Misconceptions being promulgated by reviewers are not uncommon. No one seems to be that interested in pinning some of these things down. That is because it is very difficult. Suppose real tests on different RAM sticks consistently result in differences between RAM run at the same settings. What does it mean? Does it mean anything? Suppose you do the test all day today and they come out the same way, then next week you do the same thing and it comes out different. What does it mean?

If the figures calculate down to the thousandth, I have no problem with reporting them as such. They are after all real measurements, and the essence of unbiased science. For instance 200 FPS for 10 minutes is 120 000. You could get fps down to one part in a hundred thousand. The difficulty is discovering what these measurements mean. In general, no one knows what it means even when the fps is 1 part in 10.

I'd prefer normalized graphs, so I would not have to figure out percentage differences in my head. The thing that makes be boil is to see bar graphs that show just the top part, so 1% looks like 25%. Of course we understand that putting up identical bar graphs is something a journalist would not want to do. It may be accurate but it's boring.
 
Back
Top