DxOMark: Why Canonians hate it so....

[JohnnyRebel]

I just realized why Canon shooters hate DxOMark scores so much. There's no Canon cameras in sight. Check out the top 10:

http://nikonrumors.com/2012/09/19/nikon-d600-gets-second-best-dxomark-score-after-the-d800e.aspx/



JR

 

[randomrogue]

Is this benchmark worth looking at? I saw the D600 score and it's basically as good as the D800 and D4. The high iso score is insane compared to my current camera. It's over 2.5 stops better.

 

[JohnnyRebel]

Is this benchmark worth looking at? I saw the D600 score and it's basically as good as the D800 and D4. The high iso score is insane compared to my current camera. It's over 2.5 stops better.

The criticism is that the ratings are only about the sensor, while there is so much more to a camera system. I have been critized on AT for bringing up these scores, usually by Canon fans. I like the work these guys do.

 

[fuzzybabybunny]

Is this benchmark worth looking at? I saw the D600 score and it's basically as good as the D800 and D4. The high iso score is insane compared to my current camera. It's over 2.5 stops better.

I haven't made up my mind on these yet either. I'd say you should just compare actual images yourself:

http://www.imaging-resource.com/IMCOMP/COMPS01.HTM

I did the D600 vs 5DMK3

At similar high ISOs I felt that the D600 was a LOT noisier but it was able to retain a bit more detail in textures. The 5D had very little noise but also less detail. On the other hand, when shooting something like text (black on white fine details) the Canon appears sharper than the Nikon at the same high ISO. Basically, it's not really a clear win for either camp. I would have to give the nod to the Nikon because at least you can attempt to clear up the noise in post yourself and save as much data as you can. The Canon just doesn't have any data to save.

When shooting video I think that the Nikon, at least the D800, absolutely sucks at noise. Sucks sucks sucks sucks sucks. Maybe the Nikon D600 with its lower resolution will be better, but man, the change would have to be night and day from the D800.

 

[Cattykit]

I haven't made up my mind on these yet either. I'd say you should just compare actual images yourself:

http://www.imaging-resource.com/IMCOMP/COMPS01.HTM

I did the D600 vs 5DMK3

At similar high ISOs I felt that the D600 was a LOT noisier but it was able to retain a bit more detail in textures. The 5D had very little noise but also less detail. On the other hand, when shooting something like text (black on white fine details) the Canon appears sharper than the Nikon at the same high ISO. Basically, it's not really a clear win for either camp. I would have to give the nod to the Nikon because at least you can attempt to clear up the noise in post yourself and save as much data as you can. The Canon just doesn't have any data to save.

When shooting video I think that the Nikon, at least the D800, absolutely sucks at noise. Sucks sucks sucks sucks sucks. Maybe the Nikon D600 with its lower resolution will be better, but man, the change would have to be night and day from the D800.

1. Those were shot on default settings. In Canon's case, Standard Picture Style with Standard NR. The matter of fact is this: Standard PS itself smears detail for noise and Standard NR also smears detail for noise. So, what happens is that details get massaged not only once but twice. Even worse, same dumb processing is applied when you're using low ISO like 100.
The good side is that, compared to Nikon, you actually have more choices regarding detail and noise balance. In my case, since I care more about detail like you, I only use Neutral or Neutral based custom PS and have NR at Low (it only targets color noise and loss of detail is very very minimal to none.)

Since you have T2i, try shooting a scene using Neutral Picture Style with NR low or off. The difference is quite big and visible. Oh and Keep it mind all the other built-in Picture Styles are base on Standard one except Neutral.

2. Nikon can't really do anything unless they use their own sensor. Since most of Nikon cameras use Sony ones, it's up to Sony to fix that situation. That makes the situation interesting. Sony as a corporation needs to protect its video market. If they make such a sensor give it to Nikon, they(Nikon) may not only use the sensor but add killer features that would jeopardize video market where Sony is the king. Look at Canon, even after they have achieved huge followings for the first time, they're playing the new found game very safe. Although their video market share is low, they don't want jeopardize it.

 

[RobDickinson]

DXO mark is an interesting way of comparing some sensor information if you normalise for print size.

Its in no way shape or form a way of choosing a camera. A camera is a lot more than just the sensor.

 

[mchammer187]

I just realized why Canon shooters hate DxOMark scores so much. There's no Canon cameras in sight. Check out the top 10:

http://nikonrumors.com/2012/09/19/nikon-d600-gets-second-best-dxomark-score-after-the-d800e.aspx/

BTW, notice BlastingCap's previous rebuff in my sig. To save time & space just type BR (for BlastingCap Rocks) instead of redundantly blasting me over pointing this out. LOL


JR

DXO's measurements are quite useful. I think their scoring methodology is way off though. As a canon user would I love the D800E or even D600 sensor in my 5D mark III. I sure as hell would. But when you have APS-C sensors outscoring MF camera's something is truly truly off base here and no way is DXO the be all and end all in regards to Image Quality.

 

[JohnnyRebel]

no way is DXO the be all and end all in regards to Image Quality

I guess this is my gripe. People keep saying this, even though nobody is saying otherwise. Everybody already knows this.

I can only guess that the need to declare what is already known is to somehow minimize the effect of results we either don't like or can't imagine.

AND, just maybe Canon is really behind the curve on camera sensors and that is reflected in the scores. BUT, since that is not the end-all be-all of a camera system Canon is still a great tool.

JR

EDIT: I missed RobDickinson saying "A camera is a lot more than just the sensor." Just so, and yet we have to say it again and again..

 

[blastingcap]

Sony is the goliath of the sensor world. Even archrivals like Panasonic use Sony image sensors in some of their cameras. Sony makes most smartphone and compact camera sensors. Sony also makes sensors for its high-end videocameras and interchangeable-lens cameras as well. Plus they supply Oly interchangeable lens camera sensors, Pentax, and Nikon, which is why Nikon's DSLRs typically outclass Canon-built stuff. Sony sensors are built on a smaller and more advanced process than Canon's... probably because Sony has such huge scale that they can upgrade their fabs faster and more economically than the competition can.

Analogy: Sony = Intel fabs. Canon = GloFo/TSMC.

I just realized why Canon shooters hate DxOMark scores so much. There's no Canon cameras in sight. Check out the top 10:

http://nikonrumors.com/2012/09/19/nikon-d600-gets-second-best-dxomark-score-after-the-d800e.aspx/

BTW, notice BlastingCap's previous rebuff in my sig. To save time & space just type BR (for BlastingCap Rocks) instead of redundantly blasting me over pointing this out. LOL


JR

 

[ElFenix]

I haven't made up my mind on these yet either. I'd say you should just compare actual images yourself:

http://www.imaging-resource.com/IMCOMP/COMPS01.HTM

of course the problem with studio comparisons is that they use well-lit targets. who the heck is using 50,000+ sensitivity setting on a well-lit target?

 

[corkyg]

Here's one Canonian (love the term!) that does not hate DXO. Fact is, I never heard of DXO until this thread.

 

[mchammer187]

I guess this is my gripe. People keep saying this, even though nobody is saying otherwise. Everybody already knows this.

I can only guess that the need to declare what is already known is to somehow minimize the effect of results we either don't like or can't imagine.

AND, just maybe Canon is really behind the curve on camera sensors and that is reflected in the scores. BUT, since that is not the end-all be-all of a camera system Canon is still a great tool.

JR

EDIT: I missed RobDickinson saying "A camera is a lot more than just the sensor." Just so, and yet we have to say it again and again..

I did admit Nikon/Sony sensors are superior Canon. I am saying when they rate an aps-c sensor higher than a MF digital sensor by the same margins than something is wrong with the way they are scoring sensor image quality which is different than a total camera system.

 

[RobDickinson]

Its funny that when canon sensors were superior DXO still didnt show it and completely ignored the resolution advantages.

 

[blastingcap]

Btw as a followup to my previous message in this thread: Apparently everyone but Canon is already at the 65-90nm range, with Sony apparently getting very good yields on its 90nm process = better performance, lower cost. http://www.imagesensors.org/Past Workshops/2011 Workshop/2011 Papers/R02_Fontaine_Review.pdf

I have heard a lot about how Canon is *still* stuck at 180nm which could explain why its a stop behind Sony's latest and greatest. I'll see if I can dig up a citation for that. This is the closest I could come up with in a few minutes of searching: http://image-sensors-world.blogspot.com/2012_06_01_archive.html

I hang out mostly in the CPU and GPU forums here where we know how much a node shrink helps in multiple ways, so I hope Canon moves to 90nm sooner rather than later, to stimulate competition in the marketplace.

 

[CptObvious]

I don't consider myself a Canonian or Nikonian. I must have switched systems about 4 times now (actually have owned more Nikons). One thing though is I take these sensor reviews with a grain of salt. When I switched from the 60D to a D5100 for a lighter travel camera, I was expecting to be blown away based on DxOMark scores but the difference was actually minimal in real world use. I loaded up some RAW files from my old Nikon D40 into LR4 and it surprised me how much detail can be recovered - a lot closer to the new cameras than I was expecting. Now I'm shooting with an old 5D and much prefer the look of the RAW files over the D5100, even if the D5100's are 'technically' better.

 

[twistedlogic]

Its funny that when canon sensors were superior DXO still didnt show it and completely ignored the resolution advantages.

From DXO themselves:

"Sensor Overall Score AND resolution are two independent metrics of sensor performance. This means that just because camera A has more pixels than camera B (and thus sees more details) does not mean that camera A’s Sensor Overall Score will be better. Rather, Sensor Overall Score measures the quality of the captured signal, either at a pixel level or at the full sensor level. So before comparing cameras with Sensor Overall Score, it is important to first determine the resolution you are looking for (which largely depends on the size of the screen or the print you intend to use or produce).
Once you choose an appropriate resolution, the Sensor Overall Score becomes a fair and powerful tool with which to make comparisons.
In a camera, resolution is dependent on both sensor and lens performances. So to compare and rank digital cameras while taking resolution into account, you should look at the DxOMark Score for lenses (with camera), which weighs a number of image quality parameters, including resolution."

I loaded up some RAW files from my old Nikon D40 into LR4 and it surprised me how much detail can be recovered - a lot closer to the new cameras than I was expecting. Now I'm shooting with an old 5D and much prefer the look of the RAW files over the D5100, even if the D5100's are 'technically' better.

The sony sensors excel at lifting the shadows with minimal noise. My D40 has horrific noise in low light shadows @ ISO 1600 without touching the exposure. But the D40 always impresses me on how easy it is to take a crisp image, I'm guessing because the photosites are so big that there is less chance of blurring two pixels.

I've never had a problem with DXO's scores. They have been pretty spot on from my experience, much closer than the perspective of the general masses.

 

[arrfep]

I don't consider myself a Canonian or Nikonian. I must have switched systems about 4 times now (actually have owned more Nikons). One thing though is I take these sensor reviews with a grain of salt. When I switched from the 60D to a D5100 for a lighter travel camera, I was expecting to be blown away based on DxOMark scores but the difference was actually minimal in real world use. I loaded up some RAW files from my old Nikon D40 into LR4 and it surprised me how much detail can be recovered - a lot closer to the new cameras than I was expecting. Now I'm shooting with an old 5D and much prefer the look of the RAW files over the D5100, even if the D5100's are 'technically' better.

Exactly this. IMO, 99% of users have reached the point of diminishing returns. Even the iPhone 4 camera produces results that the majority of users would find satisfactory.

People constantly ask me for camera recommendations, and I simply tell them to buy the cheapest currently produced DSLR they can find. The resulting IQ from all modern DSLRs is startlingly similar for most shooting situations.

Of course, if everyone only ever bought what they needed, there'd be nearly no innovation in the industry. I think the industry knows and fears this, which is why we see things like Canon adding featurs piece-meal, like WiFi and GPS in an enthusiast camera, but not pro model, etc.

 

[randomrogue]

Exactly this. IMO, 99% of users have reached the point of diminishing returns. Even the iPhone 4 camera produces results that the majority of users would find satisfactory.

People constantly ask me for camera recommendations, and I simply tell them to buy the cheapest currently produced DSLR they can find. The resulting IQ from all modern DSLRs is startlingly similar for most shooting situations.

Of course, if everyone only ever bought what they needed, there'd be nearly no innovation in the industry. I think the industry knows and fears this, which is why we see things like Canon adding featurs piece-meal, like WiFi and GPS in an enthusiast camera, but not pro model, etc.

I'm going to partially disagree. At base ISO? Sure then maybe. However I have a D3000 and anything above ISO 400 is just too noisy to use (most of the time). The noise is ugly. I pretty much judge a camera by its ability to photograph in low light since I tend to do that a lot. That's where you'll see a huge difference. Also, as I understand it, not only do cameras like the D600 have much less noise at high ISO but the noise is more fine and usable. I want a camera that when noise is present produces a decent grain like old b/w film that I can actually use. Blotchy digital noise is just ugly.

 

[JohnnyRebel]

However I have a D3000 and...

Didn't you read Ken Rockwell's review before you bought it?!?

BTW, you're right.

 

[randomrogue]

Didn't you read Ken Rockwell's review before you bought it?!?

BTW, you're right.

lol that review is pretty funny. I've had this camera since 2008 or 2009 (whenever it was released) and replaced a point and shoot. That was definitely a step up at least.

 

[arrfep]

I'm going to partially disagree. At base ISO? Sure then maybe. However I have a D3000 and anything above ISO 400 is just too noisy to use (most of the time). The noise is ugly. I pretty much judge a camera by its ability to photograph in low light since I tend to do that a lot. That's where you'll see a huge difference. Also, as I understand it, not only do cameras like the D600 have much less noise at high ISO but the noise is more fine and usable. I want a camera that when noise is present produces a decent grain like old b/w film that I can actually use. Blotchy digital noise is just ugly.

True, true. I was speaking more for the soccer-mom crowd and such, people buying DSLRs because they think the machine itself will make their photos 1000x better. Very little reason for any of them to pick, say, a D600 over a D5100.

 

[Eug]

I'll have to check out ParaNorman. It was shot on the Canon 5D Mark II (stop motion).

http://fstoppers.com/paranorman-stop-motion-with-the-canon-5d-mark-ii

FWIW, an episode of a TV show was shot here, and they used the Canon 5D for low budget 2nd unit video shots, for some supporting close up footage.

 

[RobDickinson]

From DXO themselves:

"Sensor Overall Score AND resolution are two independent metrics of sensor performance. This means that just because camera A has more pixels than camera B (and thus sees more details) does not mean that camera A’s Sensor Overall Score will be better. Rather, Sensor Overall Score measures the quality of the captured signal, either at a pixel level or at the full sensor level. So before comparing cameras with Sensor Overall Score, it is important to first determine the resolution you are looking for (which largely depends on the size of the screen or the print you intend to use or produce).
Once you choose an appropriate resolution, the Sensor Overall Score becomes a fair and powerful tool with which to make comparisons.
In a camera, resolution is dependent on both sensor and lens performances. So to compare and rank digital cameras while taking resolution into account, you should look at the DxOMark Score for lenses (with camera), which weighs a number of image quality parameters, including resolution."

yes, so you get two 'views'

A pixel level view - which if your pixels are 1/4 the size of the other camera its no use.

And a full frame view, which is normalised for print size for noise and dynamic range, this is much better but still ignores the extra detail you would get with more resolution.

 

[blastingcap]

The Panasonic GH2 was already on par with all but the highest-end Canon APS-C cameras, but the Olympus E-M5 was widely seen as even better. (The Oly is the first of several micro four thirds cameras slated to use Sony sensors.)

Recently the E-M5 finally got DXOmark'ed and--this is probably embarrassing for Canon--the smaller-sensored E-M5 beats every single Canon APS-C camera to date. Canon seriously needs to upgrade their fabs to be competitive--they can't keep using 180nm sensors hoping to compete with Sony's 90nm sensors.

http://www.dxomark.com/index.php/Pu...s-OM-D-E-M5-The-best-of-the-micro-4-3-cameras

 

[hmcindie]

Recently the E-M5 finally got DXOmark'ed and--this is probably embarrassing for Canon--the smaller-sensored E-M5 beats every single Canon APS-C camera to date. Canon seriously needs to upgrade their fabs to be competitive--they can't keep using 180nm sensors hoping to compete with Sony's 90nm sensors.

Except that the nm size isn't the "be all end all". This is being propagated because a smaller nm fabsize allows you to make 36megapixels for a fullframe sensor. Canon makes at the most, 22mp for that same sensor size so you don't NEED that smaller nm. Because the actual sensor gathering pixels are supposed to be bigger with 22mp than 36mp. Not smaller.

Now when Canon decides to compete with mp and go for a 40+ sensor, then they would need to upgrade.

Also I don't consider current Sony sensors to be much better than even old Canon APS-C sensors. The sensor in the Sony nex-5r (which is a very new APS-C sensor) does not outperform the Canon 7d (old APS-C) at all. DXOMark says it does, but the differences are very, very negligible. Canon 7d makes slightly less noise at higher ISOs and the Nex-5r does slightly better with ISO 100 (no shadow sensor noise so DR is a bit better). But the differences are very slight and I am comparing sensor technology that is years apart for Sonys advantage.