Anand's 9800XT vs NV38 "Review"

[Jeff7181]

Oh please...

 

[First]

To answer a few questions...

First off, all in depth IQ results and high resolution testing will be done in Part 2. That is why you don't see 1280x1024 or 1600x1200 testing or screen shots with detailed analysis right now. We mentioned these facts in Part 1.

Secondly, I don't really see how using beta drivers for a review is somehow wrong because they're not available to the public. Last I checked, HL2 wasn't available to the public yet, does that mean we shouldn't use it in a review? Now, of course, I think comparing the official 45.23 drivers to the beta 52.13 drivers we used would have been very useful to readers. However, the final Det 50's are going to be out very very soon, and they will reflect the 52 series beta drivers very closely as far as we've been told. We'll do a comparison to see if that's true. And of course, you'll have all the IQ and high resolution results to compare for yourself. I think a little patience here is the best medicine for most people.

Originally posted by: spam<b>BTW Evan Leib, would Nvidia have let you test their upcomiing card without using their Beta drivers? If not, then you should have refused to test their card.

No, NVIDIA had no influence over our testing, they simply gave us their NV38 reference card and we tested it.

Originally posted by: Pete
Coup d'etat, not coup de tat.

Evan, I'm curious why 9600P results were left out of the Homeworld 2 benches. I also thought ATi's framerate problems with NWN were known, because the devs coded the engine around nVidia cards (though I also know ATi seems to be working to fix this).

Good question, I?m not sure. Email Anand and see if he responds. Or better yet, post this question in the blog.

Originally posted by: lifeguard1999
Evan Lieb, see? I am officially eating my words. Munch.... Munch..... Munch.....

No you're not.

 

[Sazar]

Originally posted by: Evan Lieb
To answer a few questions...

First off, all in depth IQ results and high resolution testing will be done in Part 2. That is why you don't see 1280x1024 or 1600x1200 testing or screen shots with detailed analysis right now. We mentioned these facts in Part 1.

Secondly, I don't really see how using beta drivers for a review is somehow wrong because they're not available to the public. Last I checked, HL2 wasn't available to the public yet, does that mean we shouldn't use it in a review? Now, of course, I think comparing the official 45.23 drivers to the beta 52.13 drivers we used would have been very useful to readers. However, the final Det 50's are going to be out very very soon, and they will reflect the 52 series beta drivers very closely as far as we've been told. We'll do a comparison to see if that's true. And of course, you'll have all the IQ and high resolution results to compare for yourself. I think a little patience here is the best medicine for most people.

Originally posted by: spam<b>BTW Evan Leib, would Nvidia have let you test their upcomiing card without using their Beta drivers? If not, then you should have refused to test their card.

No, NVIDIA had no influence over our testing, they simply gave us their NV38 reference card and we tested it.

Originally posted by: Pete
Coup d'etat, not coup de tat.

Evan, I'm curious why 9600P results were left out of the Homeworld 2 benches. I also thought ATi's framerate problems with NWN were known, because the devs coded the engine around nVidia cards (though I also know ATi seems to be working to fix this).

Good question, I?m not sure. Email Anand and see if he responds. Or better yet, post this question in the blog.

Originally posted by: lifeguard1999
Evan Lieb, see? I am officially eating my words. Munch.... Munch..... Munch.....

No you're not.

as it stands evan... anand did not publish hl2 results therefore your first point is moot... if anand had chosen to publish a benchmark of hl2 instead of posting the results in his blog from a reputable source (?) then there would be less problems with using the beta drivers...

historically... there are many instances where beta drivers have been released to rain on somoenes parade and then never publicly released... in light of that it is only prudent to bench with official drivers... naturally as I stated in my first post in this thread I can understand the use of the drivers if there is no support for the nv38 in the current driver set (which would be hard to believe seeing as this is supposedly a higher clocked part.. marchitecture should not have changed AFAIK)

I agree completely that official driver scores should have been posted as well... and that was one of the questions I had raised... was there a reason that no det 44.03 or 45.23 results were posted ?

anyways.. cheers for your time I'll post anand a coupla points in the blog..

 

[Sazar]

btw...

is anyone going to change the figures for the tests with and without AA/AF for some of the graphs ?

did anyone dbl check the homeworld 2 figures ?



is there a specific reason why the nv31 is SIGNIFICANTLY faster with AA and AF turned on compared to it turned off ? (ie over 50%)

even the nv38 picks up speed

/me wonders if there should be a correction issued with notes ASAP to prevent a splintergate type fiasco...

 

[First]

as it stands evan... anand did not publish hl2 results therefore your first point is moot... if anand had chosen to publish a benchmark of hl2 instead of posting the results in his blog from a reputable source (?) then there would be less problems with using the beta drivers...

Not sure what you're saying. It doesn't matter which review we published HL2 results in, we have published them. This fact is directly applicable to the argument that posting results of beta drivers, that aren't available to the public, is wrong.

historically... there are many instances where beta drivers have been released to rain on somoenes parade and then never publicly released...

A common misconception. It's not as if NVIDIA/ATI will release beta drivers only for a review and then release final drivers that perform worse. From a engineer's standpoint that makes zero sense, they'll want to include the code from the beta drivers in the final drivers if it speeds up performance.

in light of that it is only prudent to bench with official drivers... naturally as I stated in my first post in this thread I can understand the use of the drivers if there is no support for the nv38 in the current driver set (which would be hard to believe seeing as this is supposedly a higher clocked part.. marchitecture should not have changed AFAIK)

This is another issue. Should we really use 45.23 drivers for NV38 if every NV38 in the world will be shipping with Det 50 drivers?

I agree completely that official driver scores should have been posted as well... and that was one of the questions I had raised... was there a reason that no det 44.03 or 45.23 results were posted ?

Yeah, I agree, some comparison results would have been good. It was likely a time issue for Anand and Derek.

 

[First]

Originally posted by: Sazar
btw...

is anyone going to change the figures for the tests with and without AA/AF for some of the graphs ?

did anyone dbl check the homeworld 2 figures ?



is there a specific reason why the nv31 is SIGNIFICANTLY faster with AA and AF turned on compared to it turned off ? (ie over 50%)

even the nv38 picks up speed

/me wonders if there should be a correction issued with notes ASAP to prevent a splintergate type fiasco...

Driver problems most likely. We'll have to wait for Part 2.

 

[Jeff7181]

What I wanna know is why such a "low end" system was used. Granted a 2.8 Ghz P4 isn't exactly low end... but shouldn't you be using the best performing CPU you can get your hands on if you're going to compare results from video cards? More than 512 MB of RAM may not have made any difference in those benchmarks, but I think it's safe to say, 1 GB is the standard amount of RAM now for a gaming rig.

 

[Sazar]

Originally posted by: Evan Lieb

Not sure what you're saying. It doesn't matter which review we published HL2 results in, we have published them. This fact is directly applicable to the argument that posting results of beta drivers, that aren't available to the public, is wrong.

the published scores are none achieved by anand per the weblog but those provided to him by a reliable source... I agree publicly available drivers/benchs should be used... you used the det's but not hl2 which made your point seem a little odd...

A common misconception. It's not as if NVIDIA/ATI will release beta drivers only for a review and then release final drivers that perform worse. From a engineer's standpoint that makes zero sense, they'll want to include the code from the beta drivers in the final drivers if it speeds up performance.

looking @ previous examples and going through information posted @ b3d.. it would seem that there have been instances... also... as an engineer I concur... it makes no sense from my view point.. but it is a powerful marketing tool and these ihv's are not engineering powerhouses alone they are businesses and as such their decisions are not purely engineering based... look @ ati itself... speed bumps of their r300 lineup instead of greater r&d and other expenses to release a new gen card this year.. business decisions...

This is another issue. Should we really use 45.23 drivers for NV38 if every NV38 in the world will be shipping with Det 50 drivers?

in light of the current benchmarks the official dets should have been tried... like others have also mentioned... it would be a good watermark to test expected gains... this is not an anti-nvidia statement... it is a practical statement given the number of nvidia users... it gives a good indicator of expected improvements in more dx9 intensive situations using PP...

however to directly answer your question... YES the benchmark should have included the 45.23 figures IMO...

Yeah, I agree, some comparison results would have been good. It was likely a time issue for Anand and Derek.

yah I had suggested that as a possible explanation... and I reckon time was an issue looking @ some of the figures being posted

ie much higher fps WITH AA and AF turned on lol...

thanks for your time evan... always a pleasure

 

[Sazar]

Originally posted by: Evan Lieb

Originally posted by: Sazar
btw...

is anyone going to change the figures for the tests with and without AA/AF for some of the graphs ?

did anyone dbl check the homeworld 2 figures ?



is there a specific reason why the nv31 is SIGNIFICANTLY faster with AA and AF turned on compared to it turned off ? (ie over 50%)

even the nv38 picks up speed

/me wonders if there should be a correction issued with notes ASAP to prevent a splintergate type fiasco...

Driver problems most likely. We'll have to wait for Part 2.

lol... come on evan

driver problems unlikely looking @ the results correlation in other benchmarks...

its likely something was fubar when benching this particular test... retesting it would probably be prudent... maybe something was left on the first time round ? or something was not applied the second time round?



unlikely to be a driver bug seeing as this was not reported by either anand or derek...

 

[Nebor]

Originally posted by: Jeff7181
What I wanna know is why such a "low end" system was used. Granted a 2.8 Ghz P4 isn't exactly low end... but shouldn't you be using the best performing CPU you can get your hands on if you're going to compare results from video cards? More than 512 MB of RAM may not have made any difference in those benchmarks, but I think it's safe to say, 1 GB is the standard amount of RAM now for a gaming rig.

I agree that they should have gone w/ 1GB of ram... But keep in mind that they didn't use a 2.8ghz P4.... They used a 2.8ghz P5. Although early reports seem to be telling us that it performs significantly slower than the P4 3.2. Don't you guys have a P4 3.2 EE you can use for this stuff?

 

[First]

Originally posted by: Jeff7181
What I wanna know is why such a "low end" system was used. Granted a 2.8 Ghz P4 isn't exactly low end... but shouldn't you be using the best performing CPU you can get your hands on if you're going to compare results from video cards? More than 512 MB of RAM may not have made any difference in those benchmarks, but I think it's safe to say, 1 GB is the standard amount of RAM now for a gaming rig.

A 2.8GHz Prescott is pretty close apparently. None of the games really needed more than 512MB AFAICT, though I agree 1GB just to be safe I suppose.

the published scores are none achieved by anand per the weblog but those provided to him by a reliable source...

No, we physically tested HL2 and had full control over everything save for the setup, which Valve wanted to keep as a 3.0C desktop system for whatever reason. We also tested HL2 with mobile parts with a setup of our choice.

looking @ previous examples and going through information posted @ b3d.. it would seem that there have been instances... also... as an engineer I concur... it makes no sense from my view point.. but it is a powerful marketing tool and these ihv's are not engineering powerhouses alone they are businesses and as such their decisions are not purely engineering based... look @ ati itself... speed bumps of their r300 lineup instead of greater r&d and other expenses to release a new gen card this year.. business decisions...

In regards to R300 speed bumps, who really knows if that's purely a business decision (though it certainly is partly one). If ATI's next generation part still isn't ready (R4xx), then R300 speed bumps makes perfect engineering sense. In regards to performance of beta drivers, it really literally makes no sense from a business or engineering standpoint to leave out code that increases performance in a final driver. I can't think of a reason why that would happen unless it's not reliable code.

in light of the current benchmarks the official dets should have been tried... like others have also mentioned... it would be a good watermark to test expected gains... this is not an anti-nvidia statement... it is a practical statement given the number of nvidia users... it gives a good indicator of expected improvements in more dx9 intensive situations using PP...

however to directly answer your question... YES the benchmark should have included the 45.23 figures IMO...

Perhaps. We'll have to see how the final Det 50's perform, and so if the IQ and fps are the same or better than the 52 series AT used, then 45.23 testing would have proven pointless.

lol... come on evan

driver problems unlikely looking @ the results correlation in other benchmarks...

its likely something was fubar when benching this particular test... retesting it would probably be prudent... maybe something was left on the first time round ? or something was not applied the second time round?



unlikely to be a driver bug seeing as this was not reported by either anand or derek...

Well...ask Anand then.

thanks for your time evan... always a pleasure

No problem.

 

[Jeff7181]

A 2.8GHz Prescott is pretty close apparently.

So what you're saying is, you tested the 2.8 Ghz pre-production Prescott and it performs about the same as a 3.2 Ghz P4... but you can't show us because of some sort of NDA =) Interesting... not quite the jump I was expecting from the much touted Prescott... but still very good progress!

 

[Jeff7181]

Oh... by the way... have you noticed that the 51.75 Detonators don't render fog correctly? Not talking about in AquaMark3... but they don't seem to render it correctly in ANY game... in Soldier of Fortune 2, on the MP jor1 map... scenery off in the distance is perfectly clear... but other people have LOTS of fog covering them... and on the kam5 map, the fog is so thick I can't see 6 inchs in front of me... but I can see other players ALL the way on the other side of the map... which makes for easy sniping... black body on a pure white background isn't hard to spot and shoot =) And in Rainbow Six 3: Raven Shield, it switches from no fog, to thick fog depending where you run to. Have you noticed anything like that Evan? If so, is it fixed in the 52.x Detonators?

 

[Sazar]

Originally posted by: Evan Lieb

the published scores are none achieved by anand per the weblog but those provided to him by a reliable source...

No, we physically tested HL2 and had full control over everything save for the setup, which Valve wanted to keep as a 3.0C desktop system for whatever reason. We also tested HL2 with mobile parts with a setup of our choice.

from anand's blog evan... this is what I linked to on page 1 of this thread... and what I raised a question about...

Here are some Half Life 2 numbers for you to look at; they were provided by a reliable source, but I could not verify anything myself so take them with a grain of salt. ATI was running in their DX9 codepath and the mixed mode codepath was used for NVIDIA. No AA/AF was enabled and we're looking at 1024x768 scores

 

[Genx87]

Evan, are you saying from what you have seen of the 52.xx driver it is very close to the 45.xx in terms of iq?

I must admit if Nvidia can pull this off it would be amazing.

 

[Sazar]

Originally posted by: Genx87
Evan, are you saying from what you have seen of the 52.xx driver it is very close to the 45.xx in terms of iq?

I must admit if Nvidia can pull this off it would be amazing.

comparison really should be with 44.03's seeing as the iq is a lot better in such things as AM3...

if 5x.xx is same as 44.03 iq... well and good...

 

[Genx87]

Puhlease. 45.xx is pretty damn close..............if they can get 45.xx quality and get that kind of speed in HL2. That is a major win..............


Sorry but to be honest the iq looked plenty fine in 45.xx. 51.xx did indeed have some issues.

 

[Sazar]

Originally posted by: Genx87
Puhlease. 45.xx is pretty damn close..............if they can get 45.xx quality and get that kind of speed in HL2. That is a major win..............


Sorry but to be honest the iq looked plenty fine in 45.xx. 51.xx did indeed have some issues.

if you look @ direct screenshot comparisons between 44.03's and 45.23's you see missing light sources and effects [edit] in AM3... fog rendering issues and the like have been repeatedly reported with builds of 5x.xx floating around the internet...

as an owner of a product... it is good news for fps increase like this since it vindicates a purchase to an extent... even if it is using partial precision...

however... from a consumer standpoint... the consumer should be made aware of issues... it is irrelevant comparing a driver set with lowered IQ v/s another driver set with even lower IQ and saying thats not bad... compare it to the original and voila... you have a valid comparison...

I will be quite shocked if anand does in fact do his IQ comparisons only using 45.23 as the base... it has been shown by a multitude of websites that the 44.03 IQ is superior... by what quantity is in the eye of the beholder

 

[Pete]

Evan, I took your advice and posted my Q in both the article comment thread and Anand's blog. Thanks for the tip, and we'll see if either he or Derek answers. I tend not to ask Anand directly because I figure he gets too much spam already.

 

[First]

Sazar, that's not what I'm talking about. I'm talking about previous HL2 scores AT has published, so my argument still stands.

Originally posted by: Pete
Evan, I took your advice and posted my Q in both the article comment thread and Anand's blog. Thanks for the tip, and we'll see if either he or Derek answers. I tend not to ask Anand directly because I figure he gets too much spam already.

Good idea. Anand likes to answer his blog a lot AFAICT.

Originally posted by: Genx87
Evan, are you saying from what you have seen of the 52.xx driver it is very close to the 45.xx in terms of iq?

I don't know, I haven't done much testing.

 

[spam]

Hi Evan,

Thanks for answering our questions,. I do apologize for the edge in some of my questions.

I do think that using published drivers are needed in the review to establish a baseline of performance. The way it is currently structured adds to the complexity needlessly. It raises a lot of questions rather than answering them as this thread suggests. Over all I do want to emphasize this reviews form of testing and methodology is moving in the right direction.

 

[Sazar]

Originally posted by: Evan Lieb
Sazar, that's not what I'm talking about. I'm talking about previous HL2 scores AT has published, so my argument still stands.

/me looks @ thread title...

hmm... you sure about that evan ?

 

[First]

Originally posted by: spam
Hi Evan,

Thanks for answering our questions,. I do apologize for the edge in some of my questions.

I do think that using published drivers are needed in the review to establish a baseline of performance. The way it is currently structured adds to the complexity needlessly. It raises a lot of questions rather than answering them as this thread suggests. Over all I do want to emphasize this reviews form of testing and methodology is moving in the right direction.

No problem, and I agree.

Originally posted by: Sazar

Originally posted by: Evan Lieb
Sazar, that's not what I'm talking about. I'm talking about previous HL2 scores AT has published, so my argument still stands.

/me looks @ thread title...

hmm... you sure about that evan ?

Uh, what does that have to do with the thread title?

 

[Sazar]

Originally posted by: Evan Lieb

Originally posted by: Sazar

Originally posted by: Evan Lieb
Sazar, that's not what I'm talking about. I'm talking about previous HL2 scores AT has published, so my argument still stands.

/me looks @ thread title...

hmm... you sure about that evan ?

Uh, what does that have to do with the thread title?

point is simple... all this time I have been referring to implied hl2 iq in the nv38 review and from anand's blog... which is why I can't see where we went back to the hl2 scores from before...

 

[First]

point is simple... all this time I have been referring to implied hl2 iq in the nv38 review and from anand's blog... which is why I can't see where we went back to the hl2 scores from before...

OK, and like I said before...

"I don't really see how using beta drivers for a review is somehow wrong because they're not available to the public. Last I checked, HL2 wasn't available to the public yet, does that mean we shouldn't use it in a review?"...

And of course, this had nothing to do with the HL2 blog scores you brought up.