7970 xfire Windows performance problem - determined the fault

[BrightCandle]

I have crappy Windows Aero performance with my 2 x7970's. Lots of people have suggested many things but nothing helps. I recently determined there was a warning being reported in the event logs about Windows performance. You can check if you are also getting the same events by going to control panel>administrative tools>event viewer and then in there use the menu Applications and Services>Microsoft>Windows>Diagnostics-Performance>Operational. The event details I see as XML are as follows:

- <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
- <System>
  <Provider Name="Microsoft-Windows-Diagnostics-Performance" Guid="{CFC18EC0-96B1-4EBA-961B-622CAEE05B0A}" /> 
  <EventID>500</EventID> 
  <Version>1</Version> 
  <Level>3</Level> 
  <Task>4006</Task> 
  <Opcode>39</Opcode> 
  <Keywords>0x8000000000010000</Keywords> 
  <TimeCreated SystemTime="2012-06-08T18:39:32.188063700Z" /> 
  <EventRecordID>1451</EventRecordID> 
  <Correlation ActivityID="{02F04A90-F800-000A-E311-F1659D45CD01}" /> 
  <Execution ProcessID="1156" ThreadID="5112" /> 
  <Channel>Microsoft-Windows-Diagnostics-Performance/Operational</Channel> 
  <Computer>XXXX-PC</Computer> 
  <Security UserID="S-1-5-19" /> 
  </System>
- <EventData>
  <Data Name="DisplayDeviceFriendlyNameLength">26</Data> 
  <Data Name="DisplayDeviceFriendlyName">AMD Radeon HD 7900 Series</Data> 
  <Data Name="MemoryBandwidth">86872300</Data> 
  <Data Name="MemorySize">1765773312</Data> 
  <Data Name="Scenario">3</Data> 
  </EventData>
  </Event>

Based on this event I get some Op codes and this and another event convert into the following:
Video Memory Responsiveness
Video Memory Degradation

This gave me an idea as to how I could perhaps test to see if one card was a problem. SiSoft Sandra has a GPU memory bandwidth test tool so I decided I would run that and potentially gain some insight. I expected to see perhaps one card outperforming the other pointing to a hardware related problem, something I could clearly RMA or perhaps I would just learn nothing. I did however learn everything I need to know about the problem to accurately diagnose its cause:

- What happens is if I run both cards together in the test I get the result I am expecting, its twice the bandwidth of a single card and the test is as expected.
- If I run the test against the first card all is normal, bandwidth is ~half of the dual card test.
- When I run the test on the second card the bandwidth figures are very similar to the first card and show no performance problems. However something else happens during the benchmark. All the performance problems I have in Windows disappear for a few seconds while the test is running. I can move windows normally, the mouse stops lagging and jumping around and everything is smooth as its meant to be. Then all of a sudden it hits a phase in the test and the mouse starts jumping centimetres across the screen, Window lag is at least 1 second behind the motion made to move it. Then the test ends and it all returns to the usual poor performance.

Yet in all this GPU-Z shows the GPU memory speed to be the same, GPU load to be zero and thus the running of the test should not impact anything. I now believe that GPU-Z and MSI Afterburner I being lied to by the drivers. There is some power saving going on with the second card that is not being reported, and its this power saving on the second card causing my issue. ULPS is off and I have forced the card awake by using MSI Afterburners alternative overclocking mode. There is no clearer pointer to it being related to the second card and more to the point that it is clearly power saving related as it should perform worse when the card is under extreme load not better. It is crossfire related and its certainly software driver related, specifically power saving in Windows rendering mode.

AMD and their crappy drivers are now quite evidently at fault. I have reported this to AMD but having given them months to fix it they wont even acknowledge there is a problem so I decided to make the knowledge of this problem more public. This is a massive performance problem in Windows Aero and every user will suffer from it. Not everyone will notice as its input latency related and causes about 250ms of additional latency and a low frame rate. Its a direct performance problem and it'll only happen when using crossfire.

AMD's supported couldn't have behaved worse in the tracking of this problem down, they did nothing but hinder me making progress by making me test pointless things that clearly weren't the problem. They asked me to test the same thing 4-5 times. Its taken me 5 months off and on to finally diagnose the direct cause of it and find a (all be it very temporary) workaround. AMD you need to fix that support team, its staffed by morons.

Now I have finally found the cause of the fault I can safely move to NVidia and get a working windows Aero again without it running at about 5 FPS.

 

[blackened23]

I'm not using 7970s anymore but when I was, I never had this problem. Aero performance was fine. With 12.3, I did notice that 2d clocks kicked in when youtube videos were playing in the background so you generally had to close browsers while gaming. Did you try the 2nd (not 1st) June 2012 beta drivers? Many users are reporting that they fixed all of the tri xfire issues previous drivers had. My other question is, who the hell cares about what the aero benchmarks say? Personally regardless of what card I have, I don't give a F. I didn't really notice any issues with aero being slow.

While I didn't really have issues - That said, these issues being reported with AMD cards is pretty insane. I really hope AMD puts their money where their mouth is, because they've stated they're increasing the size of their software development staff. I've always thought AMD makes pretty good hardware and its just ridiculous to see some of the potential lost by the hardware being held back by software.

 

[BrightCandle]

I'm not using 7970s anymore but when I was, I never had this problem. Aero performance was fine. With 12.3, I did notice that 2d clocks kicked in when youtube videos were playing in the background so you generally had to close browsers while gaming. Did you try the 2nd (not 1st) June 2012 beta drivers? Many users are reporting that they fixed all of the tri xfire issues previous drivers had. My other question is, who the hell cares about what the aero benchmarks say? Personally regardless of what card I have, I don't give a F. I didn't really notice any issues with aero being slow.

Personally I consider 250ms of unnecessary lag to be pretty important. But its certainly at the level where many users don't notice there is a problem. If you had that in a game however you would be screaming at AMD to fix it.

 

[lopri]

I honestly am not understanding the problem you "identified" in the first post. It seems to me what you're saying is that you're getting a lag while testing GPU memory? Am I understanding it correctly?

That's like saying that your system is not as responsive when you run Linpack. I have no idea how that is a "problem," and let alone how you measured 250ms of delay. (which could be caused by anything)

 

[lopri]

Out of curiosity I went through the steps you've taken - that is I checked out the Event Viewer, and found this. I was stress-testing the GPU at the time as well. (a single GPU, no SLI/CF)

I won't say which GPU this is but I can tell you its 2D performance is awesome.

 

[blackened23]

I honestly am not understanding the problem you "identified" in the first post. It seems to me what you're saying is that you're getting a lag while testing GPU memory? Am I understanding it correctly?

That's like saying that your system is not as responsive when you run Linpack. I have no idea how that is a "problem," and let alone how you measured 250ms of delay. (which could be caused by anything)

I wonder if it has anything to do with the popup that appears if you try to game with aero enabled. you'll get a popup saying that performance is degraded, do you want to keep aero - that happens on all cards. I haven't had an issue with aero performance though on either side of the fence, shrug.

 

[lopri]

I wonder if it has anything to do with the popup that appears if you try to game with aero enabled. you'll get a popup saying that performance is degraded, do you want to keep aero - that happens on all cards. I haven't had an issue with aero performance though on either side of the fence, shrug.

Yes it can happens to any video card on any system depending on workload and what Windows "thinks." But I read the OP as a very typical scenario and if anything that he sometimes doesn't get lag while running Sandra is interesting, not the other way around.

 

[Ferzerp]

I honestly am not understanding the problem you "identified" in the first post. It seems to me what you're saying is that you're getting a lag while testing GPU memory? Am I understanding it correctly?

That's like saying that your system is not as responsive when you run Linpack. I have no idea how that is a "problem," and let alone how you measured 250ms of delay. (which could be caused by anything)

The post is pretty clear, if a bit verbose.

Non-stressed : card is clocked too low to work in 2D
Stressed: card clocks increase, and 2D works fine
Workaround: do not allow card to reduce clock speed ever.

 

[BrightCandle]

The post is pretty clear, if a bit verbose.

Non-stressed : card is clocked too low to work in 2D
Stressed: card clocks increase, and 2D works fine
Workaround: do not allow card to reduce clock speed ever.

Exactly. I have just programmed a little program that connects via openCL to the second card and creates a queue for a kernel of work. Then it just goes to sleep. The card walks up, runs at full clock speed and all the lag in Windows is gone. Something is severely broken in AMD's drivers for crossfire when running Windows.

 

[lopri]

Ah. I see. His detailed focus on Sandra confused me. Well, if that's the case he could have tested the same thing with Wildlife.wmv.

Since it sounds like CF-specific (second card powering off when not used?), I will say no more.

 

[BrightCandle]

Ah. I see. His detailed focus on Sandra confused me. Well, if that's the case he could have tested the same thing with Wildlife.wmv.

Since it sounds like CF-specific (second card powering off when not used?), I will say no more.

The users who will suffer:
7000 series users
who use crossfire
and have a lot of screen space, ie eyefinity

Its related to the new power saving technology in the 7000 series cards. Even if you turn off ULPS it wont fix the underlying problem because the second card will hum along at like 300/150 instead of 925/1350 which isn't enough to render 3 screens.

If you overclock using MSI Afterburner in its unofficial overclocking mode GPU-Z will tell you the second card is running at the full clock speed, but it isn't the drivers are passing duff information. This also solves the problem users are having with crossfire and windows tearing and jolting, that is all performance related as well.

 

[Ferzerp]

Can you mod a bios and change the lower powered clock table entries to be a bit faster to avoid this?

 

[BrightCandle]

Can you mod a bios and change the lower powered clock table entries to be a bit faster to avoid this?

The 7970's don't yet have support in Radeon Bios Editor. I used the RBE on the 5970 I had before to increase the 2D voltage as the setting it came with caused it to crash when steam was open and the clock speeds were too low to run Youtube video smoothly. Another power saving snafu by AMD in a long history of them.

Honestly I am not surprised to find that AMD's new power saving is worse than the old one, they don't have any clue how to get this right.

 

[BrightCandle]

I am now solving the problem with a custom bit of software. All it does is attach to the second card using OpenCL and tell it work is coming - the card wakes up and runs at full speed. This removes the lag I was seeing constantly.

If you have a similar problem (only really any use for crossfire + eyefinity users with windows lag and tearing problems) then take a look at
https://github.com/PaulKeeble/7970OpenCLWorkaround
and then download
https://github.com/downloads/PaulKeeble/7970OpenCLWorkaround/2DBooster1.0.zip

 

[thilanliyan]

I am happy I never had issues with AMD cards, but then I never really ran XFire, even now with 2 6950s I just game on one of them.

Kudos to you for having solved it on your own. You have the knowledge to do it...unfortunately average people probably don't and AMD should fix these kinds of issues, especially for the average person who wouldn't know how.

 

[Ferzerp]

I am happy I never had issues with AMD cards, but then I never really ran XFire, even now with 2 6950s I just game on one of them.

Kudos to you for having solved it on your own. You have the knowledge to do it...unfortunately average people probably don't and AMD should fix these kinds of issues, especially for the average person who wouldn't know how.

The fix would be trivial too, but if their support is like most organizations, they have no mechanism for dealing with true issues that should be addressed by the driver/bios teams.

 

[BrightCandle]

Alas my custom software isn't reliably fixing the issue. I am getting odd mouse accelerations and slow downs. I have had enough, I just bought 2x 680 GTX's. I would never buy AMD again, not because their hardware isn't good or even that their drivers are bad but because the support is so terrible its beyond useless. This was an expensive waste of time and I should have replaced them earlier.

I warn anyway away from getting 7970 crossfire, there are crashing bugs, performance problems in 2D windows and AMD has no intention of fixing any of it, they wont even let you report a bug in it.

 

[SolMiester]

You are probably the 4 person on this forum I have read that has changed their 7970s for 680s..
You cant be in the States because apparently there is a supply issue!

 

[digitaldurandal]

Funny that they still have issues with their 2d clocks.. this is one of the reasons I went back to nvidia from my 5870.

Oh you want to run 3 monitors and overclock your video card? Well you will have to edit the BIOS to do that because when 2d clocks kick in.. good luck doing anything! TROLOLOLOL

 

[BrightCandle]

Funny that they still have issues with their 2d clocks.. this is one of the reasons I went back to nvidia from my 5870.

Oh you want to run 3 monitors and overclock your video card? Well you will have to edit the BIOS to do that because when 2d clocks kick in.. good luck doing anything! TROLOLOLOL

You know if it was just the 2D clocks I could live with it and fix it, but what I don't understand is why the second card has anything to do whatsoever. The second card according to AMD shouldn't be running in 2D mode, it should be sleeping ULPS. Instead what appears to be happening is every other frame is not getting rendered at all, and if you wake it up the second card the performance is just all over the place frame to frame (presumably the 2D equivalent of microstutter!!). Its unusable and I really do hope the 680's work in 2D as they are meant to.

I had problems with 2D clocks on my 5970 and the 4870X2 before it but never like this. I bought AMD this time because they were first and my 5970 was creaking, I should have taken the lack of driver support for my 5970 as a sign of what the company had turned into. Oh the hindsight.