PCIe dropping down to 60MHz

[Jovec]

Gigabyte GA-890GPA-UD3H 890GX mobo with a 1090t. Occasionally AMD Overdrive will report the PCIe speed as 60MHz. Tried setting the PCIe speed in BIOS to Auto and 100. Mostly seems to happen coming out of sleep but not confirmed. I'm worried this is a second symptom my mobo is defective as often my DEL keypresses are not registered to get into the bios during boot. I typically have to restart 4 or 5 times average to get into the bios.

Running an overclock at 3.7/2.8NB (multis only, no FSB/HTT oc) but it happens at stock too. Wondering if a bios update would fix it. Currently running the latest. Board is about 3 weeks old and it's a minor pain to send things back to the Egg.

 

[lopri]

My first instinct would be to ignore what AOD reports. Occasionally AOD reports my PCIe speed as 120 MHz, too. (instead of 100, or 60) It's got to be a bug in AOD. On a second thought, it also reports wrong vCore, vDIMM, vNB, and wrong fan speed.. etc. 😉

I'd not worry about it although there might be a better tool to measure correct PCIe frequency. Maybe Everest or Sandra would do it? I haven't really thought about it.

 

[Jovec]

Came out of sleep at 60MHz. Everest doesn't seem to have a free edition to check with, and Sandra Lite doesn't seem to report the PCIe speed directly. And just to quickly test, I ran a Sandra video benchmark at 60MHz and at 100MHz and there is a difference. I guess the PCIe bus is really dropping down to 60MHz.

60MHz PCIe

SiSoftware Sandra

Benchmark Results
Aggregate Memory Performance : 63.45GB/s
Internal Memory Bandwidth : 84.2GB/s
Data Transfer Bandwidth : 3GB/s
Results Interpretation : Higher scores are better.

Detailed Benchmark Results
Internal Memory Bandwidth : 84.2GB/s
Bandwidth Efficiency : 67.35%
System to Device Bandwidth : 3GB/s
Bandwidth Efficiency : 39.21%
Device to System Bandwidth : 2.54GB/s
Bandwidth Efficiency : 32.51%
Interface : STREAM
Results Interpretation : Higher scores are better.

Performance vs. Speed
Aggregate Memory Performance : 64.97MB/s/MHz
Internal Memory Bandwidth : 86.21MB/s/MHz
Data Transfer Bandwidth : 3.08MB/s/MHz
Results Interpretation : Higher scores are better.

Performance vs. Power
Device(s) Power : 170.00W
Aggregate Memory Performance : 382.20MB/s/W
Internal Memory Bandwidth : 507.14MB/s/W
Data Transfer Bandwidth : 18.14MB/s/W
Results Interpretation : Higher scores are better.

100MHz PCIe

SiSoftware Sandra

Benchmark Results
Aggregate Memory Performance : 66.42GB/s
Internal Memory Bandwidth : 84.32GB/s
Data Transfer Bandwidth : 5GB/s
Results Interpretation : Higher scores are better.

Detailed Benchmark Results
Internal Memory Bandwidth : 84.32GB/s
Bandwidth Efficiency : 67.46%
System to Device Bandwidth : 5GB/s
Bandwidth Efficiency : 65.21%
Device to System Bandwidth : 3.45GB/s
Bandwidth Efficiency : 44.15%
Interface : STREAM
Results Interpretation : Higher scores are better.

Performance vs. Speed
Aggregate Memory Performance : 68.02MB/s/MHz
Internal Memory Bandwidth : 86.35MB/s/MHz
Data Transfer Bandwidth : 5.05MB/s/MHz
Results Interpretation : Higher scores are better.

Performance vs. Power
Device(s) Power : 170.00W
Aggregate Memory Performance : 400.09MB/s/W
Internal Memory Bandwidth : 507.92MB/s/W
Data Transfer Bandwidth : 29.69MB/s/W
Results Interpretation : Higher scores are better.

 

[lopri]

Hmm.. So it does indeed lower the PCIe frequency, it seems.. Are you using onboard GPU? If not, make sure to turn it off. Do you have any other PCIe device other than a video card? Also, try setting the PCIe frequency manually @102MHz.

I wonder if it has anything to do turbo.. Doubtful but it shouldn't hurt to check. When in the BIOS, press "Ctrl+F1" (will reveal a few more controls) and see if there is anything like "PCIe Speed Spectrum" or even "SB750/SB850 Speed Spectrum". For overclocking those are advised to be turned off.

There is another tool to measure PCIe bandwidth, directly from AMD.

http://developer.amd.com/gpu/ATISTREAMPOWERTOY/Pages/default.aspx

It will open a command prompt window upon execution and it runs real fast, so you may want to take a screenshot and close the window. Otherwise it will keep trying higher values and system will become unresponsive. The test screen looks like this.



Oh, and one more thing. Since you mentioned S3 - What is your OS' power plan? High Performance, Balanced, or Power Saver? I'm sure you are aware of it but there are advanced options for each plan. Control Panel -> Power Options and if you click on "Advanced Options", then there is a "PCIe Link State" or something like that. See if changing that value makes a difference.

 

[Jovec]

More quick benchmarks:

@60
CineBench 11.5 OpenGL 3 run average: 57.25
Dawn of War II 3 run average, ultra settings: 56.54 avg / 123.91 max / 10.58 min

@100
CineBench 11.5 OpenGL 3 run average: 60.03
Dawn of War II 3 run average, ultra settings: 59.36 avg / 126.08 max / 14.25 min

Are you using onboard GPU? If not, make sure to turn it off. Do you have any other PCIe device other than a video card? Also, try setting the PCIe frequency manually @102MHz.

Running a 5850 at stock. Onboard video is disabled in the bios. Only PCIe device is the video card besides what's normally run off PCIe internally - no other add-in boards. I have the PCIe speed set at 100MHz in the bios, but maybe 101 or 102 might help to keep it there.

I wonder if it has anything to do turbo.. Doubtful but it shouldn't hurt to check. When in the BIOS, press "Ctrl+F1" (will reveal a few more controls) and see if there is anything like "PCIe Speed Spectrum" or even "SB750/SB850 Speed Spectrum". For overclocking those are advised to be turned off.

Turbo/Core Performance Boost is disabled. I'll double check the spread spectrum settings (edit: both were disabled) but I'm fairly sure they are disabled.

What is your OS' power plan? High Performance, Balanced, or Power Saver? I'm sure you are aware of it but there are advanced options for each plan. Control Panel -> Power Options and if you click on "Advanced Options", then there is a "PCIe Link State" or something like that. See if changing that value makes a difference.

Balanced, with the PCIe link state set to "moderate power savings." I'll research this more.

There is another tool to measure PCIe bandwidth, directly from AMD.

@60

===> Testing device 0 <===
Device type: Unknown
Max resource 2D width/height: 16384/16384
Total GPU memory size: 1024 MB
Total CPU cached space size: 2047 MB
Total CPU uncached space size: 2047 MB
GPU engine clock: 725 MHz
GPU memory clock: 1000 MHz
Number of timing loops: 100
[        16 bytes] CPU->GPU= 426.242 KB/sec, GPU->CPU= 156.964 KB/sec
[        32 bytes] CPU->GPU=   1.138 MB/sec, GPU->CPU=   1.259 MB/sec
[        64 bytes] CPU->GPU=   2.585 MB/sec, GPU->CPU= 911.957 KB/sec
[       128 bytes] CPU->GPU=   5.081 MB/sec, GPU->CPU=   5.162 MB/sec
[       256 bytes] CPU->GPU=  10.434 MB/sec, GPU->CPU=  10.286 MB/sec
[       512 bytes] CPU->GPU=  20.826 MB/sec, GPU->CPU=  20.104 MB/sec
[      1024 bytes] CPU->GPU=  41.869 MB/sec, GPU->CPU=  41.271 MB/sec
[      2048 bytes] CPU->GPU=  37.695 MB/sec, GPU->CPU=  81.210 MB/sec
[      4096 bytes] CPU->GPU= 164.240 MB/sec, GPU->CPU= 165.415 MB/sec
[      8192 bytes] CPU->GPU= 328.480 MB/sec, GPU->CPU= 331.163 MB/sec
[     16384 bytes] CPU->GPU= 649.893 MB/sec, GPU->CPU= 640.263 MB/sec
[     32768 bytes] CPU->GPU=   1.359 GB/sec, GPU->CPU=   1.317 GB/sec
[     65536 bytes] CPU->GPU=   1.896 GB/sec, GPU->CPU=   1.854 GB/sec
[    131072 bytes] CPU->GPU=   2.392 GB/sec, GPU->CPU=   2.644 GB/sec
[    262144 bytes] CPU->GPU=   2.714 GB/sec, GPU->CPU=   2.924 GB/sec
[    524288 bytes] CPU->GPU=   2.939 GB/sec, GPU->CPU=   3.092 GB/sec
[   1048576 bytes] CPU->GPU=   3.025 GB/sec, GPU->CPU=   3.202 GB/sec
[   2097152 bytes] CPU->GPU=   3.109 GB/sec, GPU->CPU=   3.266 GB/sec
[   4194304 bytes] CPU->GPU=   3.161 GB/sec, GPU->CPU=   3.305 GB/sec
[   8388608 bytes] CPU->GPU=   3.176 GB/sec, GPU->CPU=   3.317 GB/sec
[  16777216 bytes] CPU->GPU=   3.187 GB/sec, GPU->CPU=   3.329 GB/sec
[  33554432 bytes] CPU->GPU=   3.195 GB/sec, GPU->CPU=   3.341 GB/sec
[  67108864 bytes] CPU->GPU=   3.195 GB/sec, GPU->CPU=   3.342 GB/sec

@100

===> Testing device 0 <===
Device type: Unknown
Max resource 2D width/height: 16384/16384
Total GPU memory size: 1024 MB
Total CPU cached space size: 2047 MB
Total CPU uncached space size: 2047 MB
GPU engine clock: 725 MHz
GPU memory clock: 1000 MHz
Number of timing loops: 100
[        16 bytes] CPU->GPU= 383.694 KB/sec, GPU->CPU= 151.716 KB/sec
[        32 bytes] CPU->GPU=   1.276 MB/sec, GPU->CPU=   1.272 MB/sec
[        64 bytes] CPU->GPU=   2.306 MB/sec, GPU->CPU= 775.365 KB/sec
[       128 bytes] CPU->GPU=   5.173 MB/sec, GPU->CPU=   4.930 MB/sec
[       256 bytes] CPU->GPU=  10.220 MB/sec, GPU->CPU=  10.345 MB/sec
[       512 bytes] CPU->GPU=  21.003 MB/sec, GPU->CPU=  19.218 MB/sec
[      1024 bytes] CPU->GPU=  42.136 MB/sec, GPU->CPU=  41.345 MB/sec
[      2048 bytes] CPU->GPU=  84.909 MB/sec, GPU->CPU=  83.898 MB/sec
[      4096 bytes] CPU->GPU= 167.190 MB/sec, GPU->CPU= 168.121 MB/sec
[      8192 bytes] CPU->GPU= 327.535 MB/sec, GPU->CPU= 334.986 MB/sec
[     16384 bytes] CPU->GPU= 657.617 MB/sec, GPU->CPU= 660.331 MB/sec
[     32768 bytes] CPU->GPU=   1.304 GB/sec, GPU->CPU=   1.329 GB/sec
[     65536 bytes] CPU->GPU=   2.499 GB/sec, GPU->CPU=   2.649 GB/sec
[    131072 bytes] CPU->GPU=   3.364 GB/sec, GPU->CPU=   3.796 GB/sec
[    262144 bytes] CPU->GPU=   4.142 GB/sec, GPU->CPU=   4.465 GB/sec
[    524288 bytes] CPU->GPU=   4.548 GB/sec, GPU->CPU=   4.793 GB/sec
[   1048576 bytes] CPU->GPU=   4.820 GB/sec, GPU->CPU=   5.249 GB/sec
[   2097152 bytes] CPU->GPU=   5.010 GB/sec, GPU->CPU=   5.566 GB/sec
[   4194304 bytes] CPU->GPU=   5.092 GB/sec, GPU->CPU=   5.672 GB/sec
[   8388608 bytes] CPU->GPU=   5.154 GB/sec, GPU->CPU=   5.754 GB/sec
[  16777216 bytes] CPU->GPU=   5.168 GB/sec, GPU->CPU=   5.732 GB/sec
[  33554432 bytes] CPU->GPU=   5.211 GB/sec, GPU->CPU=   5.677 GB/sec
[  67108864 bytes] CPU->GPU=   5.193 GB/sec, GPU->CPU=   5.618 GB/sec

 

[lopri]

Well, it's bizzare. I've seen reports of PCIe lane shrinking (x4 or x8 instead of x16) but this is something new to me. Another suggestion I'd make is to uninstall AOD and set the PCIe frequency manually (@100 or 102)

 

[Jovec]

Manually set the PCIe speed to 101 in Bios. Sleep/Resume and now it reports 60.6.

Also just noticed that there is a rev 2.1 on this mobo.... not necessarily a good sign to have 3 revs in 4-5 months.

As mentioned previously, board has trouble entering bios.

Set the HT multi to auto and the board is running it at x11 instead of x10. Don't know if it's a bug or by design (in an attempt to better match the OC).

Win7 x64 install was originally on a different mobo (790gx DDR2 board). If I had a spare HDD I'd try a fresh install just in case. I might have an IDE one somewhere.

I've got a ticket in with Gigabyte since yesterday but no response. Update: GB replied with the generic "remove everything but 1 stick of RAM" response in an attempt to get past the system "hanging" (which is isn't). Probably need to work the problem past the initial techs. I might try reflashing the bios.

 

[lopri]

HT multi changing to x11 when left auto while overclocking is not uncommon. Especially if you overclock CPU-NB. The board is actually meant to support up to 2600 MHz HT Link.

I think RMA is in order. If your PCIe is dropping to 60 MHz, there is a possibility your network and USB speed are affected as well.

P.S. If NewEgg is nice to your RMA request and possibly replace the board with something else, check out some of the ASUS boards.

 

[Jovec]

Tech support exchange with Gigabyte. They tried to be helpful, but didn't seem to accept the fact that I was getting measurable speed differences between 60MHz and 100MHz. They claim that at 60Mhz I would experience severe GPU problems at 60MHz - that might be true, but I think the few benchmarks I ran showed that something was causing a slowdown.

Read from the bottom up.

Answer : Dear Customer

Even if the PCIe clock reading changes to 60MHz, this has no physical impact on the system and will not degrade the system performance whatsoever.


-----


Question - 941667
From : Jovec
Sent : 7/2/2010 03:56
Question : Yes, I can move the PCIe speed slider in AOD to fix this. When AOD reports 60, the slider sits at 100, so I have to move it up slightly to 101, hit apply, then back down to 100 and apply again for it to take effect.

Your measured results do not match with mine.

Cinebench 11.5 OpenGL
Dawn of War 2
AMD PCIe Speed Test
Sandra Lite

All show a statistically meaningful difference in results between 60MHz and 100MHz that AOD reports. While the PCIe bus may not be running at 60MHz per your oscilloscope testing, something is causing a slowdown. I fear that this my be indicative of further problems in the future. As such, I have RMA'ed the motherboard back to Newegg.

We can consider this ticket closed. Thank you for your time.

User Attached : CPUZ-CPU.jpg, CPUZ-Mainboard.jpg


-----


Answer - 940713
Answer : Dear Customer

Even though AOD reads the PCIe clock at 60Mhz after S3 resume, we tested by measuring the PCIe clock using an oscilloscope and the PCIe Clock is actually at 100MHz and did not drop to 60MHz (if it was 60Mhz the card would have serious problems operating).

Please try this procedure, after resuming from S3 the PCIe will show as 60MHz now please adjust the level bar and know you will see the PCIe Clock default back to 100MHz.


-----


Question - 940713
From : Jovec
Sent : 6/30/2010 02:10
Question : All settings set to Auto in Bios. When the system boots, Overdrive shows 100MHz PCIe. Enter sleep, then resume, now the PCIe bus is at 60MHz. As previously mentioned, I ran 4 different benchmarks that show a speed difference so this is not a case where the software is reporting an incorrect value.

Please note that you must also close and restart Overdrive. If the computer enters sleep with Overdrive still running, after the computer resumes Overdrive will still show 100MHz.

User Attached : BoardStatus.jpg, NovicePerformance.jpg


-----


Answer - 939504
Answer : Dear Customer

Our checked this issue and after resume from sleep in Win7 64bit AMD overdrive shows that PCIe still runs at 100MHz. Can you provide us with some screenshots of your test results


-----


Question - 939504
From : Jovec
Sent : 6/26/2010 11:52
Question : The system does not hang. The system is automatically downclocking the PCIe bus to 60MHz when it comes out of S3 sleep. You can read this thread here I created for more information.

http://forums.anandtech.com/showthread.php?t=2085197

I ran benchmarks and the PCIe bus is indeed running at 60MHz. There is an obvious speed difference in the benchmarks between 60MHz and 100MHz. This is not a case where software is incorrectly reporting the PCIe bus value.

I just tried setting the PCIe speed to 101 in the bios. Upon resuming, the bus is now running at "60.6"


-----

Answer - 938977
Answer : Hi,

Please remove all components from the motherboard except for the following:

- cpu along with heatsinkfan
- one stick of memory
- videocard or onboard video

If it still hangs at this point then you will need to swap out these components one at a time. If no issue please reinsert back the remaining components back into your motherboard one at a time to determine which may be the cause of the issue.

As for PCIE it will auto detect and is locked, whether if the software is registering correctly we are unable to determined


-----


Question - 938977
From : Jovec
Sent : 6/25/2010 05:08
Question : 1) Motherboard only occasionally recognizes DELete key presses to enter BIOS during boot and restarts. When it doesn't, I reset and try again. I might have to do this 4 or 5 times before I can get into the BIOS. Microsoft USB keyboard that previously had no problems when used with my 790GX Gigabyte motherboard.

2) AMD Overdrive will sometimes show the PCIe speed at 60MHZ instead of 100MHz. I have set the PCIe speed setting in the BIOS to both auto and manually to 100MHz.

Both of these issues happen with stock settings and overclocked settings.


-------------------------------------------------------------------------------
Model Name : GA-890GPA-UD3H(rev. 2.0)
--------------------------
M/B Rev : 2.0
BIOS Ver : FC
Serial No. :
Purchase Dealer : Newegg
-------------------------------------------------------------------------------
VGA Brand : ATi Model : 5850
CPU Brand : AMD Model : 1090t Speed : 3.2
Operation System : Win 7 64-bit SP :
Memory Brand : G.Skill Type : DD3
Memory Size : 8GB (4x2GB) Speed : 1600
Power Supply : 700 W

 

[Lazlo Panaflex]

When in the BIOS, press "Ctrl+F1" (will reveal a few more controls) and see if there is anything like "PCIe Speed Spectrum" or even "SB750/SB850 Speed Spectrum". For overclocking those are advised to be turned off.

Dude, thanks for that info...I thought GB finally did away with that stupid secret squirrel shit! WTF 😱

 

[Jovec]

I picked up a second GA-890GPA-UD3H on a whim and this board exhibits the same behavior. Both are Rev2.0 boards. Same hardware in an Asus M4A89GTD PRO/USB3 doesn't exhibit this problem.

 

[6arrep]

Same here with my Gigabyte GA-890FXA-UD7 :thumbsdown:

AOD helps me to force PCIe between 100-110, then it's ok up to next sleep 😡