DPC latency issues as of the last few weeks ...lots of them!

[d3fu5i0n]

Hi, guys,

Over the past few weeks, I've had several people complain about me sounding like a 'robot' at times during calls on a TS3 server.
At first I thought that since it hadn't really appeared elsewhere that it wasn't me.
So, last night, I decided to check my DPC latency and noticed this consistent pattern:

Looking into the details further with LatencyMon, I found this:

A long time ago, I used to hit about 800 micro-seconds but now it's considerably more and I'm not sure what to do with this particular set of issues -- there are quite a lot.

The OS is Windows 8.1 (Update 1) 64-bit/x86_64

The system specs (that matter) are as follows:
Motherboard: P8P67 Deluxe B3
Storage:
- Samsung 840 Pro 512GB SSD
- Samsung 830 128GB SSD
- Seagate 4TB NAS HDD
- OCZ Vertex 2 (3.5") 240GB SSD
- LG CH10LS20 BD Combo ODD

With the motherboard, I do have the Renesas USB 3 controller enabled and basically every rear USB 2.0 I/O utilised on the back of the motherboard.

The 840 Pro and 830 are running on the two native SATA 3 controllers via the Intel P67 southbridge, whilst the LG BD Combo drive is running on the native SATA 2 connection, followed lastly by the Vertex 2 and the Seagate NAS drives, which are running on the Marvell SATA 3 controller.

So, I have tested disabling the JMicron controller, the Renesas USB 3 controller, the Maxwell SATA 3 controller, along with FireWire 1394 and the on-board audio, which are already both disabled ...and this has changed nothing.

As you should be able to see from the attached images, the complete list is:
* USBPORT.SYS [USB 1.1/2.0]
* nvlddmkm.sys [nVidia]
* Wdf01000.sys
* storport.sys
* afd.sys
* dxgkrnl.sys
* ntoskrnl.exe
* CLASSPNP.sys
* ndis.sys

* tcpip.sys is a little rough, but nowhere near the others.

* ataport.sys doesn't have a high DPC count but does have a very high ISR count.

Usually I can diagnose things blindfolded but I'm truly lost this time around and hope that someone out there can get to the bottom of this with me.

Thank you in advance!

- Alistair

 

[Ketchup]

I like seeing what you have covered so far. As you probably know, it makes it a lot easier to look for other avenues.

One thing I didn't see mentioned is software updates. This may be as easy as going to the control panel and sort by date. Look for anything installed between now and when it was working properly. For example, I have seen where the Google updater can cause issues like this. Look at device driver updates as well.

I also saw a link that may help. I haven't read all of it:
http://www.sysnative.com/forums/win...-high-dpc-latency-issue-sound-distortion.html

 

[Elixer]

What happens if you use an older nvidia driver?
I would also start to unplug all USB devices except keyboard & mouse, then start adding them one by one, and see if you can find the issue that way.

 

[d3fu5i0n]

I like seeing what you have covered so far. As you probably know, it makes it a lot easier to look for other avenues.

One thing I didn't see mentioned is software updates. This may be as easy as going to the control panel and sort by date. Look for anything installed between now and when it was working properly. For example, I have seen where the Google updater can cause issues like this. Look at device driver updates as well.

I also saw a link that may help. I haven't read all of it:
http://www.sysnative.com/forums/win...-high-dpc-latency-issue-sound-distortion.html

Thanks.
Well, I always keep things updated -- everything; browsers, the OS, general applications, media players, video filter plugins, drivers (including NICs and chipset drivers).

It's just that, compared to most searches I've performed, I have many more system components with high latencies.

 

[d3fu5i0n]

What happens if you use an older nvidia driver?
I would also start to unplug all USB devices except keyboard & mouse, then start adding them one by one, and see if you can find the issue that way.

Yeah, I got around to that not long after posting ...and with no luck.
I removed every USB device -- nothing changed.

Older nVidia drivers don't seem to be an issue, at least the ones of recent times; and I can't really afford to be using older ones due to game-specific driver releases.

 

[SeanFL]

This looks very similar to a pattern I was seeing on a Lenovo Thinkpad after the network driver was updated. Once I disabled the LAN (gigabit) and turned off wireless, the spikes went away. Searching further on the Lenovo forums, I found a number of people that do professional audio work and were having trouble with newer drivers for the LAN.

It's worth disabling them inside windows and seeing if the spikes go away. Device manger, select the hardware, properties, and disable.

Hope you find what's causing it. Sean

 

[d3fu5i0n]

This looks very similar to a pattern I was seeing on a Lenovo Thinkpad after the network driver was updated. Once I disabled the LAN (gigabit) and turned off wireless, the spikes went away. Searching further on the Lenovo forums, I found a number of people that do professional audio work and were having trouble with newer drivers for the LAN.

It's worth disabling them inside windows and seeing if the spikes go away. Device manger, select the hardware, properties, and disable.

Hope you find what's causing it. Sean

I disabled both of the NICs on the hardware level as well; I should've mentioned that before.

The issue was still precisely the same.
It all seems to be occurring at the driver level.
I blame Windows. I'm waiting for the day when I can happily move to Linux, given how much of a *nix fan I am.

 

[d3fu5i0n]

So, this is interesting.
I played around with disabling lots and lots of NIC drivers, rather than purely relying on disabling things on the hardware level and I encountered this:

The Realtek NIC doesn't have the spikes but the Intel one does, even if I leave the driver enabled for the Intel one.
So, it seems that the Intel NIC drivers are a bit borked.
At least I'm not getting the massive spikes ...but everything is still a bit high.

So, tackling the other issues?
USBPORT.SYS is the worst culprit.
...or is it? Am I reading LatencyMon incorrectly; or is it badly presenting the details?

...and oh, whilst writing this, storport.sys just spiked up out of nowhere, whilst ndis.sys was calmly sitting at about 533 micro-seconds.

 

[Rhonda the Sly]

From DPC Latency Checker's website:

Windows 8 Compatibility: The DPC latency utility runs on Windows 8 but does not show correct values. The output suggests that the Windows 8 kernel performs badly and introduces a constant latency of one millisecond, which is not the case in practice. DPCs in the Windows 8 kernel behave identical to Windows 7. The utility produces incorrect results because the implementation of kernel timers has changed in Windows 8, which causes a side effect with the measuring algorithm used by the utility. Thesycon is working on a new version of the DPC latency utility and will make it available on this site as soon as it is finished.

 

[d3fu5i0n]

From DPC Latency Checker's website:

Windows 8 Compatibility: The DPC latency utility runs on Windows 8 but does not show correct values. The output suggests that the Windows 8 kernel performs badly and introduces a constant latency of one millisecond, which is not the case in practice. DPCs in the Windows 8 kernel behave identical to Windows 7. The utility produces incorrect results because the implementation of kernel timers has changed in Windows 8, which causes a side effect with the measuring algorithm used by the utility. Thesycon is working on a new version of the DPC latency utility and will make it available on this site as soon as it is finished.

Thank you for mentioning this!

Well, in all honesty, I was confused, as LatencyMon looked rather clean after moving to the Realtek NIC.

I've moved everything from the Marvell SATA 3 controller off to the on-board [Intel chipset] SATA 2 connectors.
The Vertex 2 is a SATA 2 drive anyway and the NAS-grade drive, whilst SATA 3-capable, isn't going to gain much benefit anyway. Also, it'll lower the POST time, so it's all well and good.

 

[Ketchup]

What's in your USB ports? Because when I run the test I show pretty much nothing on usbport.sys. I don't know if that is your problem, as there are a lot of things showing higher numbers than on mine, but it is one that really sticks out to me.

Edit: the input from Rhonda the Sly sounds about right.

 

[d3fu5i0n]

What's in your USB ports? Because when I run the test I show pretty much nothing on usbport.sys. I don't know if that is your problem, as there are a lot of things showing higher numbers than on mine, but it is one that really sticks out to me.

Rhonda the Sky said it perfectly.
Quoting the site, "Windows 8 Compatibility: The DPC latency utility runs on Windows 8 but does not show correct values. The output suggests that the Windows 8 kernel performs badly and introduces a constant latency of one millisecond, which is not the case in practice."
1000-ish micro-seconds == 1 millisecond.
So that's exactly what it was showing.

The main culprits were the Intel NIC and the Marvell add-on SATA 3 controller.

I'm still a bit unsure bit I'll keep an eye on things for the time being and test them.


Edit: I see that you've edited too.

 

[BonzaiDuck]

Rhonda the Sky said it perfectly.
Quoting the site, "Windows 8 Compatibility: The DPC latency utility runs on Windows 8 but does not show correct values. The output suggests that the Windows 8 kernel performs badly and introduces a constant latency of one millisecond, which is not the case in practice."
1000-ish micro-seconds == 1 millisecond.
So that's exactly what it was showing.

The main culprits were the Intel NIC and the Marvell add-on SATA 3 controller.

I'm still a bit unsure bit I'll keep an eye on things for the time being and test them.


Edit: I see that you've edited too.

That sort of conflict is similar to what I'd had on my WHS-2011 box, and the DPCs/Interrupts were causing one core to load up to 100%. I also used latency monitor, and it immediately turned my attention to a conflict arising from the nForce storage driver (this is an old NVidia motherboard).

With the ASUS P67 motherboard, I'd see what happens when you simply disable the Marvell controller. Do you REALLY NEED IT? I'd advise that you turn off every motherboard feature which you don't need or don't use. Then have a look-see again.

 

[d3fu5i0n]

That sort of conflict is similar to what I'd had on my WHS-2011 box, and the DPCs/Interrupts were causing one core to load up to 100%. I also used latency monitor, and it immediately turned my attention to a conflict arising from the nForce storage driver (this is an old NVidia motherboard).

With the ASUS P67 motherboard, I'd see what happens when you simply disable the Marvell controller. Do you REALLY NEED IT? I'd advise that you turn off every motherboard feature which you don't need or don't use. Then have a look-see again.

Oh, I don't really need it I guess; I always do prefer going native but there was a thought process at the time.
Still, the Marvell controller was only a small issue; the Intel NIC driver was the biggest problem.
...and yes, I do disable what I don't need/use.

Still, I ought to send a bug report to Intel.

 

[BonzaiDuck]

Oh, I don't really need it I guess; I always do prefer going native but there was a thought process at the time.
Still, the Marvell controller was only a small issue; the Intel NIC driver was the biggest problem.
...and yes, I do disable what I don't need/use.

Still, I ought to send a bug report to Intel.

Well, I think someone else suggested dropping back to an earlier version. As I recall it, the Device Manager interface for the Intel NIC is fairly polished; it could be something related to the settings. But usually, the default settings picked at install time shouldn't leave you with such a problem.

And by all means -- send them a bug report. Was this the motherboard NIC, or an add-in Intel "Pro" card?

It's always nice if you have a PCI-E or PCI Gbit NIC handy to see if the onboard hardware and driver was a problem. In my case, I had two motherboard NICs, disabled them both (nForce) and put an Intel Pro card in an available socket.

 

[Ketchup]

That sort of conflict is similar to what I'd had on my WHS-2011 box, and the DPCs/Interrupts were causing one core to load up to 100%. I also used latency monitor, and it immediately turned my attention to a conflict arising from the nForce storage driver (this is an old NVidia motherboard).

With the ASUS P67 motherboard, I'd see what happens when you simply disable the Marvell controller. Do you REALLY NEED IT? I'd advise that you turn off every motherboard feature which you don't need or don't use. Then have a look-see again.

Good advice. The Marvell controllers are almost always there just to advertise features, and do not perform nearly as well as the Intel controller devices, and in may cases other features suffer for it.

So what is your status? Are you back to a normal voice? I am surprised about the Intel LAN. I currently have the Realtek LAN connected on my board. The Intel LAN is installed, just happened that my cord found the Realtek connection first