Anyone done windows kernel debugging?

[Merad]

Messing with this for a work project. I have a Vmware VM set up with kernel debugging enabled on a serial port that outputs to a named pipe. Both Visual Studio and WinDbg can see the pipe, but I get an access denied error when I try to connect. Running them as administrator doesn't make a difference. Any idea what kind of permissions I need to set up to make this work?

 

[slugg]

What's giving you an access denied error? Is the host OS (that is, the machine that VMware is running inside of) giving you the error, or is the guest OS (that is, the machine that is running inside of VMware) denying it in some way?

It has been many, many years since I've touched anything even close to this. I haven't actually done any kernel debugging, but if memory serves me right, you need to create the pipe with ACL (access control list) rules. The ACL will determine security policy for both inbound and outbound access. This is true for all systems, not just Windows, although terminology and other details may differ. If I had to look somewhere, this would be the first place I look.

I like to get the easy, obvious stuff out of the way, first. Let us know if there's any more info to work with. Good luck!

 

[Z15CAM]

Like your thinking Slugg and I'm a hardwired desktop nerd;o)

 

[Merad]

Thanks for the answers - no clue what the problem with the pipes is, but I eventually got network debug working (VM is not hosted locally).

 

[Elizine]

To perform kernel-mode debugging in Microsoft Visual Studio:

1. On the host computer, in Visual Studio, from the Tools Menu, choose Attach to Process.
2. In the Attach to Process dialog box, set Transport to Windows Kernel Mode Debugger, and set Qualifier to the name of a previously configured target computer. For information about configuring a target computer.
3. Click Attach.

For setting up kernel-mode debugging please refer - https://msdn.microsoft.com/en-us/library/windows/hardware/hh439376(v=vs.85).aspx