Slow log in for remote offices

[thirdeye]

I've been having a problem with our WAN ever since I demoted 2 of our domain controllers.
I originally had 2 W2003 domain controllers and 2 W2000 domain controllers (one of these being our terminal server). I was noticing a lot of error messages pointing to the 2000 and 2003 domain controllers not communicating so in attempt to make everything happy I demoted both 2000 DCs and enabled AD in w2k3 native mode.

This when I noticed logons taking a long time (sitting at applying settings screen for an excess of 10 minutes) but it only affected the remote offices who are tied into our network via Cisco VPN set up before my hiring (I also know about this much -><- when it comes to Cisco VPNs).

I then figured it may have had something to do demoting the terminal server (this is how 90% of the remote office work gets done anyways) so I upgraded to W2003 and installed AD (this was last Friday). Initially all was well and much praise was had...until today. Everything seems to back to the slowness that was happening immediately after the demotion.

Any suggestions? I'm stuck.

 

[spidey07]

where are the logon servers for these remote users?

You may also be having fragmentation problems with the VPN - you can try setting the clients MTU to something like 1300 as a quick check to see if thats it.

I'm not much of a windows/AD guy but I do know you need to setup these remote subnets in your DNS and tell them where their logon server is.

 

[thirdeye]

The logon servers are located in the main building, the remote offices are between 15 and 30 miles (one is on the same ISP the other is on a diff. ISP)

As for setting the MTU for the client (I did check the PIX and the max is set to 1380), the clients don't actually use a VPN client.

We have a cisco 806 router that is configured to connect to our PIX so the clients don't have to use a vpn client to connect to our network. They are just on a separate subnet. I checked in DNS and it seems to be configured alright but I'll look into that in a little more depth.

I appreciate the help and anything else you think might work or I should look into would be awesome.

 

[spidey07]

well start pinging the clients with 1480 bytes and see if still have a fragmentation problem.

Its pretty common with VPNs and one of the first things to rule out.

Otherwise how big are the internet connections? Do you have any congestion/general slowness?

 

[JRock]

How is the performace once they are able to login successfully?

If its fine after the initial login why not try caching the user login credentials or a better solution in the larger scheme of things put a GC at each location. This way if your WAN links ever go dowe the remote office can still login.

I think I am following the thread correctly if not please correct me.

 

[stash]

You need some data before you start shooting in the dark. What do the application logs on these remote machines say? Any userenv errors? Netlogon errors in the system logs?

How about your domain controller logs? Have you run dcdiag and netdiag on these machines?

Also based on this

was noticing a lot of error messages pointing to the 2000 and 2003 domain controllers not communicating so in attempt to make everything happy I demoted both 2000 DCs and enabled AD in w2k3 native mode.

, your whole issue may be simply DNS. 2000 and 2003 DCs will communicate just fine, there wasn't any need to make that drastic a change.

 

[thirdeye]

^^It wasn't a dns issue (or at least at the time) it was a TON of SAM database errors about not being able to lock out the administrator account. From searching around it seemed to be an issue of the 2 different domain controllers. Which is why I did what I did to begin with. Since then the errors stopped.

 

[thirdeye]

Pinging with a 1480 byte ping results in a 2-3x higher ping, in one instance the ping actually failed.

The bandwidth should be more than adequate we have a 3mb/1.5mb business cable here and i believe a 1.5/1.5 business there.

 

[stash]

Sounds like UDP kerberos fragmentation. Check out the section on that here: http://www.microsoft.com/technet/prodte...03/technologies/security/tkerberr.mspx

You can try forcing Kerb to use TCP: http://support.microsoft.com/kb/244474

The SAM database (I'm guessing EventID 12294) is either from a malicious user trying guess a password, or (more likely) a malicious process, usually Randex. The attempts to guess the password are so frequent, the hardware can't keep up and is unable to lock the account when the lockout threshold is met. See http://support.microsoft.com/kb/887433

 

[nweaver]

I have found long login times to be related to DNS problems. Is the main location and remote location useing the same DNS server? check if nslookup can resolve the domain name (both as just "domain" and as the FQDN).

 

[thirdeye]

Originally posted by: STaSh
Sounds like UDP kerberos fragmentation. Check out the section on that here: http://www.microsoft.com/technet/prodte...03/technologies/security/tkerberr.mspx

You can try forcing Kerb to use TCP: http://support.microsoft.com/kb/244474

The SAM database (I'm guessing EventID 12294) is either from a malicious user trying guess a password, or (more likely) a malicious process, usually Randex. The attempts to guess the password are so frequent, the hardware can't keep up and is unable to lock the account when the lockout threshold is met. See http://support.microsoft.com/kb/887433

I'll look into the kerberos thing, but the SAM database problem is gone since I've switched to Native 2003 mode. It literally went from an error every 3-5 seconds to 1 every week since the switch.

 

[stash]

Possibly because one of the servers you demoted had a virus on it. Just speculating.

 

[thirdeye]

Ok so after running dcdiag on all 3 DCs and netdiag on all three there is only one thing that looks out of place.

This is what my one DC is reporting on dcdiag:

DC Diagnosis

Performing initial setup:
Done gathering initial info.

Doing initial non skippeable tests

Testing server: Default-First-Site-Name\W2KDC3
Starting test: Connectivity
W2KDC3's server GUID DNS name could not be resolved to an
IP address. Check the DNS server, DHCP, server name, etc
Although the Guid DNS name

(af007c9e-f019-4668-a943-634855e2aead._msdcs.Joinder.local) couldn't

be resolved, the server name (w2kdc3.Joinder.local) resolved to the IP

address (172.17.2.251) and was pingable. Check that the IP address is

registered correctly with the DNS server.
......................... W2KDC3 failed test Connectivity

Doing primary tests

Testing server: Default-First-Site-Name\W2KDC3
Skipping all tests, because server W2KDC3 is
not responding to directory service requests

Running enterprise tests on : Joinder.local
Starting test: Intersite
......................... Joinder.local passed test Intersite
Starting test: FsmoCheck
......................... Joinder.local passed test FsmoCheck