Slow or Laggy LAN

[SolMiester]

Hi guys, I try to manage a 2003 AD LAN which is attached to a WAN of 5 other locations around the country. All our LAN PC's are fairly new, P4 3Ghz machines, however the files server is only a dual Plll 1Ghz with RAID5, the DC is a P4 Xeon, a Dual Xeon (4 core) SQL database server, Dual Xeon Terminal Server and an another Dual Plll 1Ghz print server with a couple of apps running.

Now we have some bespoke software which query's the print server when running a certain client app on the PC, however apart from that, I dont see why the network shares etc appear at times to lag, eg: saving a word or excel file back up to a mapped share. Would this be due to the old Dual Plll server, its only shareing?

We have some no name switches, but they hardly look overrun to me and are only 1/2 loaded in the ports. There are 3, 2 x 24 and 1 x 32. the 2 24's plug into the 32 which has the servers and the WAN connectivity.

The TS is usually only used remotely except for some accounting software that PC's use occasionally.

The SQL server is quite new, 4 cores with 2gb RAM hosting 2 main 1 Gb databases, one is data mining which runs every 2 hours, the other bespoke practice software, which used to run on the old file server.

Oulook 2003 is quite slow sometimes, with stored mail in pst files and a intranet database hosted on the SQL server. Mail is exchange cache mode from the WAN.

The boss being an accountant is complaining stating 1 min x every user is lost time blah blah blah, and I'm not sure if we need a new file server/DC, or new switches.

I have had sniffers check the LAN wiring for packet errors etc, but its all good, so I'm abit lost as to where to go next.

Any suggestion greatly appreciated.

Greg


PS: , Task manager shows little load on any of the server and the network traffic is also low.....switches?, client browser service?

 

[spidey07]

Exact, detailed, precise description of the symptom, what is the action being performed, in detail are needed.

Off my head, sounds like a DNS problem with AD or something to do with name resolution. It's very common for a misconfiguration on AD and the clients to cause this.

 

[SolMiester]

Hi spidey, thats for the quik reply....cant see this as a DNS issue, this is saving back onto a mapped drive. It isnt really an issue that I can put a finger on either, sorry to be vague. Its like when you know you have slowish broadband and browse....

If we change the selected printer to a different one on the printer server, after selecting the printer there can be a 10+ wait for the printer to be ready to hit the properties button etc.

If browsing the mapped drive, selecting deeper into the folder etc, sometimes the right pane contecnts disappears for a second or 2 to refresh, file list contents appears laggy. Do you know what I mean?.

I guess I'm truing to say, the nothing is snappy considering all pc's are new, and we are hardly moving large files across the network.

I used to think it was the wiring as when on the TS, it all appeared faster, however we had samples of the cat5 checked and all good!?

 

[spidey07]

Need MUCH, MUCH more detail.

How long, what is the action being perfromed? Navigating a mapped drive to a server that is on the same local LAN as the client? Navigating to a mapped drive in a program (they are different actions). How long?

Still sounds DNS related. Most all "AD browsing/navigation is slow" is DNS.

WANS are slow. If you are getting your DNS across a WAN, not a good idea unless it is a high speed link. If I were to guess, somebody didn't put any thought into how to make AD/DNS work with a WAN properly if you are seeing a few seconds of delay. especially if you are accessing resources/share across a wan link.

 

[SolMiester]

spidey, this is all LAN access, nothing to do with WAN, well, rather, internet and e-mail come from the WAN, but, the slowness is usually saving to a mapped drive from the application, especially excel and word, up to 20 - 30 secs

We have local DNS, when I started there, they didnt have a DC and DNS was from the WAN, but we resolved that several years ago. Now we have print server instead of TCP/IP direct local mapped printers. We also have WINS service running.

 

[spidey07]

Originally posted by: SolMiester
spidey, this is all LAN access, nothing to do with WAN, well, rather, internet and e-mail come from the WAN, but, the slowness is usually saving to a mapped drive from the application, especially excel and word, up to 20 - 30 secs

We have local DNS, when I started there, they didnt have a DC and DNS was from the WAN, but we resolved that several years ago. Now we have print server instead of TCP/IP direct local mapped printers. We also have WINS service running.

Ahhh, you hit the magic number. 15 and/or 30 seconds.

You have a DNS/AD configuration problem. You're gonna have to thoroughly root through your AD hierarchy and DNS to see what is up. Good idea to take a look at how the clients are configured as well to see how they are doing name resolution. An actuall sniffer trace of the clients will make this easy to really see what is going on. Run it on a client.

 

[SolMiester]

Ta spidey, its certainly not every client and not all the time....thats why i'm struggling to find the cause or reason.
I am lost as to what could be wrong with the DNS/AD structure, the WAN site have different IP ranges, and this isnt something that has happened overnight but appears to have gradually got worse.

what sort of sniffer software would you recommend and how does it work, remembering this is a small business of 35 local users.

 

[spidey07]

Originally posted by: SolMiester
Ta spidey, its certainly not every client and not all the time....thats why i'm struggling to find the cause or reason.
I am lost as to what could be wrong with the DNS/AD structure, the WAN site have different IP ranges, and this isnt something that has happened overnight but appears to have gradually got worse.

what sort of sniffer software would you recommend and how does it work, remembering this is a small business of 35 local users.

I'm not trying to be mean, but if your compass didn't IMMEDIATELY point to a name resolution problem and if you don't understand how this REAKs of name resolution then it might be best if you have somebody else fix this.

wireshark is a free sniffer. There's also good links on how to troubleshoot AD/DNS problems on microsoft's site.

The only other thing I can think of if there is some kind of firewall/access control lists between the servers in your AD.

 

[SolMiester]

Spidey, dont worrys mate, no offense taken, I'm no MSCE, I'm just a jack of all trades bud, who learnt his connectivity on Novell 3.12....LOL. But the pays good, and mostly the boss reckons the sun shines out of my arse!....haha

 

[spidey07]

Originally posted by: SolMiester
Spidey, dont worrys mate, no offense taken, I'm no MSCE, I'm just a jack of all trades bud, who learnt his connectivity on Novell 3.12....LOL. But the pays good, and mostly the boss reckons the sun shines out of my arse!....haha

Oh!

that explains it then. You have IPX running on the clients and/or servers and still have a name resolution/binding problem.

 

[SolMiester]

sidey, just 1 more question mate, as we have DNS, do you think its okay to remove netbios from tcp/ip in the dhcp scope. isnt it an added overhead? same with with computer browser, is that really needed when only the servers have shares?

 

[spidey07]

Originally posted by: SolMiester
sidey, just 1 more question mate, as we have DNS, do you think its okay to remove netbios from tcp/ip in the dhcp scope. isnt it an added overhead? same with with computer browser, is that really needed when only the servers have shares?

that depends on a whole slew of things. Can't be answered in a forum. But that just points more towards your name resolution problem - clients and servers not configured correctly for name resolution (netbios/wins vs. dns).

 

[SolMiester]

Well spidey, I will have a look, however I am 95% positive, this issue is not DNS related, partly because this DC was added after the original WAN was created and alot of the setting migrated from the GCS and primary server, the issue is not sudden, not consistent, and the hierarchy was already designed by MSCE engineers, there are no DNS event error and all static address and the reverse lookup resolve correctly.

I was almost positive it would be switch bottleneck (poor quality swicthes as they are cheap replacements) and was hoping someone would confirm...LOL

 

[spidey07]

Originally posted by: SolMiester
Well spidey, I will have a look, however I am 95% positive, this issue is not DNS related, partly because this DC was added after the original WAN was created and alot of the setting migrated from the GCS and primary server, the issue is not sudden, not consistent, and the hierarchy was already designed by MSCE engineers, there are no DNS event error and all static address and the reverse lookup resolve correctly.

I was almost positive it would be switch bottleneck (poor quality swicthes as they are cheap replacements) and was hoping someone would confirm...LOL

Ok. You can believe what you want to believe. It REAKS, STINKS, SEEN IT A FEW HUNDRED TIMES of a name resolution/ad configuration problem.

Load up wireshark on a client, start the capture, do the problem action and look at the trace. Here's what you will see....

1) client asks for name and/or netbios name (depending on config of client)
2) nobody answers, again due to misconfiguration of the client or server
3) timeout for that name resolution procedure eventually occurs, again this depends on the servers, the protocols, the bindings, setup of AD, etc.
4) client finally tries another name resolution action, gets what it's looking for, proceeds as usual. You'll see the tell-tale 30 seconds of the timeout and the client will switch name resolution procedures. All of this behavior is controlled by the config of the client, servers and active directory.

I am literally NOT kidding when I say I've seen this at least 100 times. When you said 30 seconds it's immediately obvious what the problem is, name resolution.

 

[SolMiester]

Spidey, been running 5m, and my nvidia NF4 network card is like 30000 bad checksum error?!!!....WTF, dont tell me my pc (non standard of course) is saturating the network?!

 

[spidey07]

Originally posted by: SolMiester
Spidey, been running 5m, and my nvidia NF4 network card is like 30000 bad checksum error?!!!....WTF, dont tell me my pc (non standard of course) is saturating the network?!

possible, could be a driver problem.

A sniffer is your best friend for solving any "network" problem. What you'll find is..

it's NOT a NETWORK problem!!!!

 

[SolMiester]

Just tested on another client, and that too has quite a concerning amount of bad checksum, though no where near as bad as me.....?

So, will check drivers for NICs on server and myself!!

 

[SolMiester]

Interesting, I have a little D-Link switch in my office which I use due to the need for extra connections sometimes, removed that and connected directly to wall node and the bad checksums have reduced dramatically....

 

[robmurphy]

Could it be that the checksum errors are a sign that you are using Cat3 cabling to the client machines?

No name switches rairly allow you to set the speed and duplex of the link. This will leave the speed and duplex to be auto negotiated. Cat 3 looks just like cat 5 to the switches and NICs, and they will auto negotiate to 100 FD. This is usualy where CRC errors start to come in.

On previous project I worked on if the cabling was Cat3 then the ports on the switch were set to 10 FD.

One other think to look out for is that some NICs allow the PC to offload the TCP/IP checksum calculation to the NIC. When using wireshark this will show the packets as having a bad checksum. The checksum offload is quite common on gigabit NICs. On the Marvell Yukon on my present laptop this offload is referred to as "Hardware Checksumming".

Rob

 

[Madwand1]

AFAIK, the reported checksum errors could just be an artifact of how Wireshark interacts with the system, and not real errors. Turning off checksum offloading in this case should get rid of the messages.

To test this further, you could do transfer speed measurements with and without checksum offloading. If both rates are fast and around the same, then the checksum errors probably weren't real errors.

 

[robmurphy]

I did not mean the offload caused real errors, all I meant was that wireshark shows that the TCP/IP checksum is wrong when you offload the checksum calculation. It does make some comment aboult checksum offloading when you look at the capture.

Rob.