Can't connect MS Remote Desktop to ONE Server

[RebateMonger]

Edit:
Problem solved, with help from stash and spidey07. Disabling and then re-enabling ISA 2004's VPN Client Access fixed the problem.

Edit2:
Well, not completely solved. Turning off the SBS/ISA VPN Client Service fixes the problem. But if I turn the VPN Service back on, the problem eventually returns.

I routinely connect to several Windows Server 2003 sites using the MS Remote Desktop client. But I can't connect to my only Windows Server 2000 site. I CAN connect to that remote site from my other Servers at other sites. From my other sites, I open up the RDP client, put in the IP address, and it connects immediately. But from MY server, the RDP request consistently times out. ("The client could not connect to the remote computer. Remote connections might not be enabled or the computer might be too busy to accept new connections.....").

I'm trying to connect from my SBS 2003 Server, SP1, with ISA 2004, SP2, installed. The remote site is Windows Server 2000, SP4, with a Linksys WRT54G router.

It's probably an ISA issue, since if I replace the SBS Server with my laptop, I can RDP to that Windows Server 2000 site from the laptop just fine. So it's not a problem with network firewalls between my Server and the remote Windows 2000 Server. It's not a DNS issue, either, since I'm using the Server 2000's IP address for initiating the connection.

I've traced the connection sequence using ISA Server's logging function, but it's not clear why the connection is failing. If I simply open up a DOS box and do a "Telnet xxx.xxx.xxx.xxx 3389" there's no response from the destination server. If I do the same thing (Telnet over Port 3389) to OTHER remote Servers, I get a response.

Two other bits of weirdness:
1) This connection problem works BOTH ways. I can't RDP or Telnet from the remote Windows Server 2000 to MY SBS Server, either. I CAN connect to other Servers. Just not to MY Server.
2) My ISA Server seems to think that the responses to my RDP request are "IKE Client" protocol. As soon as the RDP connection is initiated by my Server, ISA shows an immediate "IKE Client" packet (UDP Port 500) coming back from the remote Windows Server 2000. This is obviously wrong, and may be a key to the problem.

As I mentioned, it's probably an ISA issue. But I'm REALLY confused why I can RDP to a dozen other remote Servers, but I can't RDP to this ONE Windows Server 2000 site.

Any suggestions or things to look at would be appreciated. Thanks!66

 

[stash]

Did you get a sniff with wireshark or something similar...it should give you more detail than the ISA logging.

 

[RebateMonger]

Originally posted by: stash
Did you get a sniff with wireshark or something similar...it should give you more detail than the ISA logging.

No, I haven't done that. I'll probably have to. Can I just use NetMon on my SBS Server to monitor the external NIC traffic?

I've been working on this dumb problem for two days (unpaid) now. It's driving me crazy that I can't solve it.

Edit:
I've installed Microsoft's Network Monitor (NetMon) on the SBS 2003 Server and I'm looking at the traffic on the Internet-side NIC, both with "successful" RDP logons (to a Win2003 Server) and "unsuccessful" RDP logons (those to the troublesome Win2000 Server).

In the "unsuccessful" logon, it looks like there are only five frames sent between the two Servers:
1.81 sec. From SBS to Win2000
1.99 sec From Win2000 to SBS
1.99 sec From SBS to Win2000
4.88 sec From SBS to Win2000
10.9 sec from SBS to Win2000

The "successful" logon has tons more packets exchanged.

I BARELY know what I'm doing when it comes to sniffing......

 

[stash]

Yep netmon should work fine.

 

[RebateMonger]

Comparing the only the first few packets exchanged, they are TOTALLY different.

"Successful" logon:
SBS sends a TCP packet addressed to port 3389 on the Remote Server
Remote Server sends a TCP packet in reply
Conversation continues, setting up RDP connection.....

"Unsuccessful" logon:
1.81 sec - SBS sends a TCP request that's to TCP Port 3389 on Server2000
1.99 sec - Win2000 sends a UDP response, ISAKMP (500)
(It's a proposal for a Security Association.)
1.99 sec - SBS then tries to respond to this ISAKMP request....
(it's a "No-Proposal-Chosen" response, so there won't be a Security Association set up.)
4.88 sec - ISA tries another Port 3389 request, which is ignored by the Server 2000....
(It's all over in two-tenths of a second. The two Servers have stopped talking to each other.)

Hey, at least I'm getting a chance to view my other traffic, the Vonage SIP packets. I've never looked at them before.
And, boy, there sure are a lot of Broadcase ARP Requests floating around on my Cable Modem network. All neighborhood "Home Accounts", judging by their IP addresses.

Tomorrow, I'll try doing an RDP connection to one of the XP Professional client computers on the Windows Server 2000 network. It'll be interesting to see if THEY respond correctly to my SBS Server's request. For some reason, the Server 2000 doesn't like my requests.

 

[RebateMonger]

My SBS 2003 Server recorded 39 occurences of the following Critical Error in the Security Log. These are related to my many failed RDP connection attempts.

IKE security association establishment failed because peer sent invalid proposal. Mode: Key Exchange Mode (Main Mode) Filter: Source IP Address xxx.xxx.xxx.xxx Source IP Address Mask 255.255.255.255 Destination IP Address yyy.yyy.yyy.yyy Destination IP Address Mask 255.255.255.255 Protocol 0 Source Port 0 Destination Port 0 IKE Local Addr xxx,xxx,xxx,xxx IKE Peer Addr yyy.yyy.yyy.yyy IKE Source Port 500 IKE Destination Port 500 Peer Private Addr Attribute: Authentication Method Expected value: RSA Signature with Certificates Received value: Kerberos (GSSAPI)

Where
xxx.xxx.xxx.xxx = SBS 2003 Server (my Server)
yyy.yyy.yyy.yyy = Win2000 Server (remote Server)

 

[stash]

So I'm trying to understand your infrastructure a little better. You have a site with SBS running ISA and another site with a 2000 server with a Linksys router in front of it. Is that correct?

Is there any VPN tunnel setup between the two sites?

 

[RebateMonger]

Originally posted by: stash
So I'm trying to understand your infrastructure a little better. You have a site with SBS running ISA and another site with a 2000 server with a Linksys router in front of it. Is that correct?

Yes. The remote site is a Doctor's office with a simple Linksys router forwarding TCP Port 3389, a Windows Server 2000 set up for RDP in Administrative Mode, and no other hardware firewalls.

Is there any VPN tunnel setup between the two sites?

There's no VPN relationship between my SBS Server and the Windows Server 2000. I don't believe there's been any previous remote access to this server. But I'll take a look at RRAS and make sure.

Edit:
I just looked at RRAS on the remote Windows Server 2000. It hasn't been configured, so there's no Windows VPN set up. There's nothing that should be asking for a VPN connection.

The really weird thing is that I can access the Windows Server 2000 using RDP from ANY other site, including other SBS 2003 Servers with ISA 2004 installed. And I CAN access it from my site, if I take the SBS/ISA Server out of the loop and connect directly to the Internet with my laptop.

 

[spidey07]

from the traces it looks like the remote server thinks your source IP address is a VPN connection. Hence the reply with ISAKMP.

check the routing table on the remote server.

 

[stash]

Did you check the RRAS/VPN config on ISA to make sure it isn't enabled or misconfigured? I agree with Spidey, it definitely sounds like the ISA box is sending some IKE packets, or at least the 2000 server thinks it is.

 

[RebateMonger]

Originally posted by: spidey07
from the traces it looks like the remote server thinks your source IP address is a VPN connection. Hence the reply with ISAKMP.

But if that was the case, why can I replace my SBS box with an XP laptop and get an instant RDP connection from my site?

check the routing table on the remote server.

This is the result of a "Route Print" command on the remote server. Is that what you wanted?

Active Routes:
Network Destination-----Netmask---------Gateway---------Interface---------Metric
0.0.0.0-------------------0.0.0.0----------172.16.3.1--------172.16.3.100------1
127.0.0.0----------------255.0.0.0--------127.0.0.1---------127.0.0.1----------1
172.16.3.0------------255.255.255.0 ----172.16.3.100 ----172.16.3.100-----1
172.16.3.100--------255.255.255.255---127.0.0.1---------127.0.0.1---------1
172.16.255.255-----255.255.255.255----172.16.3.100-----172.16.3.100-----1
224.0.0.0--------------224.0.0.0----------172.16.3.100------172.16.3.100----1
255.255.255.255---255.255.255.255---172.16.3.100-----172.16.3.100-----1
Default Gateway: 172.16.3.1
Persistent Routes: None

The Windows 2000 Server is at 172.16.3.100 and the Linksys Router is at 172.16.3.1

 

[spidey07]

pm me and I'll give you my e-mail.

I'll take a look at the traces. I'll need a "good" one and one that doesn't work.

routing table looks good, but I don't know how MS handles VPNs, like if there is a VPN configured would it show up in the routing tables.

a good TCP handshake should be...

src - syn
dst - syn,ack
src - ack

socket is now open/established. the remote end is not sending an appropriate syn,ack from your post.

as far as replacing the box with XP and it works fine, are the IP addresses the same?

 

[RebateMonger]

Originally posted by: stash
Did you check the RRAS/VPN config on ISA to make sure it isn't enabled or misconfigured? I agree with Spidey, it definitely sounds like the ISA box is sending some IKE packets, or at least the 2000 server thinks it is.

I almost missed your comment. My SBS Server here IS configured to RECEIVE Microsoft VPN connections. But so are a couple of my other SBS/ISA Servers. And I'm guessing I only have it set up to receive PPTP connections.

But, just in case, I'll turn off my ISA VPN Service on my SBS Server, since I'm not using it right now. I'll let you know what happens.

Thanks to both spidey07 and stash for your help and suggestions!

 

[RebateMonger]

Originally posted by: spidey07
as far as replacing the box with XP and it works fine, are the IP addresses the same?

Identical.

 

[RebateMonger]

Success!

I turned OFF ISA's "Enable VPN Client Access" on my SBS/ISA Server and now I can connect from the Server!

I see that I'd actually enabled both PPTP and L2TP VPN client access to my SBS Server, even though I don't normally use L2TP on my own VPN.

When I saw both your notes about MY VPN services, I started thinking....Most of my other SBS Servers have only PPTP-protocol VPNs enabled, if they have VPN enabled at all.

Actually, I now turned L2TP VPN acces back on in ISA, and NOW everything works fine! It looks like ISA might have somehow gotten misconfigured and turning VPN acces off and then on again fixed it. Now it works, even with L2TP VPN access turned back on. It's weird that this wasn't causing problems with my Server 2003 sites. It was only a problem with the Windows 2000 Server site. :Q

You guys are great. Thanks!
Now I can get back to "paid" work. This was driving me crazy!!

 

[spidey07]

I told you it was doing VPN. neener, neener, neener.

On a lighter note, you really should learn how to read traces or at least take a look at them. It will really help your understanding of what is going on.

packet traces don't lie or mislead. they pinpoint exactly what is happening and with experience it will solve any problem.

 

[RebateMonger]

Originally posted by: spidey07
I told you it was doing VPN. neener, neener, neener.

LOL.

But why in the world would that remote server respond to my RDP request to TCP Port 3389 with an offer for a VPN connection? Doesn't that seem a bit strange? But, who knows what my SBS/ISA box was actually sending? Apparently my Server 2003 boxes weren't fooled, but the Server 2000 box got confused.

 

[stash]

If you still have traces, I wouldn't mind taking a look. Glad everything is working again.

 

[spidey07]

Originally posted by: RebateMonger

Originally posted by: spidey07
I told you it was doing VPN. neener, neener, neener.

LOL.

But why in the world would that remote server respond to my RDP request to TCP Port 3389 with an offer for a VPN connection? Doesn't that seem a bit strange? But, who knows what my SBS/ISA box was actually sending? Apparently my Server 2003 boxes weren't fooled, but the Server 2000 box got confused.

send me the traces and I'll tell you why

 

[RebateMonger]

AAARRGH!

A couple of hours later, and RDP to that Win2000 site has stopped working!

When I (re)discovered the problem, I tried turning off just the L2TP portion of the ISA VPN. That didn't fix it.
So then I COMPLETELY turned off the VPN Client Service. Now I can RDP to that remote Windows Server 2000 again.

There's sometimes lag in the response of ISA to configuration changes. I'm going to turn the PPTP VPN Client Service back on and wait a bit and see what happens. It's obvious that the L2TP VPN Client option kills RDP connections to that remote Win2000 Server.