tracert looks like an infinite loop pings expire in transit

[Mucman]

I am sure this is an easy one, but I am still in the midst of training and not sure what the solution is (or if it has to do with us)

209.153.207.x is our router and I am guessing 209.153.196.b is their router. The packets never get to the host at 209.153.196.a they
keep on cycling through their router.

C:\>tracert 209.153.196.a

Tracing route to dome.domain [209.153.196.a]
over a maximum of 30 hops:

1 <10 ms 10 ms <10 ms 209.153.207.x
2 <10 ms <10 ms 10 ms 209.153.196.b
3 <10 ms 10 ms 10 ms 209.153.196.b
4 10 ms 10 ms 30 ms 209.153.196.b
5 10 ms 10 ms 20 ms 209.153.196.b
6 10 ms 10 ms 20 ms 209.153.196.b
7 20 ms 20 ms 20 ms 209.153.196.b
8 20 ms 20 ms 20 ms 209.153.196.b
9 40 ms 20 ms 30 ms 209.153.196.b
10 30 ms 30 ms 30 ms 209.153.196.b
11 30 ms 40 ms 30 ms 209.153.196.b
12 40 ms 30 ms 30 ms 209.153.196.b
13 60 ms 50 ms 40 ms 209.153.196.b
14 40 ms 50 ms 40 ms 209.153.196.b
15 80 ms 50 ms 50 ms 209.153.196.b
16 50 ms 50 ms 50 ms 209.153.196.b
17 60 ms 61 ms 50 ms 209.153.196.b
18 51 ms 60 ms 50 ms 209.153.196.b
19 60 ms 60 ms 70 ms 209.153.196.b
20 60 ms 70 ms 80 ms 209.153.196.b
21 60 ms 70 ms 70 ms 209.153.196.b
22 70 ms 110 ms 80 ms 209.153.196.b
23 70 ms 80 ms 80 ms 209.153.196.b
24 80 ms 121 ms 80 ms 209.153.196.b
25 121 ms 130 ms 120 ms 209.153.196.b
26 80 ms 90 ms 80 ms 209.153.196.b
27 90 ms 120 ms 120 ms 209.153.196.b
28 90 ms 90 ms 111 ms 209.153.196.b
29 151 ms 100 ms 100 ms 209.153.196.b
30 90 ms 100 ms 110 ms 209.153.196.b

Trace complete.

A ping looks like this :

C:\>ping 209.153.196.a

Pinging 209.153.196.a with 32 bytes of data:

Reply from 209.153.196.b: TTL expired in transit.
Reply from 209.153.196.b: TTL expired in transit.
Reply from 209.153.196.b: TTL expired in transit.
Reply from 209.153.196.b: TTL expired in transit.

Ping statistics for 209.153.196.a:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 0ms, Maximum = 0ms, Average = 0ms

 

[shadow]

I take it you are hiding the last portion of the IP addresses with letters. Or are you really trying to ping an IP address with a letter in it? This brings me to my next point - do letters get resolved to bits?

Judging from what I know -
1)try pinging and tracerting something without a bloody letter in it. AFAIK letters are invalid (tried it myself just now), but of course, they may be getting resolved properly anyway.
2)IF the letters are properly resolving or you are hiding the last part, then what looks like it is happening is that the router is misconfigured. I'd guess that the default route is back at you - or there is a static route back at you - or you guys are laned somehow so that the packet the router sends out comes right back to it on another port. Check your cabling layout, and routing tables.

anyone correct me / have any other ideas?

 

[n0cmonkey]

Thier routing looks messed up. Have them check it. I usually see (pulling numbers off the top of my head)

43.43.43.43
43.43.43.44
43.43.43.43
43.43.43.44

stuff like that, but I think I have seen this particular problem a couple of times a few years ago. Anyhow, have them take a look at thier routing and maybe try from a different location.

 

[RagManX]

Every time I've seen this, the problem has been on the remote router. I would expect you can get this cleared up by having them check their router setup. I would not expect this to be a problem on your end.

RagManX

 

[ScottMac]

Try to get a physical map of the areas containing the affected routers, look for redundant links. Check the redundant links (assuming their existance) against the routing tables of the affected routers.

Check the configs on the routers, look to see that split horizon is turned on (and/or poison reverse) and that whichever is consistant across the affected paths.

It may be associated with the default parameters the Cisco (assuming Cicso infrastructure) assigns when using point-multipoint and point-to-point subinterfaces to frame relay setups (like multiple DLCIs, each with it's own subinterface). When you create a subinterface on a serial port, you must choose whether it's point-to-point, or point-to-multipoint. I believe if you choose point-to-point, split horizon defaults to off. If the other side of the connection is configured for point-to-multipoint, split horizon is on by default...depending on how your mesh is set up, you may be creating a loop.

Edit: I check out a couple things, this is the story:

If you create subinterfaces (like for multiple DLCIs / single physical interface), Split Horizon IS enabled by default. If you are using a single physical interface (no subs), Cisco defaults Split Horizon to OFF. So, if you have a central router, with multiple DLCIs (maybe one coming from each of several remote locations), and a remote location getting one DLCI on one physical interface, unless Split Horizon has been enabled at the remote location, it may advertise route back out the receiving interface and convince the other routers he's got a better path....creating your loop.

I may have some of the information swapped, but the basic jist is to check the router configs, verify that the settings (especially subinterface type and Split Horizon or Poison Reverse) agree for all of the affected routers.

If there's an ISDN backup link in the system, make sure it's not being triggered on (or at least if it is, the floating static route metrics are correct..i.e., slower path gets a higher metric).

I'll ask Doug/L3Guy to check out the post and see if he has any suggestions...he likes chasing routing bugs..... ;->

FWIW

Scott

 

[Mucman]

LOL, thanks for the replies guys! It turns out the company that the IP routes to was having a power outage , their power is back up and I just had to
reset something on the router and all is well.

ScottMac - I can't wait to have the knowledge to understand what you just said I am not the network admin here.... yet, I am just trying to
help out whenever I can.

<edit>I had way too many smileys </edit>

 

[Garion]

Actually, what you saw in your traceroute is a bit wierd. When something like that happens, you'd typically see a traceroute bouncing between two IP's - Each has the other as the route to that subnet (or as the default gateway). It's very unusual to see a single address show up more than once in a traceroute. Only time I could think of that happening would be a misconfiguration when the router actually has itself listed as a secondary route or has it's own interface listed as a default gateway (Not sure if you can do that?!?).

So, you were definitely asking a good question, an a very interesting one. I'd be curious exactly why it was happening.

- G

 

[Mucman]

Garion - Too bad this isn't something that I can repeat often . I wish I was further into my networking course so I could understand exactly what was going on. It really got
me inspired to read up a bit in my CCNA book!

What gets me is if they had a power outage why did we not just receive a host not found error? This company is the only T1 customer we have. We have a Cisco 3600 which goes into a Cisco manage switch (model number escapes me), and they have a port on our switch which goes to a router that we have as well (209.153.196.b). From there it sends the T1 to our customer (I do not know what kind of network they have on their end).

The 2nd router is an Ascend (again not sure of the model number).