Here's the scenario.



My operation center currently has a Cisco 3640 router connected into

port 2/48 of a Cisco 2980G-A switch.  Port 2/47 is connected to an SMC

hub.  The hub has connections going to our firewall, an Internet

monitor server (running Websense), and email monitoring server.



Over the past couple days I've received calls saying some of our

branch offices are intermittently dropping connections to applications

at the operation center.  I started a ping to the router at the remote

branch and noticed some random timeouts.  Our frame relay vendor

indicated there were no problems with their lines (typical response,

but this time they were correct).



Fast forward to today.  I was monitoring our switch and noticed the

following message every once in a while:



Console> (enable) 2004 Mar 25 13:12:57 %SYS-4-P2_WARN: 1/Host

00:03:e3:89:9f:81 is flapping between port 2/47 and port 2/48



That MAC address is for the ethernet port on our Cisco 3640 router.



I had a constant ping going to the router at the operation center (so

basically it's going from my computer ---> switch ---> ethernet

interface of the router) and noticed some timeouts when that message

would get logged on the switch.



Reply from 172.17.1.1: bytes=32 time<10ms TTL=255

Reply from 172.17.1.1: bytes=32 time<10ms TTL=255

Reply from 172.17.1.1: bytes=32 time<10ms TTL=255

Reply from 172.17.1.1: bytes=32 time<10ms TTL=255

Reply from 172.17.1.1: bytes=32 time<10ms TTL=255

Request timed out.

Request timed out.

Request timed out.

Request timed out.

Request timed out.

Request timed out.

Request timed out.

Request timed out.

Request timed out.

Request timed out.

Reply from 172.17.1.1: bytes=32 time<10ms TTL=255

Reply from 172.17.1.1: bytes=32 time<10ms TTL=255

Reply from 172.17.1.1: bytes=32 time<10ms TTL=255

Reply from 172.17.1.1: bytes=32 time<10ms TTL=255



I played around a little and noticed when I plugged the firewall

directly into port 2/47 and the internet monitor and email monitor

into other ports in the switch, the "Flapping" errors went away.  I

was also able to ping the ethernet interface of the router without any

problems.  (I did have an occasional timeout, but nothing even close

to what I posted above.)



Everything I've read points to possible problems with the spanning

tree on the switch.  What I can't understand is how would connecting a

hub into the switch all of a sudden cause these intermittent

connection errors as well as the flapping errors?  With our previous

setup (email monitor connected to a port that monitors the firewall

port) we had no problems whatsoever.



I guess any ideas are most welcome...this is really frustrating.  Oh

yea, one more thing.  Portfast is disabled for port 2/47 and 2/48.



Thanks - Jake

In article <f00681f7.0403251520.36d12e57@posting.google.com>,   jakes06

@comcast.net says...

> Here's the scenario.

> My operation center currently has a Cisco 3640 router connected into

> port 2/48 of a Cisco 2980G-A switch.  Port 2/47 is connected to an SMC

> hub.  The hub has connections going to our firewall, an Internet

> monitor server (running Websense), and email monitoring server.

[snip]

> Fast forward to today.  I was monitoring our switch and noticed the

> following message every once in a while:

>

> Console> (enable) 2004 Mar 25 13:12:57 %SYS-4-P2_WARN: 1/Host

> 00:03:e3:89:9f:81 is flapping between port 2/47 and port 2/48

>

> That MAC address is for the ethernet port on our Cisco 3640 router.

>

> I had a constant ping going to the router at the operation center (so

> basically it's going from my computer ---> switch ---> ethernet

> interface of the router) and noticed some timeouts when that message

> would get logged on the switch.

>

> Reply from 172.17.1.1: bytes=32 time<10ms TTL=255

> Request timed out.

[snip]

> Request timed out.

> Reply from 172.17.1.1: bytes=32 time<10ms TTL=255

>

> I played around a little and noticed when I plugged the firewall

> directly into port 2/47 and the internet monitor and email monitor

> into other ports in the switch, the "Flapping" errors went away.  I

> was also able to ping the ethernet interface of the router without any

> problems.  (I did have an occasional timeout, but nothing even close

> to what I posted above.)



Even an occasional timeout is not acceptable for this type of an

environment (unless the router was SO busy that it could not respond to

your pings.





> Everything I've read points to possible problems with the spanning

> tree on the switch.



The problem isn't spanning tree, but perhaps lack of spanning tree.





> What I can't understand is how would connecting a

> hub into the switch all of a sudden cause these intermittent

> connection errors as well as the flapping errors?



Is there a device acting as a bridge?



>  With our previous

> setup (email monitor connected to a port that monitors the firewall

> port) we had no problems whatsoever.

>

> I guess any ideas are most welcome...this is really frustrating.  Oh

> yea, one more thing.  Portfast is disabled for port 2/47 and 2/48.



Do you have multiple interfaces on the 3640?  Couple of things to try.

Disable proxy arp on the router interfaces and see what happens.  Under

the interface (int e0/0 for example) type in "no ip proxy-arp"



Also, check the router's log "sho log" to see what's going on on the

router.



--



hsb



"Somehow I imagined this experience would be more rewarding"  Calvin

***************  USE ROT13 TO SEE MY EMAIL ADDRESS  ****************

**************************************************  ******************

Due to the volume of email that I receive, I may not not be able to

reply to emails sent to my account.  Please post a followup instead.

**************************************************  ******************

Hansang Bae <uonr@alp.ee.pbz> wrote in message news:<MPG.1acf775facc39881989c7c@news-server.nyc.rr.com>...

> In article <f00681f7.0403251520.36d12e57@posting.google.com>,   jakes06

> @comcast.net says... 

>  [snip]

[color=blue] 

[color=blue] 





> The problem isn't spanning tree, but perhaps lack of spanning tree.

[color=blue] 

>

> Is there a device acting as a bridge?

> 



> Also, check the router's log "sho log" to see what's going on on the

> router.



You have a loop. (well _just_ maybe it's something subtle but this is

where to start).



1.

I like to have spanning tree fully operational on all infrastructure

connected ports, (i.e. All routers, hubs, bridges.) even if the

topology as designed is loop free.



So:-

Disable portfast on ALL infrastructure connected ports on the switch.

In your case disable portfast on 47 and 48, (others?)



2.

Find the undocumented network loop.





I had somilar "flapping" messages recently during a network

reconfiguration process. I suspect but did not confirm that

I created a loop with miss-configured trunks and inappropriate

portfast settings. It went away when I was finished with the work.

Thanks for the reply.  After reviewing all the possible elements of

this problem I have pinpointed and fixed the cause of the packet loss.



Websense has a setting enabled by default called "MAC Spoofing".  My

understanding is that when the Websense server responds to the user

for a blocked or filtered website it spoofs the sending MAC address.

In this case it was spoofing the MAC of our main router, the Cisco

3640.  When the switch saw traffic from this MAC address on port 2/48

AND 2/47 it was doing one or both of the following:



1.  Creating a temporary spanning loop.

2.  When the MAC was entering and leaving the switch on 2 different

ports numerous times in a matter of seconds it was causing the switch

to utilize 100% of it's processor.  This would cause the "Flapping"

errors as well as the packet loss.



What I meant by the occasional timeout was 51 timeouts during 227538

ping attempts.  I think that's pretty reasonable.



To answer your questions - we do not have a device acting as a bridge.

We also only have 1 T1 WIC in our 3640.  I did not run the "no ip

proxy-arp" command on the ethernet interface yet.  If I see any more

problems I'll check into it.  But, for now, it's smooth sailing.



Thanks - Jake



Hansang Bae <uonr@alp.ee.pbz> wrote in message news:<MPG.1acf775facc39881989c7c@news-server.nyc.rr.com>...

> In article <f00681f7.0403251520.36d12e57@posting.google.com>,   jakes06

> @comcast.net says... 

>  [snip] 

>  [snip] 

>

> Even an occasional timeout is not acceptable for this type of an

> environment (unless the router was SO busy that it could not respond to

> your pings.

>

> 

>

> The problem isn't spanning tree, but perhaps lack of spanning tree.

>

> 

>

> Is there a device acting as a bridge?

> 

>

> Do you have multiple interfaces on the 3640?  Couple of things to try.

> Disable proxy arp on the router interfaces and see what happens.  Under

> the interface (int e0/0 for example) type in "no ip proxy-arp"

>

> Also, check the router's log "sho log" to see what's going on on the

> router.

>

> --

>

> hsb

>

> "Somehow I imagined this experience would be more rewarding"  Calvin

> ***************  USE ROT13 TO SEE MY EMAIL ADDRESS  ****************

> **************************************************  ******************

> Due to the volume of email that I receive, I may not not be able to

> reply to emails sent to my account.  Please post a followup instead.

> **************************************************  ******************