[11985] in bugtraq
Re: Nmap and Cisco Dos, clarification --
daemon@ATHENA.MIT.EDU (Lisa Napier)
Sun Sep 26 01:10:24 1999
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed
Message-Id: <4.2.0.58.19990923171636.00c83a90@twoguys>
Date: Thu, 23 Sep 1999 17:30:54 -0700
Reply-To: Lisa Napier <lnapier@CISCO.COM>
From: Lisa Napier <lnapier@CISCO.COM>
X-To: "Lancashire, Andrew" <LancashireA@SUTTERHEALTH.ORG>,
BUGTRAQ@SECURITYFOCUS.COM
To: BUGTRAQ@SECURITYFOCUS.COM
In-Reply-To: <E96A622162DED211831F0008C75DDD7B011BB5AC@gpsc1dx.sutterhea
lth.org>
Hi Andrew,
The message you are quoting from me was my attempt to refute the possible
causes suggested by another poster, not as a suggestion for the reasoning
behind the nmap problem. I was attempting to indicate that I did not
think that ARP was the problem, but was waiting for further testing and
details, which I did not have at the time.
Apologies, as I was not clear in my message. I believe my later follow up
message of 9/15/99 is exactly in-line with what Kenny had tested in house.
Just further clarification.
Thank you,
Lisa Napier
Product Security Incident Response Team
Cisco Systems
At 01:44 PM 9/22/1999 -0700, Lancashire, Andrew wrote:
>This is to clarify what is being put out by Cisco and what we are being told
>by Cisco.
>
>Two e-mails below is what Cisco is telling us and makes allot more sense
>than what Cisco is telling Bugtraq. The last post to Bugtraq made mention
>that the arp cache was filling up and allocating memory for both reachable
>hosts and unreachable hosts (incompletes). Although what Lisa describes is
>true regarding the arp cache, it would not be true for our or most other
>sane persons environment. Since routers will only arp for what is local,
>that would mean that for the arp cache to fill up and us all the memory all
>networks in the 10.x.x.x range would need to be local. So that's not gonna
>happen but if you read the e-mail below that from Kenny (also at Cisco ) his
>explanation makes allot more sense considering we have hundreds of routers.
>
>Thank You
>
>Andrew
>
>P.S. Congratulations on the re-opening of PacketStorm
>___________________________________
>
>
>Subject: Re: Cisco and Nmap Dos
>To: BUGTRAQ@SECURITYFOCUS.COM
>
>
>Hi all,
>
>
>I wanted to address the items listed here. We are still investigating this
>problem, and hope to have more information later on in the week.
>
>
>Item 1, OSPF is not an issue. According to the configuration information
>provided to us by the customer, OSPF is not in use.
>
>
>Items 2, 3 & 4. IOS actually handles ARP in the following manner:
>
>
>When we receive a packet destined for something not already in our ARP
>table, we enter an "incomplete" entry in the ARP table. Then we will rate
>limit ARPs to once every 2 seconds to that destination. Any additional
>packets to that same destination will be dropped until the ARP entry is
>completed. This is on a per destination basis.
>
>
>"Incompletes" (ARP requests that have not been responded to) get dropped
>after approximately 1 minute from the last time we sent an ARP request for
>that non existent address.
>
>
>Incomplete entries MAY stay around longer, as the process that is
>responsible for cleaning up the ARP table may not have enough time to
>complete before it is called again, if we have a lot to clean up, which may
>be relevant to this case. The incomplete entries will eventually get
>cleaned up, but it may take two or three minutes, two or three cycles of the
>process that cleans up the table.
>
>
>Under a dedicated, intense nmap scan, a very large number of ARP requests
>may be generated, causing the ARP table to grow very large with "incomplete"
>entries. These entries consume memory. As the amount of free memory
>declines and demand on the processor to handle outstanding packets
>increases, ARP processing falls behind and throughput on the router may
>decline significantly. Once the scan is stopped, processing catches up and
>things return to normal.
>
>
>So, to my knowledge IOS should be doing the right thing, we only queue one
>ARP request at a time, every 2 seconds, until the ARP entry is resolved, we
>rate limit requests, dropping all additional packets, until the ARP entry is
>resolved, and we clean up the outstanding incomplete requests within a few
>minutes.
>
>
>I hope that helps address some of the concerns put forth. Hopefully we will
>have further updates shortly.
>
>
>Thank you,
>
>
>Lisa Napier
>Product Security Incident Response Team
>Cisco Systems
>___________
>
>_______________
>
>-----Original Message-----
>From: khollis [SMTP:khollis@cisco.com]
>Sent: Wednesday, September 15, 1999 11:31 AM
>To: wescotd@sutterhealth.org
>Subject: Regarding Case Number V44290
>
>Hello Dave, I've done some testing here with Nmap. I was able to create a
>test bed that can cause problems & symptoms similar to those you reported.
>But in summary, the router is functioning normally & depending on the
>network topology the behavior you experienced would be expected.
>
> >From show processor memory, the ip input process is the process that
>maintains the ip fast switching cache. This fast switching cache is used
>when forwarding packets to avoid interrupting the cpu for repetitive route
>table lookups. The key issue is the behavior of the fast cache and the way
>it gets built.
>
>There are a number of scenarios that will cause the fast cache to install an
>entry that essentially looks like a host route. For instance, with only 1
>path to a destination, we will install an entry into the fast cache that
>covers the entire network. Example: 100.0.0.0/8. However, when multiple
>equal cost paths to a destination exist, we will install an entry into the
>fast cache for each destination. Example: 100.0.0.1/32, 100.1.1.1/32,
>100.2.2.2/32...and so on. This helps ensure load balancing. Additionally,
>depending on whether routes are directly connected, and/or subnetted, or the
>next hop of a static route, the results can vary.
>
>When running Nmap & scanning every address in a class A ip network, if
>conditions warrant the installation of a /32 entry into the fast cache this
>would allow the fast cache to consume a tremendous amount of memory and in
>that scenario all available dram could be consumed. This creates additional
>problems because there isn't enough memory to support other features on the
>router.
>
>Since Nmap isn't a normal application ran on networks, this issue isn't a
>concern in most networking environments. However, if this is a major concern
>you could run CEF (Cisco Express Forwarding). The behavior I just explained
>does not occur when running CEF. But you will need to run 12.0 on the Cat5
>RSM. Other workarounds such as disabling fast switching (no ip route-cache)
>or using max-paths 1 aren't really feasible. CEF is the better solution.
>
>Thanks,
>KennyH.
>
>_________________
