[112710] in North American Network Operators' Group
Re: Netflow on SUP720-3BXL
daemon@ATHENA.MIT.EDU (Andy Bierlair)
Sat Mar 14 22:20:35 2009
In-Reply-To: <6069A203FD01884885C037F81DD75080CA0CA730@wsc-mail-01.intra.nwresd.k12.or.us>
Date: Sun, 15 Mar 2009 03:20:20 +0100
From: Andy Bierlair <globichen@gmail.com>
To: nanog@nanog.org
Errors-To: nanog-bounces@nanog.org
yes ip cef, this is enabled:
IP fast switching is enabled
IP fast switching on the same interface is disabled
IP Flow switching is enabled
IP CEF switching is enabled
IP Flow switching turbo vector
IP Flow CEF switching turbo vector
and so on...
-
Andy
On Sun, Mar 15, 2009 at 3:08 AM, Bill Blackford
<BBlackford@nwresd.k12.or.us> wrote:
>
> just a shot in the dark. Do you have 'ip cef' in global config?
>
> -b
> ________________________________________
> From: Andy Bierlair [globichen@gmail.com]
> Sent: Saturday, March 14, 2009 6:55 PM
> To: nanog@nanog.org
> Subject: Netflow on SUP720-3BXL
>
> I=92m trying to run netflow on one of our Cisco core routers (SUP720-3BXL=
),
> but I think I am hitting some limitations because of this:
>
>
>
> %EARL_NETFLOW-SP-4-TCAM_THRLD: Netflow TCAM threshold exceeded, TCAM
> Utilization [99%]
>
>
>
> The setup of netflow looks like this:
>
>
>
> =A0ip flow-cache entries 524288
>
> =A0mls aging fast time 5 threshold 32
>
> =A0mls aging long 300
>
> =A0mls aging normal 60
>
> =A0mls netflow usage notify 80 300
>
> =A0mls flow ip full
>
> =A0no mls flow ipv6
>
> =A0mls nde sender version 5
>
> =A0no mls verify ip checksum
>
> =A0no mls acl tcam share-global
>
>
>
> =A0ip flow-export source Loopback0
>
> =A0ip flow-export version 5 origin-as
>
> =A0ip flow-export destination <ip> <port>
>
>
>
> Then I have this enabled on all border interfaces/vlans (peering / transi=
t /
> other core routers) that are of interest for my stats:
>
>
>
> =A0ip route-cache flow
>
>
>
> Some more details about the problem:
>
>
>
> #sh mls netflow table-contention detailed Earl in Module 5 Detailed Netfl=
ow
> CAM (TCAM and ICAM) Utilization
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>
> TCAM Utilization =A0 =A0 =A0 =A0 =A0 =A0 : =A0 100%
>
> ICAM Utilization =A0 =A0 =A0 =A0 =A0 =A0 : =A0 13%
>
> Netflow TCAM count =A0 =A0 =A0 =A0 =A0 : =A0 262033
>
> Netflow ICAM count =A0 =A0 =A0 =A0 =A0 : =A0 17
>
> Netflow Creation Failures =A0 =A0: =A0 4822220
>
> Netflow CAM aliases =A0 =A0 =A0 =A0 =A0: =A0 1
>
>
>
>
>
> #sh mls netflow table-contention aggregate Earl in Module 5 Aggregate
> Netflow CAM Contention Information
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>
> Netflow Creation Failures =A0 =A0: =A0 130003616
>
> Netflow Hash Aliases =A0 =A0 =A0 =A0 : =A0 4
>
>
>
>
>
> I understand that the TCAM is full, but what can I do against it? This is=
a
> busy core router:
>
>
>
> Aggregated traffic: 7-8 GBIT/s
>
> Packets per Second: 1.0 - 1.2 Million
>
>
>
> I wouldn't mind analyzing only every 10th or 100th flow, which seems to b=
e a
> common practice.
>
>
>
> Any good piece of advice is welcome.
>
>
>
> Thanks!
>
>
>
> -
> Andy
