[112719] in North American Network Operators' Group
Re: Netflow on SUP720-3BXL
daemon@ATHENA.MIT.EDU (Olof Kasselstrand)
Sun Mar 15 04:13:37 2009
In-Reply-To: <d626d8700903141920p3c6ef3bfs54a647b58c4a6b8e@mail.gmail.com>
Date: Sun, 15 Mar 2009 09:13:24 +0100
From: Olof Kasselstrand <olof.kasselstrand@gmail.com>
To: Andy Bierlair <globichen@gmail.com>
Cc: nanog@nanog.org
Errors-To: nanog-bounces@nanog.org
Have a look at http://www.cisco.com/en/US/products/hw/switches/ps700/produc=
ts_tech_note09186a00801b42bf.shtml#prob1a
// Olof
On Sun, Mar 15, 2009 at 3:20 AM, Andy Bierlair <globichen@gmail.com> wrote:
> yes ip cef, this is enabled:
>
> =A0IP fast switching is enabled
> =A0IP fast switching on the same interface is disabled
> =A0IP Flow switching is enabled
> =A0IP CEF switching is enabled
> =A0IP Flow switching turbo vector
> =A0IP Flow CEF switching turbo vector
>
> and so on...
>
> -
> Andy
>
> On Sun, Mar 15, 2009 at 3:08 AM, Bill Blackford
> <BBlackford@nwresd.k12.or.us> wrote:
>>
>> just a shot in the dark. Do you have 'ip cef' in global config?
>>
>> -b
>> ________________________________________
>> From: Andy Bierlair [globichen@gmail.com]
>> Sent: Saturday, March 14, 2009 6:55 PM
>> To: nanog@nanog.org
>> Subject: Netflow on SUP720-3BXL
>>
>> I=92m trying to run netflow on one of our Cisco core routers (SUP720-3BX=
L),
>> but I think I am hitting some limitations because of this:
>>
>>
>>
>> %EARL_NETFLOW-SP-4-TCAM_THRLD: Netflow TCAM threshold exceeded, TCAM
>> Utilization [99%]
>>
>>
>>
>> The setup of netflow looks like this:
>>
>>
>>
>> =A0ip flow-cache entries 524288
>>
>> =A0mls aging fast time 5 threshold 32
>>
>> =A0mls aging long 300
>>
>> =A0mls aging normal 60
>>
>> =A0mls netflow usage notify 80 300
>>
>> =A0mls flow ip full
>>
>> =A0no mls flow ipv6
>>
>> =A0mls nde sender version 5
>>
>> =A0no mls verify ip checksum
>>
>> =A0no mls acl tcam share-global
>>
>>
>>
>> =A0ip flow-export source Loopback0
>>
>> =A0ip flow-export version 5 origin-as
>>
>> =A0ip flow-export destination <ip> <port>
>>
>>
>>
>> Then I have this enabled on all border interfaces/vlans (peering / trans=
it /
>> other core routers) that are of interest for my stats:
>>
>>
>>
>> =A0ip route-cache flow
>>
>>
>>
>> Some more details about the problem:
>>
>>
>>
>> #sh mls netflow table-contention detailed Earl in Module 5 Detailed Netf=
low
>> CAM (TCAM and ICAM) Utilization
>> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>>
>> TCAM Utilization =A0 =A0 =A0 =A0 =A0 =A0 : =A0 100%
>>
>> ICAM Utilization =A0 =A0 =A0 =A0 =A0 =A0 : =A0 13%
>>
>> Netflow TCAM count =A0 =A0 =A0 =A0 =A0 : =A0 262033
>>
>> Netflow ICAM count =A0 =A0 =A0 =A0 =A0 : =A0 17
>>
>> Netflow Creation Failures =A0 =A0: =A0 4822220
>>
>> Netflow CAM aliases =A0 =A0 =A0 =A0 =A0: =A0 1
>>
>>
>>
>>
>>
>> #sh mls netflow table-contention aggregate Earl in Module 5 Aggregate
>> Netflow CAM Contention Information
>> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>>
>> Netflow Creation Failures =A0 =A0: =A0 130003616
>>
>> Netflow Hash Aliases =A0 =A0 =A0 =A0 : =A0 4
>>
>>
>>
>>
>>
>> I understand that the TCAM is full, but what can I do against it? This i=
s a
>> busy core router:
>>
>>
>>
>> Aggregated traffic: 7-8 GBIT/s
>>
>> Packets per Second: 1.0 - 1.2 Million
>>
>>
>>
>> I wouldn't mind analyzing only every 10th or 100th flow, which seems to =
be a
>> common practice.
>>
>>
>>
>> Any good piece of advice is welcome.
>>
>>
>>
>> Thanks!
>>
>>
>>
>> -
>> Andy
>
>
