[112708] in North American Network Operators' Group
Netflow on SUP720-3BXL
daemon@ATHENA.MIT.EDU (Andy Bierlair)
Sat Mar 14 21:56:10 2009
Date: Sun, 15 Mar 2009 02:55:51 +0100
From: Andy Bierlair <globichen@gmail.com>
To: nanog@nanog.org
Errors-To: nanog-bounces@nanog.org
I=92m trying to run netflow on one of our Cisco core routers (SUP720-3BXL),
but I think I am hitting some limitations because of this:
%EARL_NETFLOW-SP-4-TCAM_THRLD: Netflow TCAM threshold exceeded, TCAM
Utilization [99%]
The setup of netflow looks like this:
ip flow-cache entries 524288
mls aging fast time 5 threshold 32
mls aging long 300
mls aging normal 60
mls netflow usage notify 80 300
mls flow ip full
no mls flow ipv6
mls nde sender version 5
no mls verify ip checksum
no mls acl tcam share-global
ip flow-export source Loopback0
ip flow-export version 5 origin-as
ip flow-export destination <ip> <port>
Then I have this enabled on all border interfaces/vlans (peering / transit =
/
other core routers) that are of interest for my stats:
ip route-cache flow
Some more details about the problem:
#sh mls netflow table-contention detailed Earl in Module 5 Detailed Netflow
CAM (TCAM and ICAM) Utilization
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
TCAM Utilization : 100%
ICAM Utilization : 13%
Netflow TCAM count : 262033
Netflow ICAM count : 17
Netflow Creation Failures : 4822220
Netflow CAM aliases : 1
#sh mls netflow table-contention aggregate Earl in Module 5 Aggregate
Netflow CAM Contention Information
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
Netflow Creation Failures : 130003616
Netflow Hash Aliases : 4
I understand that the TCAM is full, but what can I do against it? This is a
busy core router:
Aggregated traffic: 7-8 GBIT/s
Packets per Second: 1.0 - 1.2 Million
I wouldn't mind analyzing only every 10th or 100th flow, which seems to be =
a
common practice.
Any good piece of advice is welcome.
Thanks!
-
Andy
