[157585] in North American Network Operators' Group
Re: Network scan tool/appliance horror stories
daemon@ATHENA.MIT.EDU (Ryan Malayter)
Tue Oct 30 01:04:43 2012
From: Ryan Malayter <malayter@gmail.com>
In-Reply-To: <955E521BF6D5DE48A9413310E8C0A14716EFA07631@MAIL1.rose.portland.local>
Date: Tue, 30 Oct 2012 00:04:21 -0500
To: nanog@nanog.org
Errors-To: nanog-bounces+nanog.discuss=bloom-picayune.mit.edu@nanog.org
On Oct 29, 2012, at 3:55 PM, "Rutis, Cameron"=20
> =20
> 6) large stacks of 3750s (six or more members) have issues around CPU duri=
ng certain SNMP commands (I want to say some sort of getbulk type of command=
)
>=20
> The first four were pretty minor although #3 could generate a lot of calls=
to the support center. #5 was a big deal due to the nature of the applicat=
ion. #6 was impactful because we dropped routing neighbors for about 10 sec=
onds but this was a couple of years ago so may have been an old IOS bug.
Saw the same. All of our 3750 stacks (which are small) committed suicide dur=
ing a trial of Foglight. We had discovery timings turned way down, but it st=
ill caused a reload on a mix of the last supposedly really stable releases o=
f 12.x.
Not confidence inspiring. TAC was useless and suggested a v15 upgrade despit=
e no known fix. The proposed v15 upgrade sent our lab boxes into continuous r=
eload unless you broke the stack and manually wiped each switch. Oh, and por=
t 28 was invisible on each switch after upgrade, and Gi2/0/28 would throw a s=
yntax error. Wait for new releases, lather, rinse, repeat.
Total time to resolution in production was several man-weeks on our side, an=
d a few months calendar time, all because the discovery scan revealed how gr=
eat a "software company" Cisco has become.=
