[121973] in North American Network Operators' Group
Re: Mitigating human error in the SP
daemon@ATHENA.MIT.EDU (Michael Dillon)
Tue Feb 2 20:39:23 2010
In-Reply-To: <e7667f301002020714k19797e9bw2f182359a2b94c38@mail.gmail.com>
Date: Wed, 3 Feb 2010 01:36:59 +0000
From: Michael Dillon <wavetossed@googlemail.com>
To: Chadwick Sorrell <mirotrem@gmail.com>
Cc: nanog@nanog.org
Errors-To: nanog-bounces+nanog.discuss=bloom-picayune.mit.edu@nanog.org
> The actual error happened when someone was troubleshooting a turn-up,
> where in the past the customer in question has had their ethertype set
> wrong. =A0It wasn't a provisioning problem as much as someone
> troubleshooting why it didn't come up with the customer. =A0Ironically,
> the NOC was on the phone when it happened, and the switch was rebooted
> almost immediately and the outage lasted 5 minutes.
This is why large operators have a "ready for service" protocol. The custom=
er
is never billed until it is officially RFS, and to make it RFS requires mor=
e
than an operational network, it also requires the customer to agree in writ=
ing
that they have a fully functional connection.
This is another way of hiding human error, because now the up-down-up is
just part of the provisioning process. There is a record of the RFS date-ti=
me
so if the customer complains about an outage BEFORE that point, they can
be politely reminded that when RFS happened and that charging does not
start until AFTER that point.
--Michael Dillon
