[121974] in North American Network Operators' Group
Re: Mitigating human error in the SP
daemon@ATHENA.MIT.EDU (David Hiers)
Tue Feb 2 21:39:13 2010
In-Reply-To: <877585b01002021736l62483870m3fab2780e02746bb@mail.gmail.com>
Date: Tue, 2 Feb 2010 18:38:40 -0800
From: David Hiers <hiersd@gmail.com>
To: nanog@nanog.org
Errors-To: nanog-bounces+nanog.discuss=bloom-picayune.mit.edu@nanog.org
If your manager pretends that they can manage humans without a few
well-worn human factor books on their shelf, quit.
David
On Tue, Feb 2, 2010 at 5:36 PM, Michael Dillon
<wavetossed@googlemail.com> wrote:
>> The actual error happened when someone was troubleshooting a turn-up,
>> where in the past the customer in question has had their ethertype set
>> wrong. =A0It wasn't a provisioning problem as much as someone
>> troubleshooting why it didn't come up with the customer. =A0Ironically,
>> the NOC was on the phone when it happened, and the switch was rebooted
>> almost immediately and the outage lasted 5 minutes.
>
> This is why large operators have a "ready for service" protocol. The cust=
omer
> is never billed until it is officially RFS, and to make it RFS requires m=
ore
> than an operational network, it also requires the customer to agree in wr=
iting
> that they have a fully functional connection.
>
> This is another way of hiding human error, because now the up-down-up is
> just part of the provisioning process. There is a record of the RFS date-=
time
> so if the customer complains about an outage BEFORE that point, they can
> be politely reminded that when RFS happened and that charging does not
> start until AFTER that point.
>
> --Michael Dillon
>
>
