[129997] in North American Network Operators' Group

home help back first fref pref prev next nref lref last post

Re: Facebook Engineering on today's outage

daemon@ATHENA.MIT.EDU (vijay gill)
Sat Sep 25 01:07:51 2010

In-Reply-To: <3355550.3590.1285294632830.JavaMail.root@benjamin.baylink.com>
Date: Fri, 24 Sep 2010 22:07:36 -0700
From: vijay gill <vgill@vijaygill.com>
To: "Jay R. Ashworth" <jra@baylink.com>
Cc: outages@outages.org, nanog@nanog.org
Errors-To: nanog-bounces+nanog.discuss=bloom-picayune.mit.edu@nanog.org

On Thu, Sep 23, 2010 at 7:17 PM, Jay R. Ashworth <jra@baylink.com> wrote:
> http://www.facebook.com/notes/facebook-engineering/more-details-on-todays=
-outage/431441338919
>
> Apparently, our surmise about Akamai notwithstanding, the problem was act=
ually
> internal to their app-specific caching facilities, which went into Sorcer=
er's
> Apprentice mode, and they had to kill them all and let ghod sort them out=
.
>
> More if I get it; hope that posting's public.

That was a model postmortem. Wish more companies had that sort of
detail and clarity around what went wrong and what was being done to
fix it.

/vijay

>
> Cheers,
> -- jra
>
> --
> Jay R. Ashworth =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 Baylink =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0jra@baylink.com
> Designer =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 The Things I Think =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 RFC 2100
> Ashworth & Associates =A0 =A0 http://baylink.pitas.com =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 '87 e24
> St Petersburg FL USA =A0 =A0 =A0http://photo.imageinc.us =A0 =A0 =A0 =A0 =
=A0 =A0 +1 727 647 1274
>
> =A0 =A0Start a man a fire, and he'll be warm all night.
> =A0 =A0 Set a man on fire, and he'll be warm for the rest of his life.
>
>


home help back first fref pref prev next nref lref last post