[129968] in North American Network Operators' Group

home help back first fref pref prev next nref lref last post

Facebook Engineering on today's outage

daemon@ATHENA.MIT.EDU (Jay R. Ashworth)
Thu Sep 23 22:17:27 2010

Date: Thu, 23 Sep 2010 22:17:12 -0400 (EDT)
From: "Jay R. Ashworth" <jra@baylink.com>
To: outages@outages.org, nanog@nanog.org
In-Reply-To: <44017.1922.qm@web59605.mail.ac4.yahoo.com>
Errors-To: nanog-bounces+nanog.discuss=bloom-picayune.mit.edu@nanog.org

http://www.facebook.com/notes/facebook-engineering/more-details-on-todays-outage/431441338919

Apparently, our surmise about Akamai notwithstanding, the problem was actually
internal to their app-specific caching facilities, which went into Sorcerer's
Apprentice mode, and they had to kill them all and let ghod sort them out.

More if I get it; hope that posting's public. 

Cheers,
-- jra

-- 
Jay R. Ashworth                   Baylink                      jra@baylink.com
Designer                     The Things I Think                       RFC 2100
Ashworth & Associates     http://baylink.pitas.com                     '87 e24
St Petersburg FL USA      http://photo.imageinc.us             +1 727 647 1274

    Start a man a fire, and he'll be warm all night.
     Set a man on fire, and he'll be warm for the rest of his life.


home help back first fref pref prev next nref lref last post