[129482] in North American Network Operators' Group
RE: yahoo crawlers hammering us
daemon@ATHENA.MIT.EDU (Nathan Eisenberg)
Wed Sep 8 06:25:03 2010
From: Nathan Eisenberg <nathan@atlasnetworks.us>
To: "nanog@nanog.org" <nanog@nanog.org>
Date: Wed, 8 Sep 2010 10:24:38 +0000
In-Reply-To: <AANLkTin0Y7JeZi+avf9ayx=FMkbgiQKEef9mFiMaYPj4@mail.gmail.com>
Errors-To: nanog-bounces+nanog.discuss=bloom-picayune.mit.edu@nanog.org
> Possibly because that other user is who the customer pays have their cont=
ent
> delivered to?
Customers don't want to deliver their content to search engines? That seem=
s silly.
=20
http://www.last.fm/robots.txt (Note the final 3 disallow lines...)
