[129489] in North American Network Operators' Group
Re: yahoo crawlers hammering us
daemon@ATHENA.MIT.EDU (Valdis.Kletnieks@vt.edu)
Wed Sep 8 10:08:33 2010
To: Bruce Williams <williams.bruce@gmail.com>
In-Reply-To: Your message of "Wed, 08 Sep 2010 02:21:31 PDT."
<AANLkTin0Y7JeZi+avf9ayx=FMkbgiQKEef9mFiMaYPj4@mail.gmail.com>
From: Valdis.Kletnieks@vt.edu
Date: Wed, 08 Sep 2010 10:07:46 -0400
Cc: nanog@nanog.org
Errors-To: nanog-bounces+nanog.discuss=bloom-picayune.mit.edu@nanog.org
--==_Exmh_1283954866_77395P
Content-Type: text/plain; charset=us-ascii
On Wed, 08 Sep 2010 02:21:31 PDT, Bruce Williams said:
> > I *am* curious--what makes it any worse for a search engine like Google
> > to fetch the file than any other random user on the Internet
>
> Possibly because that other user is who the customer pays have their
> content delivered to?
Seems to me that if you're doing content-for-pay and are upset when
some(body|thing) downloads it without paying for it, you shouldn't be leaving
gigabytes of said content where even a stupid bot can find it and download it
without paying for it. Just sayin'.
--==_Exmh_1283954866_77395P
Content-Type: application/pgp-signature
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Exmh version 2.5 07/13/2001
iD8DBQFMh5iycC3lWbTT17ARAmgUAJ9HCxUoSN40i7zpvjghhbibmxcFugCgvs9J
Rh2ezRs/0jUgp10UBFzNEyQ=
=/x5K
-----END PGP SIGNATURE-----
--==_Exmh_1283954866_77395P--
