[54467] in North American Network Operators' Group
RE: Google Crawler
daemon@ATHENA.MIT.EDU (Andy Ellifson)
Fri Jan 3 12:10:28 2003
Date: Fri, 3 Jan 2003 09:08:59 -0800 (PST)
From: Andy Ellifson <andy@ellifson.com>
Reply-To: andy@ellifson.com
To: Mike Damm <MikeD@irwinresearch.com>, nanog@merit.edu
In-Reply-To: <8DFC77C97E310148B08AB96CC3ED047E0AB8@mailserv.irwinresearch.com>
Errors-To: owner-nanog-outgoing@merit.edu
Thank you!
--- Mike Damm <MikeD@irwinresearch.com> wrote:
>
> http://www.google.com/bot.html for issues with the crawler.
> mailto:googlebot@google.com will get you a human bean to talk to.
> Normally
> when there is a problem with their robot, they are pretty responsive.
>
> -Mike
>
> ---
> Michael Damm, MIS Department, Irwin Research & Development
> V: 509.457.5080 x298 F: 509.577.0301 E: miked@irwinresearch.com
>
>
> -----Original Message-----
> From: Andy Ellifson [mailto:andy@ellifson.com]
> Sent: Friday, January 03, 2003 8:45 AM
> To: nanog@merit.edu
> Subject: Google Crawler
>
>
> We are a domain registrar and we host/park over 750,000 domain names.
>
> Every now and then the Google Crawler decides to bury the machines
> that
> host our 'parked' domain pages. We use robots.txt but that doesn't
> help under these circumstances. I have tried sending a message to
> Google using their web site. They don't have a NOC entry on
> puck.nether.net either. Our only alternative right now is to block
> the
> crawler at the router level.
>
> Does anbody have a contact at Google or is anyone at Google
> listening?
>
> Thanks!
> Andy Ellifson
