[187953] in North American Network Operators' Group
Re: google search threshold
daemon@ATHENA.MIT.EDU (Keenan Tims)
Mon Feb 29 11:49:58 2016
X-Original-To: nanog@nanog.org
From: Keenan Tims <ktims@stargate.ca>
To: "nanog@nanog.org" <nanog@nanog.org>
Date: Mon, 29 Feb 2016 16:49:52 +0000
In-Reply-To: <726393422.785405.1456761234633.JavaMail.yahoo@mail.yahoo.com>
Errors-To: nanog-bounces@nanog.org
FWIW I have seen the captchas more often on IPv6 both from home and the off=
ice than when both networks were using a single shared IPv4; not sure if th=
is is just related to chronology or a real effect. Once a month or so I see=
m to get them for a couple of days, then they go away.=0A=
=0A=
No idea what's triggering it. It would be *really* helpful if Google could =
provide some useful technical details beyond a generic FAQ page. As it is I=
just get annoyed by it and have no way to troubleshoot or correct the cons=
tant false positives. How is Google detecting "robots"? My sense is that I =
tend to trigger the captcha thing when iterating similar search terms (part=
icularly due to removal of the + operator and extremely poor "change my sea=
rch terms because you think you know better than I do what I want to search=
for" behaviour. My search patterns haven't really changed since turning up=
IPv6 everywhere, so I have to think either the captcha trigger has gotten =
more aggressive, or somehow prefers to blacklist IPv6 users.=0A=
=0A=
In any case, just going to IPv6 is definitely not a complete fix for this. =
It seems to be related to search behaviour and $blackbox_magic.=0A=
=0A=
Keenan Tims=0A=
Stargate Connections=0A=
________________________________________=0A=
From: NANOG <nanog-bounces@nanog.org> on behalf of Philip Lavine via NANOG =
<nanog@nanog.org>=0A=
Sent: February 29, 2016 7:53 AM=0A=
To: Damian Menscher=0A=
Cc: nanog@nanog.org=0A=
Subject: Re: google search threshold=0A=
=0A=
I have about 2000 users behind a single NAT. I have been looking at netflow=
, URL filter logs, IDS logs, etc. The traffic seems to be legit.=0A=
=0A=
I am going to move more users to IPv6 and divide some of the subnets into d=
ifferent NATS and see if that alleviates the traffic load.=0A=
Thanks for the advice.=0A=
-Philip=0A=
=0A=
=0A=
From: Damian Menscher <damian@google.com>=0A=
To: Philip Lavine <source_route@yahoo.com>=0A=
Cc: "nanog@nanog.org" <nanog@nanog.org>=0A=
Sent: Friday, February 26, 2016 6:05 PM=0A=
Subject: Re: google search threshold=0A=
=0A=
On Fri, Feb 26, 2016 at 3:01 PM, Philip Lavine via NANOG <nanog@nanog.org> =
wrote:=0A=
=0A=
Does anybody know what the threshold for google searches is before you get =
the captcha?I am trying to decide if I need to break up the overload NAT t=
o a pool.=0A=
=0A=
=0A=
There isn't a threshold -- if you send automated searches from an IP, then =
it gets blocked (for a while).=0A=
=0A=
So... this comes down to how much you trust your machines/users. If you're=
a company with managed systems, then you can have thousands of users share=
the same IP without problems. But if you're an ISP, you'll likely run int=
o problems much earlier (since users like their malware).=0A=
Some tips: - if you do NAT: try to partition users into pools so one abus=
ive user can't get all your external IPs blocked - if you have a proxy: ma=
ke sure it inserts the X-Forwarded-For header, and is restricted to your ow=
n users - if you're an ISP: IPv6 will allow each user to have their own /6=
4, which avoids shared-fate from abusive ones=0A=
Damian (responsible for DDoS defense)-- Damian Menscher :: Security Reliabi=
lity Engineer :: Google :: AS15169=0A=
=0A=
=0A=
