[7727] in Release_7.7_team
demeter
daemon@ATHENA.MIT.EDU (Jonathan Reed)
Thu Feb 16 10:35:34 2012
From: Jonathan Reed <jdreed@MIT.EDU>
Content-Type: text/plain; charset=us-ascii
Date: Thu, 16 Feb 2012 10:35:31 -0500
Message-Id: <C89B7C2D-A429-452F-9CFA-CE72EF5C0C52@mit.edu>
To: release-team@MIT.EDU
Mime-Version: 1.0 (Apple Message framework v1084)
Content-Transfer-Encoding: 8bit
demeter exploded last night due to a Chinese webspider that thinks it's ok to request every page at once. I've put up a robots.txt, but they won't respect that until they try and re-crawl the site. I tried blocking by domain in mod_authz_host, but that failed. I may consider blocking by netblock, but I have to block all of Beijing to do that, apparently.
So for now, I've disabled any access to /trac/browser that comes from Scripts or off-campus. So the debathena.mit.edu/trac/browser URL won't work, but if you desperately need access, http://athena10.mit.edu/trac/browser will work if you're on net-18.
Ideally, I'd block non-net-18 at the debathena.mit.edu level, but you can't use <Location> in .htaccess.
-Jon
