[3558] in WWW Security List Archive
Re: Alta Vista may or may not harvest unadvertised documents
daemon@ATHENA.MIT.EDU (David M. Chess)
Fri Nov 15 17:43:38 1996
Date: Fri, 15 Nov 96 14:33:45 EST
From: "David M. Chess" <CHESS@watson.ibm.com>
To: www-security@ns2.rutgers.edu
Errors-To: owner-www-security@ns2.rutgers.edu
> From: =?iso-8859-1?Q?=22A=2E_=D6mer_K=F6ker=22?= <omer@superonline.net>
> Again I call everyones attention to robot exclusion standarts. A
> robots.txt will keep wanderers and crawlers away!!!!!!
But not as a *security* measure, of course! If an attacker wants
to ignore robots.txt, he certainly will. *8) And I wouldn't
want to have my security depend on Alta Vista (or anyone else
non-malicious) having no bugs in their robots.txt interpreter...
DC
