[3516] in WWW Security List Archive
Re: Alta Vista may or may not harvest unadvertised documents
daemon@ATHENA.MIT.EDU (John Cronin)
Tue Nov 12 17:02:59 1996
From: John Cronin <John.Cronin@oit.gatech.edu>
To: riddle@is.rice.edu (Prentiss Riddle)
Date: Tue, 12 Nov 1996 09:17:26 -0500 (EST)
Cc: www-security@ns2.rutgers.edu
In-Reply-To: <199611111632.KAA18657@is.rice.edu> from "Prentiss Riddle" at Nov 11, 96 10:32:05 am
Errors-To: owner-www-security@ns2.rutgers.edu
Once upon a time, Prentiss Riddle told me this tale:
->
->This item from a recent Risks Digest caught my eye:
->
-> http://catless.ncl.ac.uk/Risks/18.58.html#subj8
->
[details deleted]
->
->In other words, when processing a URL like:
->
-> http://www.foo.com/somepath/somefilename.html
->
->...is is alleged that the Alta Vista harvester will truncate the URL
->to:
->
-> http://www.foo.com/somepath/
->
->...in hopes that an automatically generated index of the directory
->will turn up files for which there is no explicit HREF link.
Come on, haven't we all done this at one time or another individually?
I do it all the time. Often, I have just used a web searcher to find
an interesting link, and I decide I want to go up to the main page,
and there is no convenient button to take me there. I just go up
into the "Location: " box and delete the last item in the path. Repeat
until you find what you want or are convinced it is not there.
->Regardless of whether the Alta Vista harvester is this aggressive,
->other harvesters (or individual human users) might be, so the prudent
->thing is never to put files in a world-readable web tree that you can't
->afford for the world to see. Other recent RISKS postings include a few
->horror stories on this theme.
This is the crux of it. Either that, or be VERY careful about always
putting in an appropriate index file for each directory AND make sure
all your permissions are set properly.
--
John Cronin
Office of Information Technology Customer Support Center 0710
Georgia Institute of Technology, Atlanta Georgia, 30332
Internet: john.cronin@oit.gatech.edu
phone: (404) 894-7563
