[9445] in www-talk@info.cern.ch
Re: I, Robot
daemon@ATHENA.MIT.EDU (Martijn Koster)
Mon Dec 11 22:36:44 1995
To: www-talk@w3.org
In-Reply-To: Your message of "Mon, 11 Dec 1995 17:46:21 EST."
<818721981.470000.FRANK@han1.hannah.com>
Date: Mon, 11 Dec 1995 19:26:55 -0800
From: Martijn Koster <mak@surfski.webcrawler.com>
Resent-From: www-talk@w3.org
Steve <steve@mordred.gatech.edu> wrote:
>I may have missed it in the archives, but I didn't see any topics on which
> sections of a document a robot or spider or other Web traveller might
> examine to determine content information.
"Might": every part. In practice, who knows, they all do different things.
>If there isn't a specific indexing method that is standard for all search
>engines/robots/crawlers, shouldn't there be one?
What, and give up competition? :-) But yes, it'd help to have better ways.
Frank Atkinson <FRANK@han1.hannah.com> replied:
> There is a mailing list at autoshoppers@han1.hannah.com (put subscribe in the
> subject to subscribe) examining some of these issues as they relate to bots
> which will go "shopping" on the web.
Don't know if that has a different focus, but there is the robots mailing list
on robots-request@webcrawler.com
-- Martijn
__________
Email: m.koster@webcrawler.com
WWW: http://info.webcrawler.com/mak/mak.html
