[11455] in www-talk@info.cern.ch
Informing robots that site has changed significantly?
daemon@ATHENA.MIT.EDU (Conan Callen)
Thu Dec 16 09:09:42 1999
Date: Wed, 15 Dec 1999 20:00:49 -0500 (EST)
Old-X-Envelope-From: ccallen@windowpane.com Wed Dec 15 20:00:48 1999
Message-Id: <00ea01bf4760$c5e9a740$3a8314d1@nwlink.com>
From: "Conan Callen" <ccallen@windowpane.com>
To: <www-talk@www.cern.ch>
Cc: <m.koster@webcrawler.com>
Mime-Version: 1.0
Content-Type: multipart/alternative;
boundary="----=_NextPart_000_00E7_01BF471D.B7C04CC0"
Old-X-Envelope-To: www-talk
Resent-From: gerald@w3.org
Resent-To: www-talk@w3.org
This is a multi-part message in MIME format.
------=_NextPart_000_00E7_01BF471D.B7C04CC0
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
Hi,
I was wondering if there is a way to tell a robot to drop everything it =
knows about your site and reindex the whole thing? We have recently =
reworked our whole site and now robots have been requesting pages that =
are no longer there. It would be real convient if there was a reindex =
directive that could be place into the robots.txt that would let the =
robot know that significant changes have occured to this site and it =
would probably be best to start over from the top (especially if many =
new pages have been added, and old ones moved or deleted).=20
Im not on this mailing list, if there are any replies please cc my email =
- ccallen@windowpane.com
conan
------=_NextPart_000_00E7_01BF471D.B7C04CC0
Content-Type: text/html;
charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META content=3D"text/html; charset=3Diso-8859-1" =
http-equiv=3DContent-Type>
<META content=3D"MSHTML 5.00.2314.1000" name=3DGENERATOR>
<STYLE></STYLE>
</HEAD>
<BODY bgColor=3D#ffffff>
<DIV><FONT face=3DArial size=3D2>Hi,</FONT></DIV>
<DIV> </DIV>
<DIV><FONT face=3DArial size=3D2>I was wondering if there is a way to =
tell a robot=20
to drop everything it knows about your site and reindex the whole thing? =
We have=20
recently reworked our whole site and now robots have been requesting =
pages that=20
are no longer there. It would be real convient if there was =
a reindex=20
directive that could be place into the robots.txt that would let the =
robot know=20
that significant changes have occured to this site and it would probably =
be best=20
to start over from the top (especially if many new pages have been =
added, and=20
old ones moved or deleted). </FONT></DIV>
<DIV> </DIV>
<DIV><FONT face=3DArial size=3D2>Im not on this mailing list, if there =
are any=20
replies please cc my email - <A=20
href=3D"mailto:ccallen@windowpane.com">ccallen@windowpane.com</A></FONT><=
/DIV>
<DIV> </DIV>
<DIV><FONT face=3DArial size=3D2>conan</FONT></DIV></BODY></HTML>
------=_NextPart_000_00E7_01BF471D.B7C04CC0--
