[303] in Commercialization & Privatization of the Internet
Re: A few questions re current discussions...
daemon@ATHENA.MIT.EDU (Edward Vielmetti)
Tue Mar 5 14:25:36 1991
To: Jack Haverty <jhaverty@us.oracle.com>
Cc: ddern@world.std.com (Daniel P Dern), com-priv@psi.com
In-Reply-To: Your message of Tue, 05 Mar 91 08:49:02 -0800.
Date: Tue, 05 Mar 91 14:10:30 EST
From: Edward Vielmetti <emv@ox.com>
> help me *find* things on the internet -- e.g., gives me pointers
> to all reports, news ostings, files, message etc...
> "Frame Relay and IP".
This can be done with existing tools, if you're willing to spend
some time gluing things together.
The first thing to do is to get a full usenet newsfeed, and to
subscribe to every possible mailing list that you can think of,
and have some means of dumping all of those mailing lists into
a news spool area. It woudl be helpful to have a way of automatically
subscribing to new (e.g.) IETF lists as they were created; perhaps
some other site could be coerced into mutating them into news
for you.
Next: construct for yourself a filter of some sort that matches
a high enough fraction of the messages you are interested in
without too many false matches. For this application you might
start with just "frame relay" or maybe " FR ". It would be useful
to have an example of 100 or so sample texts which meet the
classification you want, and to do some linguistic analysis to
check if there are any other characteristic phrases.
Apply the filter to every single article you can imagine, make
sure it's cheap enough that you can sustain a 0.01% hit rate.
I use the GNUS newsreader, but I think the next approach wuld
be to use Brad Templeton's "dynafeed" program to manage the
newsfeed and a standalone script to munge through the articles.
What you then have is a stream of possibly interesting articles.
Put a human in the loop there to weed through the Framemaker
discussions and the electromechanical relays and to identify
the "interesting" stuff. Have them tag the articles with
some sort of unique identifier and with a category or grouping
which might make some sense later. Stick them into a full text
database.
Fetch all of the other things that these articles mention and
stick them into the same database (or pointers to resources
as available). Glue on a front end that lets you go back and
search on texts, authors, locations, what have you.
Feed the resulting stream of marked up articles and texts
as a usenet newsgroup (with whichever distribution you would
like). Point discussion back to the appropriate place so that
users can followup to articles as they would like without having
to think about it, so that they can continue discusions as if
they were in the original newsgroup.
Like I say it's real doable. I think that Geoff Goodfellow
and his "anterior technology" does some of the same stuff,
but I don't know what all is involved there. comp.archives
is a group which I manage in this style. Given the appropriate
interest, I could probably develop a service like this for
any reasonably well defined interest area with existing discussion
going on somewhere on the internet. (Contact me for rates,
of course, this would have to be a commercial endeavor...:)
--
Msen Edward Vielmetti
/|--- moderator, comp.archives
emv@msen.com
