[1454] in Public-Access_Computer_Systems_Forum
Finding my WAIS through a Gopher hole
daemon@ATHENA.MIT.EDU (Eric Morgan)
Mon Oct 12 18:30:32 1992
Date: Mon, 12 Oct 1992 17:07:58 CDT
Reply-To: Public-Access Computer Systems Forum <PACS-L%UHUPVM1.BITNET@mitvma.mit.edu>
From: Eric Morgan <eric_morgan@library.lib.ncsu.edu>
To: Multiple recipients of list PACS-L <PACS-L%UHUPVM1.BITNET@mitvma.mit.edu>
----------------------------Original message----------------------------
Finding my WAIS through a Gopher hole
by Eric Lease Morgan, NCSU Libraries
This text describes my experiences with the Gopher and WAIS protocols and my
unsolicited opinions about how they can be used and implemented in libraries.
Late in 1991 I started hearing about Gopher. It was usually mentioned in the
same breath as Wide Area Information Server (WAIS). I was more interested in
WAIS at the time, and I had been playing with the Macintosh client and the
VT100 client at the OIT BBS (bbs.oit.unc.edu). "Maybe WAIS would be a good
method for distributing ejournals?", I said to myself.
I few months later my esteemed director asked me to become a member of an ARL
Collection Analysis Project (CAP) study team. Being the young, slightly naive
librarian I am, I gleefully accepted the responsibility. My role was as systems
librarian, the techno-weenie. "Great," I thought, "now I can begin to really
make a difference."
During the CAP process I became acutely aware of the serials crisis, its
causes, and possible solutions (including ejournals). At the same time, while
trying to teach Internet courses, the "user-nastiness" of the Internet and the
difficulty navigating it hit home. All the while people still kept talking
about Gopher and WAIS. I decided to learn more about these protocols. After
all, it was in my job description (more or less), and I was suppose to be the
expert. I started reading the USENET news groups comp.infosystems.wais and
comp.infosystems.gopher. "'Infosystems,'" I said to myself, "how appropriate
for a librarian."
One of the first things I learned was the University of Minnesota's definition
of a Gopher. That is where the Gopher protocol was invented.
gopher n. 1. Any of various short tailed, burrowing mammals of
the family Geomyidae, of North America. 2. (Amer. colloq.)
Native or inhabitant of Minnesota: the Gopher State.
3. (Amer. colloq.) One who runs errands, does odd-jobs, fetches
or delivers documents for office staff. 4. (computer tech.)
Software following a simple protocol for tunneling through a TCP/IP
internet.
The next thing I learned was that Gopher and WAIS operate on a client/server
model. In a nutshell, this means here is one program (the server) that
dispenses information, and there is another program (the client) that displays
information. This has great advantages especially when it comes to desperate
computing platforms. Under a client/server model the user interface problems
are left up to the client and the server can then concentrate on its primary
function, disseminating information. Gopher and WAIS clients come in many
operating system flavors: Macintosh, DOS, UNIX, VMS, _et cetera_. The servers
came in a few flavors too, but most of them live on UNIX machines.
Later I learned the software to implement these protocols was free. "Wow! Free!
Far out!" All you had to do is FTP the software and put it together. "Bummer!
Compile? You expect me to compile these things? I can script in HyperCard, but
I'm no C guru. Besides that, I have very little UNIX experience." The Libraries
has a few DECStations. (No, that is not a mismatched subject-verb combination.
"Libraries" is singular here at NCSU.) These DECStations are connected to the
Internet, so I thought I would persevere.
I first downloaded the WAIS software (wais/wais-8-b5.tar.Z) from think.com. I
de-Z'ed, and uncompressed the archive. I read the instructions, compiled the
code, and voile, I had successfully created the WAIS VT100 client and server.
Believe me. It wasn't too difficult. I proceeded to index everything in sight,
especially the papers I had written for various journals. "Wow, this thing
really works."
One of the biggest mysteries is the method the WAIS protocol finds and ranks
documents from a database. The following text, taken directly from the WAIS
documentation, briefly describes WAIS indexing.
The [WAIS] indexer is a very simple inverted file system with primitive
word weighting. What is created is an inverted index of all the
words over 2 characters long in the input text. Words are
truncated to 20 characters. Each word has [a] weight associated
with it; the first time it occurs in a document it gets an extra
weight of 5, and from then on, it gets a weight of 1 in that
document. Words in the headline are worth an extra 10.
Searching is done by finding the documents with the most word
weights for each word in the query. Relevant document's words
are worth 10 times less.
From this text, it is shown that documents are located and ranked based on the
"weights" of words in the documents. Documents with greater total weights are
ranked higher than documents with lower total weights. These weights are
determined by a word's position in the document and the number of times it
occurs. Therefore, "No, WAIS does not do Boolean searching." On the other hand,
to call WAIS searching "natural language" is stretching the point.
At the same time, people on comp.infosystems.gopher were talking about putting
WAIS indexes into their servers. I retrieved a Gopher client from
boombox.micro.umn.edu and found that WAIS searches really could live in Gopher
servers. So I went back to boombox.micro.umn.edu and downloaded the Gopher
server software for UNIX computers (pub/gopher/Unix/gopher1.02.tar.Z). Again, I
de-Z'ed and uncompressed the code. Again, I read the instructions and tried to
compile the code. This time I was not successful as quickly, but persistence
and the Internet community paid off. I had created a Gopher server with WAIS
indexing. It took me approximately two or three weeks to create the beginnings
of a "library without walls."
Before I could fill the "library" with information I needed a structure to
build on. I decided to create a server mirroring the library environment. The
following outline is a (slightly dated) map of the server:
Welcome!
About North Carolina State University
Happenings!, the Campus Wide Information System
About the NCSU Libraries
NCSU's "Library Without Walls"
Discipline Specific "Study Carrels"
Agriculture and Biology
Astronomy and Astrophysics
Education
Government and Law
Library and Information Science
Literature and Journalism
Medicine
Religious Studies
Sociology
Reference Desk
Software tools for gathering information
Beyond NCSU
Archies and FTP sites
BBSs
CWISs
OPACs
Gophers
Each line in this outline either links you to information, an Internet
resource, or presents you with another menu. "Welcome!" presents you with a
text file describing to server. "About NCSU" will present you with information
describing the University. The "Reference Desk" contains general indexes,
dictionaries, directories, news, and weather. Each of the "study carrels"
contain Internet resources pertinent to that discipline.
Now the real fun began, filling my server with useful information. The Gopher
protocol allows you to "link" other people's resources to your server. These
resources can include:
* USENET news groups,
* the Geographic Name Server,
* OPACs,
* FTP sites and therefore software,
* sounds,
* graphics,
* other Gophers,
* BBSs,
* CWISs,
* free-nets,
* dictionaries and thesauri,
* API news,
* email addresses,
* Archie sites,
* text files, ...
All you have to do is write a five or six line text file describing the
resources and put the files in a directory. Here is a link to our
(as-of-yet-to-be-brought-up) OPAC.
Name=The OPAC at NCSU (under development, not for outsiders)
Type=8
Port=23
Path=
Host=ncsulib.lib.ncsu.edu
Not all the lines are necessary. The first line is the description the user
sees. The second denotes the type of resource (ie. telnet, FTP, binhex'ed
document, directory, text file, WAIS index, _et cetera_). The third line is the
TCP/IP socket through which the communications will take place. The fourth line
is a directory listing. The last line is the Internet address of the resource.
In this particular example, type 8 is a telnet session to be opened on the
TCP/IP port 23 (the telnet port) and the address of the resource is
ncsulib.lib.ncsu.edu.
As I explored "GopherSpace", I used the VT100 client. This allowed me to press
the equals key (=) and retrieve links from other people's servers. I was on a
roll. I gophered here. I gophered there. I gophered everywhere. I gophered all
over the world. As I collected my links I "classified" them with the outline
above. As I found resources I couldn't catalog, I created more classifications.
In some cases, I put a resource in more than one place. For example, I put
links in the Reference Desk as well as the Agriculture study carrel.
I then started in on the WAIS indexes. I retrieved a complete listing of the
WAIS sources and began examining them one by one for relevance to North
Carolina State University (NCSU) and currency. As I found ones I liked, I
simply created a link file in the appropriate directory. (At the present time,
I have only examined about two dozen of the 300+ indexes.)
What's really cool about the Gopher protocol is I no longer have to carry my
"little black book" of IP numbers, passwords, and WAIS indexes with me. All I
have to do is know the subject of my resource, go to the appropriate study
carrel, and select the Internet resource from the menu. The client leads me
through the rest of the process. Additionally, with the link files I can give
the Internet resources meaningful names. "What does the WAIS source
'bib-dmi-ens-fr' stand for anyway?"
I have some plans for the future. Ultimately, I would like to learn about as
many Internet resources as possible and link them all to the server. This way
librarians and scholars can visit the appropriate study carrel and have ready
access to the Internet resources pertinent to their discipline.
I would like to put the Libraries's (there's that singular noun again)
telephone directory on the server and index it with WAIS. I also thought it
would be neat to put pictures and short biographies of us librarians in there
as well. These items would be included in the librarian's discipline specific
study carrel too. In one Gopher, (I believe it was University of Minnesota's)
the pathfinders and guides for the library were in the server. This is an
excellent idea that I would like to implement as well.
A Gopher/WAIS combination provides marvelous opportunities for disseminating
ejournals. First a library could subscribe to LISTSERVs disseminating
ejournals. Someone from the library would then retrieve the journals or
articles of interest and edit out the email headers. Next, the resulting text
file could be saved in a particular section of a study carrel. Lastly, the file
could be indexed with WAIS. Now, not only can you retrieve any issue or article
from an electronic journal, but you can also search the entire publication.
Furthermore, most, if not all aspects of this process can be automated with
batch files or shell scripts. "Now we're cooking with gas!"
You too (no, not U2) can create a server. They have been implemented on just
about every computer platform possible. All you need is:
* a computer,
* a medium to large hard disk,
* a direct connection to the Internet,
* time, and
* perseverance.
If you just want to use Gopher servers, then you need a Gopher client. There
are many clients available and more are coming out all the time. Again, all you
need is:
* a computer,
* a direct connection to the Internet, and
* a Gopher client for your particular computer.
For the closest Gopher client nearest you, FTP to boombox.micro.umn.edu and
look in the pub/gopher directory. Once you retrieve a client you will have to
"point your gopher" to a server. This means you configure your client with the
name and port number of a Gopher server.
To visit the Gopher at the NCSU Libraries, point your Gopher to
dewey.lib.ncsu.edu on port 70. The main Gopher hole is at the University of
Minnesota. To visit their server, point your client to gopher.micro.umn.edu on
port 70.
Gopher it!
=================
Eric considers himself a librarian first and a computer user second. His
professional goal is to discover new ways to use computers to provide better
library service. In his spare time he has been seen folding defective floppy
disks into intricate origami flora and fauna. His Internet address is
eric_morgan@ncsu.edu, and his telephone number is (919) 515-6182.
