[945] in Public-Access_Computer_Systems_Forum
Boolean & Probabilistic
daemon@ATHENA.MIT.EDU (Public-Access Computer Systems For)
Fri Aug 7 09:42:35 1992
Date: Fri, 7 Aug 1992 08:36:30 CDT
Reply-To: Public-Access Computer Systems Forum <PACS-L%UHUPVM1.BITNET@ricevm1.rice.edu>
From: Public-Access Computer Systems Forum <LIBPACS%UHUPVM1.BITNET@ricevm1.rice.edu>
To: Multiple recipients of list PACS-L <PACS-L%UHUPVM1.BITNET@ricevm1.rice.edu>
3 Messages, 189 Lines
*-----
From: buchanan@wulib-jpb.wustl.edu (Paul Buchanan)
Subject: Re: CLR-OPAC study
I agree with Hildreth that ranked retrieval systems have lots of promise for
future OPACs, but his assertion that almost any ranked non-Boolean system
is better than every Boolean system is too broad and obscures some critical
factors that are, in my view, very important in the design of future library
systems.
The heart of his argument seems to be that for any given set of keywords,
a ranked retrieval system will find every record that could have been found
from any possible Boolean expression using the same keywords, but the ranked
system is still better since it imposes some "intelligent" order on the
retrieved set; I believe this argument is based on the assumption that the
non-Boolean system always starts by building the set of documents containing
any of the keywords and then orders that set with some weighting scheme.
While this may be true in a textbook sense ("Prove that the set 'A or B or C'
includes every member of any set constructed by some other Boolean combination
of the same terms"), it may not translate automatically into demonstrably
better systems in the real world. There are several other points to consider.
While Boolean systems do not rank on the basis of relevance weights, it is not
true that they do not rank at all. Various sorting options are available with
many systems, and some of them (reverse chronological order for medical
literature, for instance) may make as much sense in some cases as the system's
internal weighting algorithm. Merging user-selected sort orders with the
various topical weighting schemes seems like an interesting area for some
innovation, but I don't think it is a closed question just yet.
Secondly, I don't think it is self-evident that users will submit exactly the
same set of keywords to an interface that invites "natural language" input as
to an interface that requires Boolean set construction. The fact that most
non-Boolean systems subtly convey the assertion that they "understand" the
users query leads, I think, to poor retrieval in some cases where the more
mechanical feel of Boolean systems leads the user to consider their keywords
more carefully. Should we put this burden on the user? Well, in an ideal
world maybe not, but in the real world I don't think we know yet how to build
systems that can substitute computer cycles for some consideration by the
user of their actual entry vocabulary.
Lastly, there is the question of set sizes. Contrary to Hildreth's
idealized view of non-Boolean systems, many practical systems do not actually
build and rank the complete set of records containing any query term, but
instead "truncate" the result set at some suitably low weight. This is not
simply a case of "screwing up" the implementation; it is frequently a case of
the necessary engineering to build a practical system that meets acceptable
response time targets for large user populations at acceptable costs. So some
relevant records may indeed be lost to the user (provided the ranking
algorithm mistakenly assigned them low scores). But more importantly, I'm
not sure how many users are going to scan large sets looking for low-scoring
records which are nevertheless highly relevant. My experience suggests most
users look at the first dozen or so records in any retrieved set before
giving up or going on. This may be more true with systems that purport to
judge relevance: the user's patience with "the stupid machine" may be quite
short when presented with a few high-scoring records that are obviously wrong
and then being told that somewhere in the next few hundred records may be the
nugget or two they are looking for.
I also think it is important to remember that most of the published research
on ranked retrieval has dealt with topical searching among unstructured
records (or at least ignoring any record structure). There are still
significant questions to be answered about the application of free-text
ranked retrieval algorithms to the variety of kinds of queries our users bring
to the catalog. What about author, title, or other known-item searches (at
least Boolean systems employ a consistent syntax for almost all search types)?
How can we exploit the record structure inherent in bibliographic information
in relevance analysis? How can we exploit the information in our authority
files and thesauri within the weighted retrieval model? And so on.
Finally:
I am aware of at least one study where some users (actual non-librarians)
still preferred Boolean searching even when presented with approximately
equal GUI interfaces to the same database. What does this mean? Do we know
better than our users what they should want? I'm in favor of adding
new search schemes to our catalogs, but I think we have to recognize that
Boolean systems can be powerful, precise tools for some users in some cases.
It shouldn't be either/or. We need catalogs where users can select from a
variety of tools (Boolean, free-text, ranked, sorted, etc, etc) embedded within
a consistent user interface. Then let the user decide.
Paul Buchanan
Washington University at St. Louis
*-----
From: Charles Hildreth <hildreth@eagle.sangamon.edu>
Subject: Re: Boolean & Probabilistic
My response to my friend and respected colleague, Michael Buckland:
> Charles Hildreth commented critically (PACS-L 4 Aug) about Boolean
> searching in his response to Kathy Kwan's earlier note. But criticism
> of Boolean approaches should not be based only on the simplest form of
> Boolean searching, namely one single, binary partitioning between
> "retrieved" and "not retrieved". Partitioning the database into only
> two sets: retrieved and non-retrieved, is the simplest, crudest use of
> Boolean. There is considerable scope using Boolean searching for
> creating and ranking subsets so that the material found can be modified
> dynamically to suit a combination of the preferred SIZE of retrieved
> set and the PREFERENCES (e.g. by language and date) the the user may
> have -- and to do so in response to the often unpredictable results of
> the initial search
O.K., Michael, Boolean retrieval techniques have a role to play, and are
especially useful when a precise expression of a well-defined information need
can be formulated in a Boolean query (this probably is the exceptional case for
an OPAC end-user), but the Boolean methods/algorithms do not do the ranking of
documents in the subset as your phrasing suggests to the unwary. No matter how
complex and sophisticated the Boolean search query, any single Boolean query
will do nothing more but partition the database into only two unranked sets:
retrieved
and not retrieved, all presented as equally relevant or non-relevant.
> However much or little users understand of Boolean operators, we
> might also ask how much they understand of the (possibly very complex)
> mathematical and other operations underlying probabilistic techniques.
> How satisfactory is it to offer retrieval (even effective retrieval) by
> techniques that cannot be grasped?
Providing the most effective retrieval is the point and the goal. Why do any
among us still insist that the end-user is not served well unless he/she
understands (can 'grasp') the internal mechanisms of the machine doing the
job. Is it our professional hubris or insecurity that causes us to still
insist on this nonsense? Users don't want the system between themselves and
the information they seek. They surely don't want to have to learn the
mechanics of the engine before they jump into the automobile and make an
effective trip down to the corner grocery store to retrieve a loaf of
bread. Both me and my wife think we have a very effective Volvo.
A simple analogy to be sure, but it could be easily extended.
We do agree completely, I think, that feedback opportunities for the user
and iterative modification of the search based on what has been retrieved
and assessed is a critical feature OPACs and other retrieval systems should
provide.
Charles Hildreth
READ Ltd.
(hildreth@eagle.sangamon.edu)
*-----
From: dhoyman@fammed.Wisc.EDU
Subject: Re: Boolean & Probabilistic
> However much or little users understand of Boolean operators, we
>might also ask how much they understand of the (possibly very complex)
>mathematical and other operations underlying probabilistic techniques.
>How satisfactory is it to offer retrieval (even effective retrieval) by
>techniques that cannot be grasped?
> -- Michael Buckland, School of Library & Information Studies,
>University of California, Berkeley, CA 94720
I cannot speak for all techniques, but it is possible to have a search
technique that is very sophisticed and have the gory details hidden from
the user. Indeed, the simplistic pattern matching done by wildcards (*) in
many environments actually has a fairly complex algorithm behind it.
One example I am aware of that could be useful for OPACs is LSI, latent
semantic indexing. LSI really only requires keywords, in no particular
order. It returns hits with an index of relevance. The LSI algorithm is
based on a sophisticate statical technique, factor analysis. But, you
don't need to know factor analysis to use it.
While you could say the same thing about boolean operators, one doesn't
really need to know the algorithms of how they are implemented to use them,
that isn't really the issue. The issue is that it's hard to SPECIFY a
boolean query. Compare this to the LSI technique, where one just gives a
list of keywords.
It sounds like the study Michael is engaged in is exploring the use of
sophisticated Boolean techniques, which hopefully will be profitable.
However, I would disagree with his comment above that the end user needs to
understand the operations of a search technique in order to use it. I
would submit that the searching is a black box. What the user cares about
is whether she/he can get the desired results in a timely manner. If the
black box uses some very sophisticated statistical techniques, so be it.
The user will only really care about how it performs.
Dirk Herr-Hoyman | Are we
University of Wisconsin-Madison | kinder and gentler
Dept. of Family Medicine and Practice | yet?
dhoyman@fammed.wisc.edu |
(608) 262-6368 voice |
(608) 263-5813 fax |
