[19261] in s-news-athena
[S] Summary on ``Comparison of authors''.
daemon@ATHENA.MIT.EDU (Rolf Turner)
Sat Sep 18 12:19:52 1999
Date: Sat, 18 Sep 1999 13:10:59 -0300 (ADT)
From: Rolf Turner <rolf@math.unb.ca>
Message-Id: <199909181610.NAA04748@tanner.math.unb.ca>
To: s-news@wubios.wustl.edu
Many, many thanks to all who responded. It seems that there is
substantial interest in this general topic, although no-one knew
of the particular item (comparing Hemingway with [???] Henry James).
(I'm ***sure*** I didn't imagine this, but never mind.)
Below is a digest of all the responses I received. Most of them
were posted to the list, so you will have seen them already, but
I thought it might be useful to put them together in a compact
form.
Again thanks to all who responded.
cheers,
Rolf Turner
rolf@math.unb.ca
===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===
(1) From: Christian Leger <leger@DMS.UMontreal.CA>
Not related to Hemingway, but here are two on Shakespeare:
Efron, Bradley;Thisted, Ronald
Estimating the number of unseen species: How many words did Shakespeare know?
76Biomtrka 63 435- 448 J
Empirical Bayes;Euler transformation;Negative binomial
Thisted, Ronald;Efron, Bradley
Did Shakespeare write a newly-discovered poem?
87Biomtrka 74 445- 455 J
Empirical Bayes estimation;Poisson regression
===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===
(2) From: Anne York <york@orca.akctr.noaa.gov>
There was an example of this sort discussing authorship of the
Federalist Papers in:
Mosteller and Tukey, 1977. Data analysis and regression: A
secondary course in statistics. Addison-Wesley.580p.
===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===
(3) From: Mario Cortina Borja <cortina@stats.ox.ac.uk>
There's a few papers on the subject of Stylometrics.
Some people have compared word lengths, other words within
contexts, others word frequencies.
I don't know of a comparison between Hemingway and Henry James, but
some good starting points may be:
Thisted & Efron (1987) Biometrika, p. 445-455 (they try to decide if
Shakespeare did write a newly discovered sonnet)
Holmes (1992) JRSS series A, (Holmes ananlyses several texts written
by Joseph Smith, the founder of the Mormon church in two ways: first
as himself, and secondly when he was allegedly being dictated the
words by an angel)
Holmes (1994) Computers and the Humanities (this is a good article on
authorship)
Mosteller and Wallace (1989) in Tanur (ed.) Statistics: a guide to
the unknown (This is the famous analysis of autorship for the
Federalist papers)
More recently, Binongo and Smith (1999) J. of Applied Statistics,
vol. 26-7, p. 781 provide a study of Oscar Wilde's works and Tweedie
et al. (Computers and the Humanities, 1996 and 1998) have re-analysed
the Federalist papers.
There's also something about Plato and on St Paul's writings in the
book by Cox & Snell 'Applied Statistics', but I don't have it near me
at the moment.
As it happens, I've just supervised an MSc student here in Oxford
(Constantinos Chappas) on the subject. We used about 264 texts from
the British Natinoal Corpus. They came from several kinds of
newspapers and other periodicals as well as news scripts (from TV).
It's striking to see how different the patterns in style are: there
are some very clearly defined clusters, with tabloids and TV scripts
(news only) having the poorest vocabulary distributions, with the
national broadsheets (Independent, Guardian and Daily Telegraph)
performing better than the 'other' national broadsheets (Belfast
Telegraph and the Scotsman). We're at the moment writing a paper
about this.
===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===
(4) From: SmithSJ@mar.dfo-mpo.gc.ca
The only article I remember in this area was the following (from CIS)
75Biomtrka 62 207- 212 J
Mendenhall's studies of word-length distribution in the works of Shakespeare
and Bacon
Williams, C. B.
Authorship;Literary statistics
===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===
(5) From: Albyn Jones <jones@reed.edu>
AUTHOR Mosteller, Frederick, 1916-
TITLE Applied Bayesian and classical inference : the case of the
Federalist papers / Frederick Mosteller, David L. Wallace.
EDITION 2nd ed. of Inference and disputed authorship, The Federalist.
PUBLICATION New York : Springer-Verlag, c1984.
===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===
(6) From: "Charles C. Berry" <cberry@tajo.ucsd.edu>
Here is a word search from CIS on 'Shakespeare'. Perhaps you were
thinking of the Efron/Thisted articles.
====================
[Number of combined matches in CIS73.V96 : 1]
====================
%AUTHOR = Rubin, Ernest
%TITLE = Probability, statistics and W. Shakespeare
%JOURNAL = The American Statistician
%VOLUME = 27
%PAGES = 186-191
%YEAR = 1973
%KEYWORDS = [Q+A]; History
====================
[Number of combined matches in CIS75.V96 : 1]
====================
%AUTHOR = Williams, C. B.
%TITLE = Mendenhall's studies of word-length distribution in the works
of Shakespeare and Bacon
%JOURNAL = Biometrika
%VOLUME = 62
%PAGES = 207-212
%YEAR = 1975
%KEYWORDS = Authorship; Literary statistics
====================
[Number of combined matches in CIS76.V96 : 1]
====================
%AUTHOR = Efron, Bradley
%AUTHOR = Thisted, Ronald
%TITLE = Estimating the number of unseen species: How many words did
Shakespeare know?
%JOURNAL = Biometrika
%VOLUME = 63
%PAGES = 435-448
%YEAR = 1976
%KEYWORDS = Empirical Bayes; Euler transformation; Negative binomial
====================
[Number of combined matches in CIS86.V96 : 2]
====================
%AUTHOR = Good, I. J.
%TITLE = The newly discovered poem, possibly by Shakespeare
%JOURNAL = Journal of Statistical Computation and Simulation
%VOLUME = 26
%PAGES = 145-146
%YEAR = 1986
%AUTHOR = Kolata, Gina
%TITLE = Shakespeare's new poem: An ode to statistics
%JOURNAL = Science
%VOLUME = 231
%PAGES = 335-336
%YEAR = 1986
%KEYWORDS = History; Goodness-of-fit
====================
[Number of combined matches in CIS87.V96 : 1]
====================
%AUTHOR = Thisted, Ronald
%AUTHOR = Efron, Bradley
%TITLE = Did Shakespeare write a newly-discovered poem?
%JOURNAL = Biometrika
%VOLUME = 74
%PAGES = 445-455
%YEAR = 1987
%KEYWORDS = Empirical Bayes estimation; Poisson regression
====================
[Number of combined matches in CIS91.V96 : 1]
====================
%AUTHOR = Elliott, Ward
%AUTHOR = Valenza, Robert
%TITLE = Who was Shakespeare?
%JOURNAL = Chance, New Directions for Statistics and Computers
%VOLUME = 4
%NUMBER = 3
%PAGES = 8-14
%YEAR = 1991
%KEYWORDS = History
===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===
(7) From: "Evans, David" <David_Evans@fishgame.state.ak.us>
Wasn't it Shakespeare and Francis Bacon? Don't know the source. Sorry.
===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===
(8) From: "Barry W. Brown" <bwb@odin.mdacc.tmc.edu>
The one that I am familiar with is the following. I found it by the
keyword Federalist in CIS.
> S\"arndal, Carl-Erik
> On deciding cases of disputed authorship
> 67ApplStat 16 251- 268 J
> Bayesian approach;Federalist Papers
>
> Mosteller, F.;Wallace, D. L.
> Applied Bayesian and classical inference: The case of the Federalist Papers
> 84Springer:Brln:NY 303p B
===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===
(9) From: "James B. Ramsey" <jbr1@is9.nyu.edu>
I have a question in my textbook along these lines; mainly as a
student team exercise. Consequently, I will be very interested to
hear the outcome of your request. Many thanks.
===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===
(10) From: tom richards <richards@toutatis.pci.upmc.edu>
I am not aware of those authors' being compared, but there is the
book-length study of Mosteller and Wallace on attributing authorship
of several of the Federalist Papers. I hope this is somewhat
helpful, but it may not be what you're looking for.
===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===
(11) From: Doug Johnson <dougj@statsci.com>
I don't know the answer to your question, but it seems to me that one
probably could distinguish between the works of Ernest Hemingway and
the works of John Milton on the basis of sentence length. Milton
often wrote paragraph-length sentences, in which the page of a book
might have only three or four sentences.
===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===
(12) From: Antoni Basinski <ashb@ices.on.ca>
You refer to a class of problems broadly known as the "authorship
problem" The earliest reference that I know of is Mosteller F,
Wallace DL 1963: Inference in an authorship problem. JASA 58,
275-309. I'm sure there are numerous other references (perhaps you
could check under "authorship" in the CIS). I believe that Mosteller
has a number of papers in this area
I'd be interested in what you find.
Good luck
===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===
(13) From: gordon.harrington@uni.edu
Hopefully someone else will be able to point you to specific
literature. If not it may be helpful to know that analysis of word
counts, word length, etc., became big in the mid to late 50's when
Harvard had just added "Psycholinguistics" to its basket of Ph.D.
programs. I had my students do such an analysis of "Robinson Crusoe"
in the late 50's. They compared with data from diaries of normal
lives and of long term isolates and found differences which were also
reflected in different time periods in the novel. If I remember
correctly some of such linguistic data were referenced in David
McClelland's "Personality".
===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===
(14) From: Tom Burr <tburr@nis.lanl.gov>
Can't help you just off the top of my head just where, but I've
definitely seen studies of multiple authors, each with multiple
works, with each work represented by "n-grams" (frequency counts of
each 2-letter word, 3 letter word, ... n-letter word, where "word"
might be loosely defined). That setup is ready for any of many
possible pattern recognition methods and if memory serves, virtually
any "reasonable" n-gram representation will lead to good author
"signatures" (good pattern recognition performance) when you would
expect it (ie, Shakespeare uses different words and word lengths than
Hemingway for example), so your:
> The surprizing conclusion of the study was that the two
> authors could ***not*** be distinguished on this basis.
is a surprise if we compare Shakespeare to Hemingway, but
maybe not if we compare ?? to Hemingway.
I'll try to find a reference, but the "text retrieval" community
is a good start:
http://www.soi.city.ac.uk/research/cisr/trec/trec.html
They consider much "harder" problems than "retrive all documents
that "look like" (in n-gram sense) query document (could easily set
up to see if 2 or more authors have distinct n-gram signatures)
===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===
(15) From: "Kurbat, Matthew A." <matthew.a.kurbat@ncmi.com>
I haven't seen that article, but have seen two related articles. One
was about whether Daniel Boone authored various documents - it was in
the latest issue of "Chance" magazine (an AmStat pub.).
The other was in a book called "Statistics: A Guide to the Unknown"
(it was an anthology; names of the editors escape me).
===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===+===
-----------------------------------------------------------------------
This message was distributed by s-news@wubios.wustl.edu. To unsubscribe
send e-mail to s-news-request@wubios.wustl.edu with the BODY of the
message: unsubscribe s-news
