[19156] in s-news-athena
Re: [S] Comparing poisson, quasi-log-mu, and negative binomial glm
daemon@ATHENA.MIT.EDU (Bill Venables)
Wed Sep 8 20:24:36 1999
Message-Id: <199909090015.KAA06194@snowy.nsw.cmis.CSIRO.AU>
To: s-news@wubios.wustl.edu
In-Reply-To: Your message of "Wed, 08 Sep 1999 10:53:00 MST."
<19990908175300.24590.rocketmail@web220.mail.yahoo.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Date: Thu, 09 Sep 1999 10:13:19 +1000
From: Bill Venables <William.Venables@cmis.CSIRO.AU>
Content-Transfer-Encoding: 8bit
>
> Dear Members of the S+ community:
>
> For the analysis of count data, the Splus glm function offers, at
> least, three useful alternatives, analyzable in three main families:
> Poisson, negative binomial, and quasi (with a log-link & a
> mu-variance). The negative binomial option is available in the glm.nb
> function of the Venables & Ripley mass library. The glm.nb and the
> quasi-log-mu are useful for overdispersed data.
....
>
> Questions:
> 1. How to compare, particularly the negative binomial and quasi-log-mu
> models? What criteria are most advisable to indicate that one Splus glm
> model is superior to the other?
A few picky points first. It's not an `Splus glm model'. Splus is a piece of
software with nothing to do with models and `glm' already has the word "model"
embedded in it, so `glm model' is in the same category as `PIN number'....
Also negative binomial models are, strictly speaking, outside the glm family,
unless the theta parameter is known.
Now for more serious issues. For the same linear structure negative binomial
and quasi-likelihood models with log link and variance proportional to the mean
have the same numbers of parameters, and comparing models with the same
parametric degree can be subtle. It requires you to say very clearly what you
really mean by one model being "superior" to the other. Presented with this
problem I think my first approach would be informal and fairly graphical. How
well both models shape up for prediction would be a primary consideration and
you can clearly explore that issue by all kinds of informal computational and
graphical methods.
There is an old paper by David Cox on this issue, known as "testing separate
families of hypotheses" that has periodically generated some sparks of interest
but the idea has really yet to take off very seriously. A few references are
given below (but not the original of David Cox).
> 2. I saw, for example, that some authors (who use other software)
> evaluate the models through a comparison of the models’
> log-likelihoods. Are such comparisons appropriate?
Likelihood ratio tests are primarily for nested hypotheses, that is, when the
inner model is strictly a special case of the outer. Since likelihood is only
defined up to a multiplicative constant, using likelihood to compare non-nested
models (as you have here) first requires you to decide on some way of
normalizing them so that they can be compared. This could be done by defining
a super model that contains both of your models as special cases. With one
model a genuine parametric model and the other as a quasi-model, this dodge is
likely to be an interesting if minor research question. Over to you.
> 3. If comparisons of log-likelihoods would be appropriate, the Splus
> “summary” does not provide the log-likelihoods for the Poisson and the
> quasi-log-mu models. Other software packages (e.g., LIMDEP) routinely
> provide log-likelihoods for the Poisson models. Can log-likelihoods be
> generated for the Splus Poisson and quasi-log-mu models? If so, how?
Routinely providing log-likelihoods is useless unless you know how they have
been normalized.
For Poisson models the deviance *is* a log-likelihood, normalized so that it is
zero at the saturated model. This normalization is not possible for negative
binomial models, though, since the parameters are not identified at the
saturated model (in fact you could choose to estimate theta by equating the
deviance to n-p, as some authors do; the deviance provided by glm.nb is
generally of no interest, except as a very informal and rough test of fit by
comparing it with n-p). For quasi-likelihood models the deviance is *not* a
log-likelihood, not even a "log-quasi-likelihood", (though it would be if the
scale parameter were known and its value 1, as is the case for Poisson).
Of course it is (trivially) possible to provide log-likelihoods (or
log-quasi-likelihoods) for all of these models, but before you do so you must
first come to some agreement, again, on how they should be normalized and that
really is quite arbitrary.
>
> Answers to any or all of these questions would be most appreciated.
Here are some of the references I mentioned above, courtesy of CIS.
@Article{Jack:68:SRT,
Author = {Jackson, O. A. Y.},
Title = {{Some} Results on Tests of Separate Families of Hypotheses},
Journal= Biomtrka,
Year = 1968,
Volume = 55,
Pages = {355--363},
Keyword= {Lognormal distribution, Exponential distributions},
}
@Article{Dyer:73:DPS,
Author = {Dyer, Alan R.},
Title = {{Discrimination} Procedures for Separate Families of Hypotheses},
Journal= JASA,
Year = 1973,
Volume = 68,
Pages = {970--974},
Keyword= {Invariance, Monte Carlo, Kolmogorov-Smirnov test, Likelihood ratio
test},
}
@Article{Chen:80:TSF,
Author = {Chen, William},
Title = {{On} the Tests of Separate Families of Hypotheses With Small Sample
Size},
Journal= JStCmpSm,
Year = 1980,
Volume = 11,
Pages = {183--187},
Keyword= {Lognormal distribution, Exponential distribution},
}
@Article{Epps:Sing:Pull:82:TSF,
Author = {Epps, T. W. and Singleton, K. J. and Pulley, L. B.},
Title = {{A} Test of Separate Families of Distributions Based on the
Empirical
Moment Generating Function},
Journal= Biomtrka,
Year = 1982,
Volume = 69,
Pages = {391--399},
Keyword= {Goodness-of-fit},
}
@Article{Sawy:83:TSF,
Author = {Sawyer, K. R.},
Title = {{Testing} Separate Families of Hypotheses: {An} Information
Criterion},
Journal= JRSS-B,
Year = 1983,
Volume = 45,
Pages = {89--99},
}
@Article{Loh:85:NMT,
Author = {Loh, Wei-Yin},
Title = {{A} New Method for Testing Separate Families of Hypotheses},
Journal= JASA,
Year = 1985,
Volume = 80,
Pages = {362--368},
Keyword= {Bootstrap, Spline},
}
@Article{Pace:Salv:90:BCT,
Author = {Pace, L. and Salvan, A.},
Title = {{Best} Conditional Tests for Separate Families of Hypotheses},
Journal= JRSS-B,
Year = 1990,
Volume = 52,
Pages = {125--134},
Keyword= {Exponential family, Similar test},
}
@Article{Al-K:Hwan:91:SPT,
Author = {Al-Khalidi, H. R. and Hwang, L. J.},
Title = {{Some} Pitfalls of Tests of Separate Families of Hypotheses:
{Normality} Vs. Lognormality},
Journal= CommStA,
Year = 1991,
Volume = 20,
Pages = {2505--2528},
Keyword= {Likelihood ratio test},
}
--
-----------------------------------------------------------------
Bill Venables, Statistician, CMIS Environmetrics Project.
Physical address: Postal address:
CSIRO Marine Laboratories, PO Box 120,
233 Middle St, Cleveland, Queensland Cleveland, Qld, 4163
AUSTRALIA AUSTRALIA
Telephone: +61 7 3826 7251 Email: Bill.Venables@cmis.csiro.au
Fax: +61 7 3826 7304
-----------------------------------------------------------------------
This message was distributed by s-news@wubios.wustl.edu. To unsubscribe
send e-mail to s-news-request@wubios.wustl.edu with the BODY of the
message: unsubscribe s-news
