[19179] in s-news-athena
Re: [S] Comparing poisson, quasi-log-mu, and negative binomial
daemon@ATHENA.MIT.EDU (Marco Riani)
Fri Sep 10 04:08:31 1999
Message-Id: <3.0.5.32.19990910100917.007dcdc0@ipruniv.cce.unipr.it>
Date: Fri, 10 Sep 1999 10:09:17 +0200
To: Bill Venables <William.Venables@cmis.CSIRO.AU>, s-news@wubios.wustl.edu
From: Marco Riani <statdue@ipruniv.cce.unipr.it>
In-Reply-To: <199909090015.KAA06194@snowy.nsw.cmis.CSIRO.AU>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 8bit
A small addition to Bill Venables' comments about tests of non-nested
hypotheses.
The Cox test (as econometricians call it) has a dibn far from its
asymptotic N(0,1) so it is necessary to simulate to get some idea of the
interpretation of an observed value. Since the test is the ratio of
likelihoods for the two models minus its expectation under the null
hypothesis, all divided by an estimated standard error, it may be just as
sensible to simulate 1,000 values of the likelihood ratio and see where
your observed value is. Or you can just look at a scatter plot of simulated
deviances or loglikelihoods. There is an example for comparing the gamma
and lognormal distributions on p.244 of my book "Plots, Transformations and
Regression". See the end of Chapter 3 of V+R for the full reference to the
book.
Anthony Atkinson
At 10.13 09/09/99 +1000, Bill Venables wrote:
>
>>
>> Dear Members of the S+ community:
>>
>> For the analysis of count data, the Splus glm function offers, at
>> least, three useful alternatives, analyzable in three main families:
>> Poisson, negative binomial, and quasi (with a log-link & a
>> mu-variance). The negative binomial option is available in the glm.nb
>> function of the Venables & Ripley mass library. The glm.nb and the
>> quasi-log-mu are useful for overdispersed data.
>
>....
>
>>
>> Questions:
>> 1. How to compare, particularly the negative binomial and quasi-log-mu
>> models? What criteria are most advisable to indicate that one Splus glm
>> model is superior to the other?
>
>A few picky points first. It's not an `Splus glm model'. Splus is a
piece of
>software with nothing to do with models and `glm' already has the word
"model"
>embedded in it, so `glm model' is in the same category as `PIN number'....
>Also negative binomial models are, strictly speaking, outside the glm family,
>unless the theta parameter is known.
>
>Now for more serious issues. For the same linear structure negative binomial
>and quasi-likelihood models with log link and variance proportional to the
mean
>have the same numbers of parameters, and comparing models with the same
>parametric degree can be subtle. It requires you to say very clearly what
you
>really mean by one model being "superior" to the other. Presented with this
>problem I think my first approach would be informal and fairly graphical.
How
>well both models shape up for prediction would be a primary consideration and
>you can clearly explore that issue by all kinds of informal computational and
>graphical methods.
>
>There is an old paper by David Cox on this issue, known as "testing separate
>families of hypotheses" that has periodically generated some sparks of
interest
>but the idea has really yet to take off very seriously. A few references are
>given below (but not the original of David Cox).
>
>> 2. I saw, for example, that some authors (who use other software)
>> evaluate the models through a comparison of the models’
>> log-likelihoods. Are such comparisons appropriate?
>
>Likelihood ratio tests are primarily for nested hypotheses, that is, when the
>inner model is strictly a special case of the outer. Since likelihood is
only
>defined up to a multiplicative constant, using likelihood to compare
non-nested
>models (as you have here) first requires you to decide on some way of
>normalizing them so that they can be compared. This could be done by
defining
>a super model that contains both of your models as special cases. With one
>model a genuine parametric model and the other as a quasi-model, this
dodge is
>likely to be an interesting if minor research question. Over to you.
>
>> 3. If comparisons of log-likelihoods would be appropriate, the Splus
>> “summary” does not provide the log-likelihoods for the Poisson and the
>> quasi-log-mu models. Other software packages (e.g., LIMDEP) routinely
>> provide log-likelihoods for the Poisson models. Can log-likelihoods be
>> generated for the Splus Poisson and quasi-log-mu models? If so, how?
>
>Routinely providing log-likelihoods is useless unless you know how they have
>been normalized.
>
>For Poisson models the deviance *is* a log-likelihood, normalized so that
it is
>zero at the saturated model. This normalization is not possible for negative
>binomial models, though, since the parameters are not identified at the
>saturated model (in fact you could choose to estimate theta by equating the
>deviance to n-p, as some authors do; the deviance provided by glm.nb is
>generally of no interest, except as a very informal and rough test of fit by
>comparing it with n-p). For quasi-likelihood models the deviance is *not* a
>log-likelihood, not even a "log-quasi-likelihood", (though it would be if the
>scale parameter were known and its value 1, as is the case for Poisson).
>
>Of course it is (trivially) possible to provide log-likelihoods (or
>log-quasi-likelihoods) for all of these models, but before you do so you must
>first come to some agreement, again, on how they should be normalized and
that
>really is quite arbitrary.
>
>>
>> Answers to any or all of these questions would be most appreciated.
>
>Here are some of the references I mentioned above, courtesy of CIS.
>
>@Article{Jack:68:SRT,
> Author = {Jackson, O. A. Y.},
> Title = {{Some} Results on Tests of Separate Families of Hypotheses},
> Journal= Biomtrka,
> Year = 1968,
> Volume = 55,
> Pages = {355--363},
> Keyword= {Lognormal distribution, Exponential distributions},
>}
>
>@Article{Dyer:73:DPS,
> Author = {Dyer, Alan R.},
> Title = {{Discrimination} Procedures for Separate Families of Hypotheses},
> Journal= JASA,
> Year = 1973,
> Volume = 68,
> Pages = {970--974},
> Keyword= {Invariance, Monte Carlo, Kolmogorov-Smirnov test, Likelihood ratio
> test},
>}
>
>@Article{Chen:80:TSF,
> Author = {Chen, William},
> Title = {{On} the Tests of Separate Families of Hypotheses With Small
Sample
> Size},
> Journal= JStCmpSm,
> Year = 1980,
> Volume = 11,
> Pages = {183--187},
> Keyword= {Lognormal distribution, Exponential distribution},
>}
>
>@Article{Epps:Sing:Pull:82:TSF,
> Author = {Epps, T. W. and Singleton, K. J. and Pulley, L. B.},
> Title = {{A} Test of Separate Families of Distributions Based on the
>Empirical
> Moment Generating Function},
> Journal= Biomtrka,
> Year = 1982,
> Volume = 69,
> Pages = {391--399},
> Keyword= {Goodness-of-fit},
>}
>
>@Article{Sawy:83:TSF,
> Author = {Sawyer, K. R.},
> Title = {{Testing} Separate Families of Hypotheses: {An} Information
> Criterion},
> Journal= JRSS-B,
> Year = 1983,
> Volume = 45,
> Pages = {89--99},
>}
>
>@Article{Loh:85:NMT,
> Author = {Loh, Wei-Yin},
> Title = {{A} New Method for Testing Separate Families of Hypotheses},
> Journal= JASA,
> Year = 1985,
> Volume = 80,
> Pages = {362--368},
> Keyword= {Bootstrap, Spline},
>}
>
>@Article{Pace:Salv:90:BCT,
> Author = {Pace, L. and Salvan, A.},
> Title = {{Best} Conditional Tests for Separate Families of Hypotheses},
> Journal= JRSS-B,
> Year = 1990,
> Volume = 52,
> Pages = {125--134},
> Keyword= {Exponential family, Similar test},
>}
>
>@Article{Al-K:Hwan:91:SPT,
> Author = {Al-Khalidi, H. R. and Hwang, L. J.},
> Title = {{Some} Pitfalls of Tests of Separate Families of Hypotheses:
> {Normality} Vs. Lognormality},
> Journal= CommStA,
> Year = 1991,
> Volume = 20,
> Pages = {2505--2528},
> Keyword= {Likelihood ratio test},
>}
>
>--
>-----------------------------------------------------------------
>Bill Venables, Statistician, CMIS Environmetrics Project.
>
>Physical address: Postal address:
>CSIRO Marine Laboratories, PO Box 120,
>233 Middle St, Cleveland, Queensland Cleveland, Qld, 4163
>AUSTRALIA AUSTRALIA
>
>Telephone: +61 7 3826 7251 Email: Bill.Venables@cmis.csiro.au
> Fax: +61 7 3826 7304
>
>
>-----------------------------------------------------------------------
>This message was distributed by s-news@wubios.wustl.edu. To unsubscribe
>send e-mail to s-news-request@wubios.wustl.edu with the BODY of the
>message: unsubscribe s-news
>
>
-----------------------------------------------------------------------
This message was distributed by s-news@wubios.wustl.edu. To unsubscribe
send e-mail to s-news-request@wubios.wustl.edu with the BODY of the
message: unsubscribe s-news
