[19180] in s-news-athena
RE: [S] Pr(F) in anova()
daemon@ATHENA.MIT.EDU (Doug Moog)
Fri Sep 10 14:39:05 1999
Message-Id: <4.1.19990910135136.00945260@pop.cwru.edu>
Date: Fri, 10 Sep 1999 14:32:01 -0400
To: s-news@wubios.wustl.edu
From: Doug Moog <dbm3@po.cwru.edu>
In-Reply-To: <B5E9E2B78CD2D2118DDE00104B66A967F17C@WARATAHINTERNET>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
I asked:
>I'm running Splus 2000, Windows 95, and I'm confused about what I'm reading
>in the output of anova() applied to a linear model, lm(). I will run a
>model with one or more terms and get a certain set of F and Pr(F) values.
>Then when I add a new term to the model, the F values of the previous terms
>increase, and their Pr(F) values decrease. But since the terms are added
>sequentially, shouldn't the F values remain the same when you add a new
>term? How could the earlier terms be affected by the later terms? Or is it
>not sequential after all? The anova() output says "Terms added
>sequentially."
I received the correct explanation from Alan Donald, Dave Krantz, George
Tomlins, Bill Venables, and Nick Locantore, whom I quote:
>The F statistic is based on the ratio of two mean squares. In the numerator
>is the mean square due to the model (term or terms), in the denominator is
>the MSE. Recall that the MS* is obtained from the sum of squares divided by
>the degrees of freedom, SS* / df*.
>
>
> Sum of squares (model) / df (model)
>F = ------------------------------------
> Sum of squares (error) / df(Error)
>
>
>S-plus uses the SSE-MSE from the *full model*, changing the SSE and
>df(Error), which will account for the non-static nature of your F and Pr(F).
>The moral of the story is that the denominator of that F ratio is not
>standing still.
Thanks to all who answered.
I know I'll be corrected by the more knowledgeable statisticians here - and
please do - but.... It seems to me like it would be valuable to have a
table with F based on only the terms up to that point. I'm hunting for
explanatory variables from a large set (around 50, but maybe only 10-15
which are prettty much independent) and trying to find a criterion (more
reproducible than my judgement) for which variables to present as
candidates, and would like to give some measure of significance. and
p-values seem to be the best I've come up with. Normally one might use
AIC, but my interest is less in the model than in the variables.
I didn't have a major problem with the changing F values, but I was
wondering what was going on because the process semes so ill-behaved. It
seems like as terms are added and the residual variance decreases, the
extra terms are more likely to have meaningless, accidental correlations to
the residuals. Sometimes, when explained variance gets to what is for my
problem a dubious level (like 85%) the process tends to run away, with all
sorts of new, probably irrelevant terms getting low p-values and
ridiculously low R-squared values.
More broadly, my next question is - what is data mining anyway? I hear
about it all the time, but to me it still seems more like a vague concept
than a science. Are there any guidelines as to how to go about it? Splus
advertising says it's great at it, and I guess that's what I'm doing, but I
feel like I'm flailing around too much, and that I have enough experience
that I *should* be able to employ a major feature of Splus effectively. My
problem is not simply that I don't understand the data I'm examining, but
that I need a procedure which will permit me to convey to colleagues the
objectivity and significance of my findings - and help ensure those in the
first place.
Doug
-----------------------------------------------------------------------
This message was distributed by s-news@wubios.wustl.edu. To unsubscribe
send e-mail to s-news-request@wubios.wustl.edu with the BODY of the
message: unsubscribe s-news
