[18854] in s-news-athena
Re: [S] nls() fails in a loop
daemon@ATHENA.MIT.EDU (Tim Hesterberg)
Wed Aug 4 13:45:00 1999
Date: Wed, 4 Aug 1999 10:37:55 -0700 (PDT)
Message-Id: <199908041737.KAA00546@quake.statsci.com>
From: Tim Hesterberg <timh@statsci.com>
To: s-news@wubios.wustl.edu
Reply-To: timh@statsci.com (Tim Hesterberg)
ROS Mathieu wrote:
>> I'm using nls() in a cross-validation loop (no way to avoid
>> it now...)
Bill Venables suggested:
>1. Don't begin every cv fit with the original rough initial values.
>Always use the best available initial values you have, for
>example, from the primary fit using all the data.
Bruce McCullough wrote:
>As another example, I have bootstrapped a few
>nonlinear models, and I have found that using
>the solution from the previous iteration can
>work well.
The first solution should converge faster in
most applications.
Let B be the solution (vector of parameters) for the original model,
and B_j be the solution for the first bootstrapped vector. Let
E_j = B_j - B. Note that the E's are random vectors, are independent,
and in most applications have mean near zero. Hence |B_j - B| would
tend to be smaller than |B_j - B_{j-1}|, (by a factor of about sqrt(2)
when using a Euclidean norm). Because B is closer to the solution B_j
than is B_{j-1}, using it as a starting value would tend to converge
faster.
If E_j has small variance and mean not near zero, then using the
previous answer, or average of previous answers, should work well.
Brian Ripley wrote:
>Can I suggest that neither approach is in general a valid one for a
>cross-validation? If there are multiple local minima (there often are)
>you are not cross-validating the original procedure (including choosing
>the start) and so will underestimate (possibly severely) the
>variability.
One approach here is to use the fast procedure (using B as a starting
value) for a large set of bootstrap or cross-validation samples, and
also to use original procedure (including choosing the start) on a
subset of those samples. If the results of the fast and slow
procedures on the subset of samples are substantially different, then
one can use the slow procedure on all samples, or do a more
sophisticated analysis based on components of variability.
One such analysis would use the fast results as a control variable
for the slow results, with the necessary expected values for the
fast results estimated from the large set of samples. E.g.
B_j^* = (B_j^* - B_j) + B_j
E(B_j^*) = E(B_j^* - B_j) + E(B_j)
where B_j is the fast result obtained using B as a starting value,
B_j^* is the slow result obtained using the original procedure,
E(B_j) is estimated using a large set of samples using only
the fast procedure, and
E(B_j^* - B_j) is estimated from a subset of samples using both
procedures.
========================================================
| Tim Hesterberg Research Scientist |
| timh@statsci.com MathSoft, Inc. |
| (206)283-8802x319 1700 Westlake Ave. N, Suite 500 |
| (206)283-0347 (fax) Seattle, WA 98109-3044, U.S.A. |
| www.statsci.com/Hesterberg |
========================================================
All statements in this message represent the opinions of the author and do
not necessarily reflect MathSoft policy or position.
-----------------------------------------------------------------------
This message was distributed by s-news@wubios.wustl.edu. To unsubscribe
send e-mail to s-news-request@wubios.wustl.edu with the BODY of the
message: unsubscribe s-news
