[6262] in s-news-athena
big logistic regression; out of memory problem
daemon@ATHENA.MIT.EDU (Simon Jackman)
Thu Jan 26 01:55:09 1995
Date: Thu, 26 Jan 1995 00:25:35 -0600 (CST)
From: Simon Jackman <acbsjack@gosnell.spc.uchicago.edu>
To: S-news <s-news@utstat.toronto.edu>
In-Reply-To: <Pine.3.89.9501250910.C4551-0100000@chinook.alz.washington.edu>
Possibly a no-brainer or a no-way-out...
Running Splus 3.1 on a HP 715/64 with 64M of memory... I have a big data
set on voter turnout (about 100,000 obs) and try to run a logistic
regression using glm(), and it dumps with the following error as it tries
to grab a big chunk of memory for storing (something) in double precision:
Error in double(2 * n + max(n, 2 * p)): Unable to obtain requested dynamic
memory
Dumped
I had to play with options(object.size) to get that far, but now I am
stumped. There has to be a way...please...
Got frustrated and cranked up SAS on the same platform; it ran just fine
(and pretty quickly):
NOTE: The data set WORK.TEMP has 98857 observations and 11 variables.
NOTE: DATA statement used:
real time 16.18 seconds
cpu time 15.90 seconds
17 proc logistic descending simple;
18 model vote=educ educ2 age age2 south gov close educcl educcl2
19 / itprint lackfit ctable;
20 run;
NOTE: PROC LOGISTIC is modeling the probability that VOTE=1.
NOTE: At least one W.D format was too small for the number to be
printed. The decimal may be shifted by the "BEST" format.
NOTE: PROCEDURE LOGISTIC used:
real time 57.24 seconds
cpu time 53.81 seconds
it would be nice to be able to do this from Splus...
any help greatly appreciated. regards -- simon jackman
========================================================================
Simon Jackman, Dept of Political Science, Univ of Chicago, 5828
S.University Ave, Chicago, IL 60637. ph: 312.702.8075. fax: x.x.1689
