[27429] in Perl-Users-Digest
Perl-Users Digest, Issue: 9072 Volume: 10
daemon@ATHENA.MIT.EDU (Perl-Users Digest)
Tue Mar 21 14:05:53 2006
Date: Tue, 21 Mar 2006 11:05:05 -0800 (PST)
From: Perl-Users Digest <Perl-Users-Request@ruby.OCE.ORST.EDU>
To: Perl-Users@ruby.OCE.ORST.EDU (Perl-Users Digest)
Perl-Users Digest Tue, 21 Mar 2006 Volume: 10 Number: 9072
Today's topics:
Re: How to release memory ? <rvtol+news@isolution.nl>
Re: How to release memory ? <zentara@highstream.net>
Re: How to release memory ? xhoster@gmail.com
Re: How to release memory ? <zdbrg@mail.con>
joining a hash of arrays <mdudley@king-cart.com>
Re: joining a hash of arrays <jgibson@mail.arc.nasa.gov>
Problems installing Unix::Syslog <jh@dontspam.spam>
Digest Administrivia (Last modified: 6 Apr 01) (Perl-Users-Digest Admin)
----------------------------------------------------------------------
Date: Tue, 21 Mar 2006 13:27:48 +0100
From: "Dr.Ruud" <rvtol+news@isolution.nl>
Subject: Re: How to release memory ?
Message-Id: <dvova7.10c.1@news.isolution.nl>
zdbrg schreef:
> So, how do perl developers handle quite commons tasks where one
> program needs to use/allocate really big variables periodicaly which
> then should be released to let others tasks run
Do you have an example of such a task? Is Perl the right language (is
perl the right implementation) for that task?
--
Affijn, Ruud
"Gewoon is een tijger."
echo 014C8A26C5DB87DBE85A93DBF |perl -pe 'tr/0-9A-F/JunkshoP cartel,/'
------------------------------
Date: Tue, 21 Mar 2006 13:04:41 GMT
From: zentara <zentara@highstream.net>
Subject: Re: How to release memory ?
Message-Id: <4nsv121nr6fpv7d79vvac9ojbg4mu9jh88@4ax.com>
On Tue, 21 Mar 2006 10:47:23 +0100, zdbrg <zdbrg@mail.con> wrote:
>Ilya Zakharevich a dit le Mon, 20 Mar 2006 21:28:26 +0000 (UTC):
>>Good rule is: do not expect your memory going back to
>>system in any real-life situation.
>>
>
>Thanks for your answer.
>
>So, how do perl developers handle quite commons tasks where one program needs
>to use/allocate really big variables periodicaly which then should be released
>to let others tasks run
>?
Reuse the space yourself, commonly by using a $global to hold the data.
When you are done with the first $data run, set $data = '', and do the
next task, using $data again. The memory from the first use of $data
will be freed internally by Perl, and will be reused, but it "probably"
won't be returned to the system.
So it is common in Perl programs, to see the system memory usage rise to
a peak, of whatever the greatest data size was. Like a peak-meter.
Usually in complex programs, objects are being used. There are various
tricks you learn to reuse the same object over-and-over, and just undef
the data portion of the object. You cannot reliably use the
"object-create-destroy" cycle, without gainig memory. Each object has
it's own technique for clearing out old data.
I'm not a Perl internals type of person, but the developers are trying
to improve Perl's memory conservation. For instance in Perl/Tk, it is
absolutely essential to reuse any object you create. While Perl/Gtk2
has made significant strides in cleaning itself up automatically, so you
can often use the create-destroy method of object handling.
Perl6 has made promises of improvements across the board.
>
>Starting a new process for those jobs which are known to use a lot of memory is
>a solution, is there others
>?
It is common to use disk-based databases to handle sharing of huge data.
That way you can fork to do whatever you need, and when the forked
process is done, it leaves it's results in the db, and totally returns
the memory to the system.
Another way to share forked data, is thru shared memory segments, but
they are tricky.
Threads will act like objects. If you continually create threads, then
destroy them when done, memory will just climb. You need to reuse
threads, just feeding them different data for each run. Threads are
great for sharing data in realtime, but they will suck system memory if
not carefully handled. Once again, the memory will rise to peak value.
So there is not going to be 1 general purpose answer to your question.
Each program needs to make it's own tradeoffs, concerning size,
performance, ease of sharing data, etc.
But for programs which use 100's of megs of data, you are best off
forking them, and storing results in a common disk-file-db. The Storable
module is often used for this.
Good luck.
--
I'm not really a human, but I play one on earth.
http://zentara.net/japh.html
------------------------------
Date: 21 Mar 2006 14:50:07 GMT
From: xhoster@gmail.com
Subject: Re: How to release memory ?
Message-Id: <20060321095746.047$Xy@newsreader.com>
zdbrg <zdbrg@mail.con> wrote:
> Ilya Zakharevich a dit le Mon, 20 Mar 2006 21:28:26 +0000 (UTC):
> >Good rule is: do not expect your memory going back to
> >system in any real-life situation.
> >
>
> Thanks for your answer.
>
> So, how do perl developers handle quite commons tasks where one program
> needs to use/allocate really big variables periodicaly which then should
> be released to let others tasks run
> ?
Mostly I just let the VM system do it's job. Are you sure that won't work
for you?
Xho
--
-------------------- http://NewsReader.Com/ --------------------
Usenet Newsgroup Service $9.95/Month 30GB
------------------------------
Date: Tue, 21 Mar 2006 17:28:25 +0100
From: zdbrg <zdbrg@mail.con>
Subject: Re: How to release memory ?
Message-Id: <442029a9$0$18999$626a54ce@news.free.fr>
xhoster@gmail.com a dit le 21 Mar 2006 14:50:07 GMT:
>Mostly I just let the VM system do it's job. Are you sure that won't work
>for you?
>
>Xho
Yes it will, at the cost of swapping hundred megs of memory to disk.
As zentara pointed out, using some kind of persistant database (which was not
wanted at the first time), will surely drop down memory use, adding a bit of
complexity, and probably slowlyness. Forking or using a crontab (on *nix
systems) for that is surely another good solution.
Thanks all for your interesting answers.
------------------------------
Date: Tue, 21 Mar 2006 11:50:06 -0500
From: Marshall Dudley <mdudley@king-cart.com>
Subject: joining a hash of arrays
Message-Id: <44202EBE.DAFB49FE@king-cart.com>
How do you join an hash of arrays, I can't seem to find, nor figure out
the syntax.
When I try:
my $color = join ' ',@{ $color{$id}};
and the array for $id contains "white","black" I get:
ARRAY(0x8051878)
instead of "white black".
Thanks,
Marshall
------------------------------
Date: Tue, 21 Mar 2006 09:31:00 -0800
From: Jim Gibson <jgibson@mail.arc.nasa.gov>
Subject: Re: joining a hash of arrays
Message-Id: <210320060931005273%jgibson@mail.arc.nasa.gov>
In article <44202EBE.DAFB49FE@king-cart.com>, Marshall Dudley
<mdudley@king-cart.com> wrote:
> How do you join an hash of arrays, I can't seem to find, nor figure out
> the syntax.
>
> When I try:
>
> my $color = join ' ',@{ $color{$id}};
>
> and the array for $id contains "white","black" I get:
>
> ARRAY(0x8051878)
>
> instead of "white black".
I don't get that. It would be best if you could show a complete program
that demonstrates the problem you are having. For example:
#!/usr/local/bin/perl
use warnings;
use strict;
my %color = ( a => [ qw( white black ) ] );
my $id = 'a';
my $color = join ' ', @{ $color{$id}};
print "$color\n";
__OUTPUT__
white black
Posted Via Usenet.com Premium Usenet Newsgroup Services
----------------------------------------------------------
** SPEED ** RETENTION ** COMPLETION ** ANONYMITY **
----------------------------------------------------------
http://www.usenet.com
------------------------------
Date: Tue, 21 Mar 2006 10:21:27 -0500
From: J Huntley Palmer <jh@dontspam.spam>
Subject: Problems installing Unix::Syslog
Message-Id: <12206fqblh7v93d@news.supernews.com>
I cannot install Unix::Sys log. Any help appreciated.
My cpan install terminates with the following error:
--------------------------------------------------------------------------
cpan> install Unix::Syslog
Running install for module Unix::Syslog
Running make for M/MH/MHARNISCH/Unix-Syslog-0.99.tar.gz
Is already unwrapped into directory /.cpan/build/Unix-Syslog-0.99
Has already been processed within this session
Running make test
PERL_DL_NONLAZY=1 /usr/bin/perl "-Iblib/lib" "-Iblib/arch" test.pl
1..
ok 1
Testing priorities:
LOG_EMERG ok 2
LOG_ALERT ok 3
LOG_CRIT ok 4
LOG_ERR ok 5
LOG_WARNING ok 6
LOG_NOTICE ok 7
LOG_INFO ok 8
LOG_DEBUG ok 9
Testing facilities
LOG_KERN ok 10
LOG_USER ok 11
LOG_MAIL ok 12
LOG_DAEMON ok 13
LOG_AUTH ok 14
LOG_SYSLOG ok 15
LOG_LPR ok 16
LOG_NEWS ok 17
LOG_UUCP ok 18
LOG_CRON ok 19
LOG_LOCAL0 ok 20
LOG_LOCAL1 ok 21
LOG_LOCAL2 ok 22
LOG_LOCAL3 ok 23
LOG_LOCAL4 ok 24
LOG_LOCAL5 ok 25
LOG_LOCAL6 ok 26
LOG_LOCAL7 ok 27
These facilities are not defined on all systems:
LOG_AUTHPRIV not ok 28
LOG_FTP not ok 29
The number of available facilities is:
LOG_NFACILITIES ok 30
LOG_FACMASK ok 31
Testing options
LOG_PID ok 32
LOG_CONS ok 33
LOG_ODELAY ok 34
LOG_NDELAY ok 35
LOG_NOWAIT ok 36
These options are not defined on all systems:
LOG_PERROR not ok 37
Testing macros for setlogmask()
LOG_MASK ok 38
LOG_UPTO ok 39
These macros are not defined on all systems:
LOG_PRI not ok 40
LOG_MAKEPRI not ok 41
LOG_FAC not ok 42
On some systems these functions return just empty strings:
priorityname(LOG_EMERG): not defined. skipped 43
priorityname(LOG_ALERT): not defined. skipped 44
priorityname(LOG_CRIT): not defined. skipped 45
priorityname(LOG_ERR): not defined. skipped 46
priorityname(LOG_WARNING): not defined. skipped 47
priorityname(LOG_NOTICE): not defined. skipped 48
priorityname(LOG_INFO): not defined. skipped 49
priorityname(LOG_DEBUG): not defined. skipped 50
facilityname(LOG_KERN): not defined. skipped 51
facilityname(LOG_USER): not defined. skipped 52
facilityname(LOG_MAIL): not defined. skipped 53
facilityname(LOG_DAEMON): not defined. skipped 54
facilityname(LOG_AUTH): not defined. skipped 55
facilityname(LOG_SYSLOG): not defined. skipped 56
facilityname(LOG_LPR): not defined. skipped 57
facilityname(LOG_NEWS): not defined. skipped 58
facilityname(LOG_UUCP): not defined. skipped 59
facilityname(LOG_CRON): not defined. skipped 60
Can't locate auto/Unix/Syslog/LOG_AUTHPRI.al in @INC (@INC contains:
blib/lib blib/arch
/opt/ActivePerl-5.8/lib/5.8.7/sun4-solaris-thread-multi
/opt/ActivePerl-5.8/lib/5.8.7
/opt/ActivePerl-5.8/lib/site_perl/5.8.7/sun4-solaris-thread-multi
/opt/ActivePerl-5.8/lib/site_perl/5.8.7
/opt/ActivePerl-5.8/lib/site_perl .) at test.pl line 110
make: *** [test_dynamic] Error 2
/usr/local/bin/make test -- NOT OK
Running make install
make test had returned bad status, won't install without force
Failed during this command:
MHARNISCH/Unix-Syslog-0.99.tar.gz : make_test NO
------------------------------
Date: 6 Apr 2001 21:33:47 GMT (Last modified)
From: Perl-Users-Request@ruby.oce.orst.edu (Perl-Users-Digest Admin)
Subject: Digest Administrivia (Last modified: 6 Apr 01)
Message-Id: <null>
Administrivia:
#The Perl-Users Digest is a retransmission of the USENET newsgroup
#comp.lang.perl.misc. For subscription or unsubscription requests, send
#the single line:
#
# subscribe perl-users
#or:
# unsubscribe perl-users
#
#to almanac@ruby.oce.orst.edu.
NOTE: due to the current flood of worm email banging on ruby, the smtp
server on ruby has been shut off until further notice.
To submit articles to comp.lang.perl.announce, send your article to
clpa@perl.com.
#To request back copies (available for a week or so), send your request
#to almanac@ruby.oce.orst.edu with the command "send perl-users x.y",
#where x is the volume number and y is the issue number.
#For other requests pertaining to the digest, send mail to
#perl-users-request@ruby.oce.orst.edu. Do not waste your time or mine
#sending perl questions to the -request address, I don't have time to
#answer them even if I did know the answer.
------------------------------
End of Perl-Users Digest V10 Issue 9072
***************************************
