[23818] in Perl-Users-Digest
Perl-Users Digest, Issue: 6021 Volume: 10
daemon@ATHENA.MIT.EDU (Perl-Users Digest)
Thu Jan 29 20:46:29 2004
Date: Thu, 29 Jan 2004 17:40:53 -0800 (PST)
From: Perl-Users Digest <Perl-Users-Request@ruby.OCE.ORST.EDU>
To: Perl-Users@ruby.OCE.ORST.EDU (Perl-Users Digest)
Perl-Users Digest Thu, 29 Jan 2004 Volume: 10 Number: 6021
Today's topics:
list context <gnmo@pml.ac.uk>
Re: list context <ittyspam@yahoo.com>
Re: list context <tadmc@augustmail.com>
Re: list context <dwall@fastmail.fm>
Re: list context <gnari@simnet.is>
Re: list context <gnmo@pml.ac.uk>
Re: Lo de Urgencias ha sido de OSCAR!!!!!!!!!!!!´(Spoil <alter.hadaqUiTaReStO@tiscali.es>
Log File Trimming <nospam.hciss@yahoo.com>
Re: Log File Trimming <uri@stemsystems.com>
Re: Log File Trimming <tore@aursand.no>
LWP, CGI and file uploads under windows (Yehuda Berlinger)
Re: LWP, CGI and file uploads under windows (Yehuda Berlinger)
LWP::Request technical question? (at)FinancialDataCorp.com (Bob Mariotti)
Mail::Sendmail cant send to yahoo.co.uk addresses <pdconetwofour_numbers_@yahoo.co.uk>
Re: Mail::Sendmail cant send to yahoo.co.uk addresses <pdconetwofour_numbers_@yahoo.co.uk>
Re: Mail::Sendmail cant send to yahoo.co.uk addresses (David Efflandt)
Memory allocation <blevin@lucent.com>
Re: Memory allocation <tassilo.parseval@rwth-aachen.de>
Re: Memory allocation <blevin@lucent.com>
Re: Memory allocation <usenet@morrow.me.uk>
Re: Memory allocation <syscjm@gwu.edu>
Re: Memory allocation (Walter Roberson)
Re: Memory Mgmt Using C and Perl <kalinaubears@iinet.net.au>
Re: Memory Mgmt Using C and Perl <mshelor@comcast.removeme.net>
Re: Memory Mgmt Using C and Perl <kalinaubears@iinet.net.au>
Re: Memory Mgmt Using C and Perl <tassilo.parseval@rwth-aachen.de>
Re: Memory Mgmt Using C and Perl <kalinaubears@iinet.net.au>
Digest Administrivia (Last modified: 6 Apr 01) (Perl-Users-Digest Admin)
----------------------------------------------------------------------
Date: Fri, 23 Jan 2004 17:19:27 +0000
From: Gareth Mottram - RSG <gnmo@pml.ac.uk>
Subject: list context
Message-Id: <4011579e$1@news.nwl.ac.uk>
Hi all I am having a bit of a problem in putting an array into a hash
structure
<snip>
for($n=0;$n <= 10;$n++)
{
$l[$n]=$n;
}
%current=();
$current{cloud}=(@l);
@x=(@l);
print " @x\n";
print $current{cloud};
</snip
@x happily contains the results of the loop whereas $current{cloud}
stubornly refuses to contain anything but the scalar of the list.
I am now very confused and my brain has melted and leaked all over the
table. Can anyone explain where I am making<pun> a hash of this?</pun>
cheers
g
--
Gareth N. Mottram
Support Officer
Remote Sensing Data Analysis Service
Plymouth Marine Laboratory
Prospect Place
Plymouth
Devon, PL1 3DH
UK
Tel : ++44 (0)1752 633485
Fax : ++44 (0)1752 633101
E-mail: gnmo@pml.ac.uk
Web : http://www.npm.ac.uk/rsdas/
Registered Charity No. 1091222
Company No. 4178503
------------------------------
Date: Fri, 23 Jan 2004 12:42:37 -0500
From: Paul Lalli <ittyspam@yahoo.com>
Subject: Re: list context
Message-Id: <20040123123901.H15931@dishwasher.cs.rpi.edu>
On Fri, 23 Jan 2004, Gareth Mottram - RSG wrote:
> Hi all I am having a bit of a problem in putting an array into a hash
> structure
>
> <snip>
> for($n=0;$n <= 10;$n++)
> {
> $l[$n]=$n;
> }
> %current=();
> $current{cloud}=(@l);
> @x=(@l);
> print " @x\n";
> print $current{cloud};
> </snip
>
> @x happily contains the results of the loop whereas $current{cloud}
> stubornly refuses to contain anything but the scalar of the list.
> I am now very confused and my brain has melted and leaked all over the
> table. Can anyone explain where I am making<pun> a hash of this?</pun>
>
You can't put an array inside of a hash. Hash keys and hash values are
each scalars. What you want to do is put a reference to that array into
your hash:
$current{cloud} = \@l;
You would then dereference the entire thing with:
print @{$current{cloud}}, "\n";
for more info: perldoc perldsc
(the data structures cookbook)
Paul Lalli
------------------------------
Date: Fri, 23 Jan 2004 13:31:04 -0600
From: Tad McClellan <tadmc@augustmail.com>
Subject: Re: list context
Message-Id: <slrnc12tjo.2u9.tadmc@magna.augustmail.com>
Gareth Mottram - RSG <gnmo@pml.ac.uk> wrote:
> Hi all I am having a bit of a problem in putting an array into a hash
> structure
>
><snip>
> for($n=0;$n <= 10;$n++)
You should always have "use strict" enabled when developing Perl code.
You are writing C in Perl. A more Perlish way is:
for my $n ( 0 .. 10 ) {
> {
> $l[$n]=$n;
> }
> %current=();
> $current{cloud}=(@l);
Scalar on the LHS, so the RHS is in scalar context.
Maybe this is what you wanted to do?
%current = @l;
> Can anyone explain where I am making<pun> a hash of this?</pun>
You wanted list context. You got scalar context.
--
Tad McClellan SGML consulting
tadmc@augustmail.com Perl programming
Fort Worth, Texas
------------------------------
Date: Fri, 23 Jan 2004 19:45:39 -0000
From: "David K. Wall" <dwall@fastmail.fm>
Subject: Re: list context
Message-Id: <Xns947996263B06Cdkwwashere@216.168.3.30>
Tad McClellan <tadmc@augustmail.com> wrote:
> Gareth Mottram - RSG <gnmo@pml.ac.uk> wrote:
>> Hi all I am having a bit of a problem in putting an array into a hash
>> structure
>>
[snip]
>> %current=();
>> $current{cloud}=(@l);
>
>
> Scalar on the LHS, so the RHS is in scalar context.
>
> Maybe this is what you wanted to do?
>
> %current = @l;
Who knows? But that term on the RHS is weird. It would be even weirder if
the OP had written something like
@k = (1..4);
@l = (0..10);
$current{cloud} = (@k, @l);
which would put 11 in $current{cloud}, not 15 or 2. Context, commas, and
terms, oh my!
--
David Wall
------------------------------
Date: Fri, 23 Jan 2004 19:13:51 -0000
From: "gnari" <gnari@simnet.is>
Subject: Re: list context
Message-Id: <burrv0$du2$1@news.simnet.is>
"Paul Lalli" <ittyspam@yahoo.com> wrote in message
news:20040123123901.H15931@dishwasher.cs.rpi.edu...
> On Fri, 23 Jan 2004, Gareth Mottram - RSG wrote:
>
> $current{cloud} = \@l;
>
> You would then dereference the entire thing with:
>
> print @{$current{cloud}}, "\n";
and $current{cloud}[2] to access one element
>
> for more info: perldoc perldsc
and
perldoc perllol
perldoc perlreftut
perldoc perlref
gnari
------------------------------
Date: Mon, 26 Jan 2004 09:39:46 +0000
From: Gareth Mottram - RSG <gnmo@pml.ac.uk>
Subject: Re: list context
Message-Id: <4014e063@news.nwl.ac.uk>
Cheers everyone for your help, that makes a fair bit of sense
g
Gareth Mottram - RSG wrote:
> Hi all I am having a bit of a problem in putting an array into a hash
> structure
>
> <snip>
> for($n=0;$n <= 10;$n++)
> {
> $l[$n]=$n;
> }
> %current=();
> $current{cloud}=(@l);
> @x=(@l);
> print " @x\n";
> print $current{cloud};
> </snip
>
> @x happily contains the results of the loop whereas $current{cloud}
> stubornly refuses to contain anything but the scalar of the list.
> I am now very confused and my brain has melted and leaked all over the
> table. Can anyone explain where I am making<pun> a hash of this?</pun>
>
> cheers
>
> g
>
>
--
Gareth N. Mottram
Support Officer
Remote Sensing Data Analysis Service
Plymouth Marine Laboratory
Prospect Place
Plymouth
Devon, PL1 3DH
UK
Tel : ++44 (0)1752 633485
Fax : ++44 (0)1752 633101
E-mail: gnmo@pml.ac.uk
Web : http://www.npm.ac.uk/rsdas/
Registered Charity No. 1091222
Company No. 4178503
------------------------------
Date: Thu, 29 Jan 2004 11:33:13 GMT
From: "alter" <alter.hadaqUiTaReStO@tiscali.es>
Subject: Re: Lo de Urgencias ha sido de OSCAR!!!!!!!!!!!!´(Spoiler)
Message-Id: <Zb6Sb.21$Ic.3317@news.ono.com>
Una cosa, en el capítulo anterior Carter dice que la carta es del Dr. Greene
y luego lee lo que ha escrito Elizabeth en otra carta que dice que eso lo
escribió por la tarde antes de morir y todoeso... pero en este último
capítulo Mark pide ayuda a Elizabeth para dejar cartas escritas para sus
hijas porque él ya no puede, no han metido un poco la pata?
------------------------------
Date: Tue, 27 Jan 2004 23:04:21 -0600
From: "Matt" <nospam.hciss@yahoo.com>
Subject: Log File Trimming
Message-Id: <101eglo7mktda52@corp.supernews.com>
I have a very large log file. Once a day I would like to parse it and
delete all the old stuff over a month old.
It has entries like this in it.
Thu Oct 16 01:31:38 2003
Start port 5
End port 10
blah
Thu Oct 16 01:45:08 2003
Start port 99
End Port 3
blah
It just keeps appending to the end of the file. I would like to delete all
entries older then 1 month. That would mean deleting everything located
above that date. I think it would be a fairly simple task of searching for
the first date younger then the cut off date. Then the hard part of
deleting everything above that line then commiting to disk. How would you
do that?
Matt
------------------------------
Date: Wed, 28 Jan 2004 06:21:13 GMT
From: Uri Guttman <uri@stemsystems.com>
Subject: Re: Log File Trimming
Message-Id: <x7broo4mqv.fsf@mail.sysarch.com>
>>>>> "M" == Matt <nospam.hciss@yahoo.com> writes:
M> It just keeps appending to the end of the file. I would like to
M> delete all entries older then 1 month. That would mean deleting
M> everything located above that date. I think it would be a fairly
M> simple task of searching for the first date younger then the cut
M> off date. Then the hard part of deleting everything above that
M> line then commiting to disk. How would you do that?
you have an interesting problem. the trouble is that your log
generator/server is still writing to the file at the seek point it knows
about. so it may leave a major gap in the file if you delete the earlier
chunk. or your server could be reopening the log file each time which
solves the first problem but you then have a race condition regarding
the removal of the early entries. but you could then copy the older
entries to a new file and rename it to the old file which is
atomic. this only works if the server reopens the file. many servers
have a way to trigger them to rotate a log file i.e. close the current
one, rename it (with some date or cycle suffix) and open a new empty log
file.
so it all comes down to how the server is managing and writing to the
log file. we would need more info on that before a proper solution can
be discussed.
uri
--
Uri Guttman ------ uri@stemsystems.com -------- http://www.stemsystems.com
--Perl Consulting, Stem Development, Systems Architecture, Design and Coding-
Search or Offer Perl Jobs ---------------------------- http://jobs.perl.org
------------------------------
Date: Wed, 28 Jan 2004 10:10:08 +0100
From: Tore Aursand <tore@aursand.no>
Subject: Re: Log File Trimming
Message-Id: <pan.2004.01.28.08.51.53.585074@aursand.no>
On Tue, 27 Jan 2004 23:04:21 -0600, Matt wrote:
> I have a very large log file. Once a day I would like to parse it and
> delete all the old stuff over a month old.
> [...]
Here's how I would have solved it:
1. Process each line in the log file, looking for a date. [1]
2. If date is inside the wanted range, one month old or newer [2],
set a 'write' flag.
3. If the 'write' flag is set, write the line to a new file.
4. When done reading through the log file, delete the it and
rename the new file to the name as the log file.
[1] Date::Parse
[2] Date::Calc
--
Tore Aursand <tore@aursand.no>
"I know not with what weapons World War 3 will be fought, but World War
4 will be fought with sticks and stones." -- Albert Einstein
------------------------------
Date: 28 Jan 2004 07:44:57 -0800
From: jon@actcom.co.il (Yehuda Berlinger)
Subject: LWP, CGI and file uploads under windows
Message-Id: <75e50dfd.0401280744.4177bf3b@posting.google.com>
Hi. I am using the latest activestate perl, and I am trying to create
a client and a server that will do file uploading via http.
My client looks like:
my $UA = LWP::UserAgent->new;
my %fields = (fn => $fn, upload => [$upload]);
my $res = $UA->post($URL_Server, \%fields, 'Content_Type' =>
'form_data');
The server is running and receives the request. The file exists, is
readable, and is a binary file of about 4 MB. The server receives the
request, but the param list is empty. Something is there inside $cgi
called FORMDATA, but it is not accessible, and neither is the uploaded
file. Any ideas? Regular www-encoded requests work fine.
The server looks like this:
my $cgi = new CGI;
# upload is the same as param
my $file = $cgi->upload('upload');
binmode($file);
my $fn = $cgi->param('fn');
$fn = File::Spec->catfile($Dir,$fn);
# This returns a filehandle
my $fh = OpenForWriting({file=>$fn,nobackup=>1});
binmode($fh);
my ($bytesread,$buffer);
while ($bytesread=read($file,$buffer,1024))
{ print $fh $buffer; }
CloseForWriting($fh);
Any other comments also welcome.
Yehuda
------------------------------
Date: 29 Jan 2004 09:04:59 -0800
From: jon@actcom.co.il (Yehuda Berlinger)
Subject: Re: LWP, CGI and file uploads under windows
Message-Id: <75e50dfd.0401290904.772eb509@posting.google.com>
jon@actcom.co.il (Yehuda Berlinger) wrote in message news:<75e50dfd.0401280744.4177bf3b@posting.google.com>...
> Hi. I am using the latest activestate perl, and I am trying to create
> a client and a server that will do file uploading via http.
>
> My client looks like:
>
> my $UA = LWP::UserAgent->new;
> my %fields = (fn => $fn, upload => [$upload]);
> my $res = $UA->post($URL_Server, \%fields, 'Content_Type' =>
> 'form_data');
Someone found the answer for me in another group. The above should
change 'form_data' to 'multipart/form-data' . (note the '-' also,
instead of the '_')
Yehuda Berlinger
------------------------------
Date: Tue, 20 Jan 2004 02:33:40 GMT
From: R.Mariotti(at)FinancialDataCorp.com (Bob Mariotti)
Subject: LWP::Request technical question?
Message-Id: <400c91af.29524497@news.cshore.com>
Perlites:
I have a perl program that uses LWP to access an https site. It was
working but a few weeks ago stopped. The remote server is complaining
that my request format is incorrect.
My code section looks like this:
$REQ= POST 'https://www.blah.com',
[ NETCONNECT_TRANSACTION => $DATA ];
$REQ->authorization_basic($USERID,$PASSWD);
$RSP=$UA->request($REQ);
When I print the $REQ as_string it shows the following:
POST https://www.blah.com/netconnect2_0Demo/servlets/NetConnectServlet
Authorization: Basic ZmluZGUGF0Y286SjVsMjEwbW4XYZO=
Content-Length: 2496
Content-Type: application/x-www-form-urlencoded
NETCONNECT_TRANSACTION=%3C%3Fxml+version%3D%221.0%22+encoding%3D%22UTF-8
%22%3F%3E%3CNetConnectRequest+xmlns%3D%22http%3A%2F%2Fwww.blah.com%2FNetConnect%22xmlns%3A
... etc ...
The webmaster at the remote server is stating that the reason for the
failure is that I DO NOT HAVE an '&' preceeding the
"NETCONNECT_TRANSACTION=" in my request.
So, my question is:
If the request is a POST then is the Ampersand required or automatic?
If it does belong there then what can be done to assure that the
request format is correct (i.e.: proper url_encoding)?
Please advise anyone???
Thanks
B
------------------------------
Date: Sun, 25 Jan 2004 18:42:02 GMT
From: p cooper <pdconetwofour_numbers_@yahoo.co.uk>
Subject: Mail::Sendmail cant send to yahoo.co.uk addresses
Message-Id: <_5UQb.9374$Kq3.101529645@news-text.cableinet.net>
I had soem problems using exim to send emails from my Cable modem dyndns.org
server
- various error messages relating to unknoqn server
the exim config file had my primary host as home network (the 192.168...LAN
name)
this was fixed by putting my dyndns.org name into exim as the 1ry hostname -
except it wouldnt appear in my @yahoo.co.uk inbox.
same thing happens when i try using Mail::Sendmail
#!/usr/bin/perl
use Mail::Sendmail;
my @to_emails=qw(xxx@isp.com me@isp2.org.uk xxx@yahoo.co.uk);
foreach my $to(@to_emails)
{ %mail = ( To =>$to,
From => 'me@isp.com',
Message => "This is a very short message"
);
sendmail(%mail) or die $Mail::Sendmail::error;
what it is it with yahoo.co.uk.
Are there any other ISP that will not accept mail from Mail::SendMail ?
------------------------------
Date: Sun, 25 Jan 2004 19:40:41 GMT
From: p cooper <pdconetwofour_numbers_@yahoo.co.uk>
Subject: Re: Mail::Sendmail cant send to yahoo.co.uk addresses
Message-Id: <ZYUQb.9435$PS3.101977353@news-text.cableinet.net>
but it can - Yahoo automatically files it in 'bulk'
I wonder why??
------------------------------
Date: Mon, 26 Jan 2004 18:46:01 +0000 (UTC)
From: efflandt@xnet.com (David Efflandt)
Subject: Re: Mail::Sendmail cant send to yahoo.co.uk addresses
Message-Id: <slrnc1ao39.qlm.efflandt@typhoon.xnet.com>
On Sun, 25 Jan 2004, p cooper <pdconetwofour_numbers_@yahoo.co.uk> wrote:
> but it can - Yahoo automatically files it in 'bulk'
>
> I wonder why??
Your IP resolves to a name that contains "cable", and may be on a dynamic
cable list (source of much spam and worms). Nothing to do with Perl or
Perl modules specifically.
That is less likely to happen if you configure your smtp server (or
scripts) to relay through your ISP's outgoing smtp server.
--
David Efflandt - All spam ignored http://www.de-srv.com/
------------------------------
Date: Mon, 26 Jan 2004 11:23:06 -0500
From: John Blevin <blevin@lucent.com>
Subject: Memory allocation
Message-Id: <40153EEA.823DB9D7@lucent.com>
Let's say my program has built up a fairly large "hash of hashes"
data structure, rooted at %my_hash. If I then say
%my_hash = ();
this wipes out the structure, but what happens to the
memory that had been allocated? Is it freed, or is
it still allocated?
Thanks!
- John
------------------------------
Date: 26 Jan 2004 16:32:08 GMT
From: "Tassilo v. Parseval" <tassilo.parseval@rwth-aachen.de>
Subject: Re: Memory allocation
Message-Id: <bv3fe8$hic$1@nets3.rz.RWTH-Aachen.DE>
Also sprach John Blevin:
> Let's say my program has built up a fairly large "hash of hashes"
> data structure, rooted at %my_hash. If I then say
>
> %my_hash = ();
>
> this wipes out the structure, but what happens to the
> memory that had been allocated? Is it freed, or is
> it still allocated?
It's freed (but not necessarily handed back to the operating system).
Most often the memory will be re-used by perl whenever it would need to
allocate new memory.
Tassilo
--
$_=q#",}])!JAPH!qq(tsuJ[{@"tnirp}3..0}_$;//::niam/s~=)]3[))_$-3(rellac(=_$({
pam{rekcahbus})(rekcah{lrePbus})(lreP{rehtonabus})!JAPH!qq(rehtona{tsuJbus#;
$_=reverse,s+(?<=sub).+q#q!'"qq.\t$&."'!#+sexisexiixesixeseg;y~\n~~dddd;eval
------------------------------
Date: Mon, 26 Jan 2004 12:14:45 -0500
From: John Blevin <blevin@lucent.com>
To: tassilo.parseval@post.rwth-aachen.de
Subject: Re: Memory allocation
Message-Id: <40154B05.8CA9F7E4@lucent.com>
"Tassilo v. Parseval" wrote:
>
> Also sprach John Blevin:
>
> > Let's say my program has built up a fairly large "hash of hashes"
> > data structure, rooted at %my_hash. If I then say
> >
> > %my_hash = ();
> >
> > this wipes out the structure, but what happens to the
> > memory that had been allocated? Is it freed, or is
> > it still allocated?
>
> It's freed (but not necessarily handed back to the operating system).
> Most often the memory will be re-used by perl whenever it would need to
> allocate new memory.
OK, thanks. Is there a way to force it to hand the memory back to
the operating system? (My program is using huge amounts of memory,
sometimes running out of memory...)
- John
>
> Tassilo
> --
> $_=q#",}])!JAPH!qq(tsuJ[{@"tnirp}3..0}_$;//::niam/s~=)]3[))_$-3(rellac(=_$({
> pam{rekcahbus})(rekcah{lrePbus})(lreP{rehtonabus})!JAPH!qq(rehtona{tsuJbus#;
> $_=reverse,s+(?<=sub).+q#q!'"qq.\t$&."'!#+sexisexiixesixeseg;y~\n~~dddd;eval
------------------------------
Date: Mon, 26 Jan 2004 17:22:08 +0000 (UTC)
From: Ben Morrow <usenet@morrow.me.uk>
Subject: Re: Memory allocation
Message-Id: <bv3ic0$7v3$2@wisteria.csv.warwick.ac.uk>
John Blevin <blevin@lucent.com> wrote:
> OK, thanks. Is there a way to force it to hand the memory back to
> the operating system? (My program is using huge amounts of memory,
> sometimes running out of memory...)
No. This is not a perl question: on most OSen, programs cannot give
memory back to the OS. It would not help, in any case: perl will reuse
the memory it has freed before allocating any more.
Ben
--
"If a book is worth reading when you are six, * ben@morrow.me.uk
it is worth reading when you are sixty." - C.S.Lewis
------------------------------
Date: Mon, 26 Jan 2004 12:25:39 -0500
From: Chris Mattern <syscjm@gwu.edu>
Subject: Re: Memory allocation
Message-Id: <40154D93.10003@gwu.edu>
John Blevin wrote:
> "Tassilo v. Parseval" wrote:
>
>>Also sprach John Blevin:
>>
>>
>>>Let's say my program has built up a fairly large "hash of hashes"
>>>data structure, rooted at %my_hash. If I then say
>>>
>>>%my_hash = ();
>>>
>>>this wipes out the structure, but what happens to the
>>>memory that had been allocated? Is it freed, or is
>>>it still allocated?
>>
>>It's freed (but not necessarily handed back to the operating system).
>>Most often the memory will be re-used by perl whenever it would need to
>>allocate new memory.
>
>
> OK, thanks. Is there a way to force it to hand the memory back to
> the operating system?
Generally not. Usually memory allocated by a process isn't handed
back to the system until the process ends.
> (My program is using huge amounts of memory,
> sometimes running out of memory...)
One technique that sees use sometimes is to fork() a new copy of
the process and then have the parent terminate. The new copy
only allocates enough space to hold what it's currently using,
and the old copy surrenders all its memory when it terminates.
Of course, you'll need to have enough free memory to do the
fork()...
Chris Mattenr
------------------------------
Date: 26 Jan 2004 17:41:44 GMT
From: roberson@ibd.nrc-cnrc.gc.ca (Walter Roberson)
Subject: Re: Memory allocation
Message-Id: <bv3jgo$k5a$1@canopus.cc.umanitoba.ca>
In article <bv3ic0$7v3$2@wisteria.csv.warwick.ac.uk>,
Ben Morrow <usenet@morrow.me.uk> wrote:
:John Blevin <blevin@lucent.com> wrote:
:> OK, thanks. Is there a way to force it to hand the memory back to
:> the operating system? (My program is using huge amounts of memory,
:> sometimes running out of memory...)
:No. This is not a perl question: on most OSen, programs cannot give
:memory back to the OS.
Unix systems -can- give back memory, but if the memory was allocated
by the standard memory allocator, it's too hard to know whether the
memory is still in use or not. There are alternative memory allocation
libraries that are available to make it easier to give back memory.
In other words, for Unix, it isn't that it isn't possible to give back
memory at all, it's that the program has to be specially constructed in
order to give back memory without crashing. And most of the alternative
allocators that I've heard of don't even try using the official
deallocation procedure sbreak(): they instead use mmap() to allocate in
a different pool and manage that pool.
--
Scintillate, scintillate, globule vivific
Fain would I fathom thy nature specific.
Loftily poised on ether capacious
Strongly resembling a gem carbonaceous. -- Anon
------------------------------
Date: Thu, 22 Jan 2004 11:21:07 +1100
From: Sisyphus <kalinaubears@iinet.net.au>
Subject: Re: Memory Mgmt Using C and Perl
Message-Id: <400f1861$0$1731$5a62ac22@freenews.iinet.net.au>
Sisyphus wrote:
> Mark Shelor wrote:
>> So, my question: is it generally required (or recommended) to avoid
>> using malloc's within the C code of Perl extensions, and to instead
>> use Perl's memory management functions (e.g. New) within the XS code?
>> Doing this would cost me quite a bit of effort, so I'm reluctant to go
>> ahead with this approach, especially if it turns out not to have been
>> the problem in the first place.
>>
>
> I believe this is recommended, but I don't think it's the cause of the
> problem - though I hasten to add that I'm not an expert on such matters
> and I've yet to investigate the problem thoroughly.
>
Just looking a little further and I find those same 'Free to wrong pool
...' errors occur on perl 5.8.0 (AS source 802) built using MSVC++6.0 on
win2k. Again there's no problem with perl 5.6.1.
I've verified that the 'shaclose()' call creates that error (no surprise
about that I guess) - which I think implies that it's 'free()' and/or
'memset()' that's causing the problem (since they're the only things
that 'shaclose()' does). I deleted the 'memset()' call from 'shaclose()'
in src/sha.c, and rebuilt the module - but 'shaclose()' still produces
the same error. Alternatively, if I leave the 'memset()' call in there
but remove the 'free()' call then I can get some of the tests to pass.
Other tests still fail with the same 'wrong pool' error .... I guess
they might be using something other than 'shaclose()', but I haven't
traced it back.
That's a pretty rough way of trying to eliminate the problem function -
and I don't know if that's a valid way of making that elimination. But
if it *is*, then it looks that, for whatever reason, you *do* need to
replace 'malloc()' and 'free()' with 'New()' and 'Safefree()'. I found a
couple of instances of 'calloc()' in your code, too. I've definitely
experienced weird errors with that in the past. ('Realloc()' is
perlapi's equivalent, I think.)
While you're at it you might want to replace 'memset()' with its perl
API equivalent. See the list in 'perldoc perlclib' - there might also be
some other functions you could rewrite with perl API equivalents (eg
'strlen()', which has also posed problems for me on win32 in the past).
Hth.
Cheers,
Rob
--
To reply by email u have to take out the u in kalinaubears.
------------------------------
Date: Wed, 21 Jan 2004 17:39:03 -0700
From: Mark Shelor <mshelor@comcast.removeme.net>
Subject: Re: Memory Mgmt Using C and Perl
Message-Id: <mLednd85h742hpLdRVn-vg@comcast.com>
Sisyphus wrote:
> What I have found is that the module (version 4.2.0) builds and tests
> fine on my MinGW/dmake-built perl 5.6.1 on Win2k (built from ActiveState
> build 626 source).
>
> However, on my MinGW/dmake-built perl 5.8.0 on Win2k (built from
> ActiveState build 802 source), the build process fails with the
> following error:
> src\sha.c:730: called object is not a function
> .
> .
> src\sha.c:751: called object is not a function
Thanks very much, Rob, for testing this out on different configurations.
The above-cited errors apparently result from the compiler's not
recognizing the pre-defined "stdin" file handle. It's supposed to be
declared in <stdio.h>, so these errors are odd indeed. I would suspect
that the MinGW definition of "stdin" is the source of the problem.
> There's a dozen or so warnings as well, but I think those are the fatal
> errors. With perl 5.6.1, instead of getting fatal errors at those lines
> I get the warnings:
> src\sha.c:730: warning: assignment from incompatible pointer type
> .
> .
> src\sha.c:751: warning: comparison of distinct pointer types lacks a cast
Same thing again, i.e., I suspect a problem with the compiler's
<stdio.h> include file regarding the definition of "stdin".
> Next I built the module on my perl 5.8.2 (built using Visual Studio
> .NET, aka MSVC++ 7, built from ActiveState build 807 source) on the same
> Win2k.
> It builds fine, but all of the tests except the first produce the 'Free
> to wrong pool 15d2bb0 not C0100 t\blah-blah.t line x' error.
Aha! This is the same problem that the user got. I suspected the cause
might be that the C library "free()" function is in conflict with the
Perl memory manager.
> Afaik there's no memory management conflict here.
That could be, but "free to wrong pool" may indicate that C's "free()"
is attempting to de-allocate memory to an area that Perl's memory
manager doesn't know about.
> I'll have some time to look more closely at this tomorrow night if need
> be. All the better if, in the meantime, someone can present some ideas
> that might save me some time and brainstrain :-)
>
> (No problems on mandrake-9.1, perl 5.8.0, btw.)
Rob, you've already been a HUGE help in narrowing down (and re-creating)
the problem. I really appreciate your looking into this.
One interesting thing: the CPAN tester for the MSWin32 platform used
ActiveState and VC++ and encountered no problems at all. His exact
configuration can be seen at:
http://www.nntp.perl.org/group/perl.cpan.testers/116990
Kind Regards, Mark
------------------------------
Date: Thu, 22 Jan 2004 11:51:06 +1100
From: Sisyphus <kalinaubears@iinet.net.au>
Subject: Re: Memory Mgmt Using C and Perl
Message-Id: <400f1f67$0$1757$5a62ac22@freenews.iinet.net.au>
Mark Shelor wrote:
>
> One interesting thing: the CPAN tester for the MSWin32 platform used
> ActiveState and VC++ and encountered no problems at all. His exact
> configuration can be seen at:
>
> http://www.nntp.perl.org/group/perl.cpan.testers/116990
>
Yes, that cpan tester was running perl 5.6.1. I've found no problems
with that build of perl. It only occurs with 5.8.*.
My original assertion that it's *not* a memory management is starting to
look a little flaky :-)
Cheers,
Rob
--
To reply by email u have to take out the u in kalinaubears.
------------------------------
Date: 22 Jan 2004 07:22:35 GMT
From: "Tassilo v. Parseval" <tassilo.parseval@rwth-aachen.de>
Subject: Re: Memory Mgmt Using C and Perl
Message-Id: <buntnr$6qh$1@nets3.rz.RWTH-Aachen.DE>
Also sprach Mark Shelor:
> Sisyphus wrote:
>> Afaik there's no memory management conflict here.
>
>
> That could be, but "free to wrong pool" may indicate that C's "free()"
> is attempting to de-allocate memory to an area that Perl's memory
> manager doesn't know about.
I suspect that it's an maybe issue with threads. The ActiveState perls
are usually threaded ones. Each thread has its own heap of memory. You
get this error when one threads allocates some memory and another thread
tries to free() it.
Also, this thread on the p5porters-list could be some help:
<http://www.nntp.perl.org/group/perl.perl5.porters/87622>
Tassilo
--
$_=q#",}])!JAPH!qq(tsuJ[{@"tnirp}3..0}_$;//::niam/s~=)]3[))_$-3(rellac(=_$({
pam{rekcahbus})(rekcah{lrePbus})(lreP{rehtonabus})!JAPH!qq(rehtona{tsuJbus#;
$_=reverse,s+(?<=sub).+q#q!'"qq.\t$&."'!#+sexisexiixesixeseg;y~\n~~dddd;eval
------------------------------
Date: Fri, 23 Jan 2004 04:49:20 +1100
From: Sisyphus <kalinaubears@iinet.net.au>
Subject: Re: Memory Mgmt Using C and Perl
Message-Id: <40100e0e$0$1730$5a62ac22@freenews.iinet.net.au>
Mark Shelor wrote:
>
> The above-cited errors apparently result from the compiler's not
> recognizing the pre-defined "stdin" file handle. It's supposed to be
> declared in <stdio.h>, so these errors are odd indeed. I would suspect
> that the MinGW definition of "stdin" is the source of the problem.
>
Those compiler warnings are not simply a mingw thing. The very same
lines produce warnings with the MSVC compilers, too.
In src/sha.c (following recomendations in 'perlclib' docs) I changed:
1) Every occurrence of 'FILE*' to 'PerlIO*'
2) Every occurrence of 'stdout' to 'PerlIO_stdout()'
3) Every occurrence of 'stdin' to 'PerlIO_stdin()'
4) Every occurrence of 'fprintf' to 'PerlIO_printf'
I now get only one compiler warning when building the module (as opposed
to the 11 warnings I used to get - some of which became fatal errors on
mingw-built perl 5.8). The one remaining warning is in relation to the
'fgets()' function at line 669 of src/sha.c. 'perldoc perlclib' has some
advice regarding this, too - but I'm a little unsure as to how to put
that advice into practice.
Anyway, with those changes in place, I've now managed to build the
module with my mingw-built perl 5.8.0. Of course, having achieved that,
I now find that it, too, suffers precisely the same problems wrt freeing
to "wrong pool" :-)
Cheers,
Rob
--
To reply by email u have to take out the u in kalinaubears.
------------------------------
Date: 6 Apr 2001 21:33:47 GMT (Last modified)
From: Perl-Users-Request@ruby.oce.orst.edu (Perl-Users-Digest Admin)
Subject: Digest Administrivia (Last modified: 6 Apr 01)
Message-Id: <null>
Administrivia:
#The Perl-Users Digest is a retransmission of the USENET newsgroup
#comp.lang.perl.misc. For subscription or unsubscription requests, send
#the single line:
#
# subscribe perl-users
#or:
# unsubscribe perl-users
#
#to almanac@ruby.oce.orst.edu.
NOTE: due to the current flood of worm email banging on ruby, the smtp
server on ruby has been shut off until further notice.
To submit articles to comp.lang.perl.announce, send your article to
clpa@perl.com.
#To request back copies (available for a week or so), send your request
#to almanac@ruby.oce.orst.edu with the command "send perl-users x.y",
#where x is the volume number and y is the issue number.
#For other requests pertaining to the digest, send mail to
#perl-users-request@ruby.oce.orst.edu. Do not waste your time or mine
#sending perl questions to the -request address, I don't have time to
#answer them even if I did know the answer.
------------------------------
End of Perl-Users Digest V10 Issue 6021
***************************************
