[32463] in Perl-Users-Digest
Perl-Users Digest, Issue: 3729 Volume: 11
daemon@ATHENA.MIT.EDU (Perl-Users Digest)
Wed Jul 4 00:09:18 2012
Date: Tue, 3 Jul 2012 21:09:03 -0700 (PDT)
From: Perl-Users Digest <Perl-Users-Request@ruby.OCE.ORST.EDU>
To: Perl-Users@ruby.OCE.ORST.EDU (Perl-Users Digest)
Perl-Users Digest Tue, 3 Jul 2012 Volume: 11 Number: 3729
Today's topics:
IPC::Open2 reporting error in sys.excepthook <justin.1207@purestblue.com>
Re: IPC::Open2 reporting error in sys.excepthook <ben@morrow.me.uk>
Re: IPC::Open2 reporting error in sys.excepthook <rweikusat@mssgmbh.com>
Re: IPC::Open2 reporting error in sys.excepthook <justin.1207@purestblue.com>
Re: IPC::Open2 reporting error in sys.excepthook <justin.1207@purestblue.com>
Re: IPC::Open2 reporting error in sys.excepthook <rweikusat@mssgmbh.com>
Re: less code best code (Seymour J.)
Re: Making CGI.pm forget <bernie@fantasyfarm.com>
Re: question concerning pipes and large strings (Seymour J.)
Re: question concerning pipes and large strings <m@rtij.nl.invlalid>
Digest Administrivia (Last modified: 6 Apr 01) (Perl-Users-Digest Admin)
----------------------------------------------------------------------
Date: Mon, 2 Jul 2012 14:53:06 +0100
From: Justin C <justin.1207@purestblue.com>
Subject: IPC::Open2 reporting error in sys.excepthook
Message-Id: <2nl8c9-8ui.ln1@zem.masonsmusic.co.uk>
I can't see what I'm doing wrong here. My program is doing what
I want it to do, using a module I wrote to interface with
Mailman, but I'm getting:
GLOB(0x2d86348)close failed in file object destructor:
Error in sys.excepthook:
Original exception was:
My program extracts recently added email addresses from a
database of prospects and adds them to Mailman mailing list
manager using the commandline Mailman tools:
package MyModules::Mailman;
use strict;
use Carp;
use IPC::Open2
sub add_subscribers {
my (undef, $mailinglist, $addresses) = @_;
my ($in, $out, $pid) = connect_to_list($mailinglist);
for my $address (@{ $addresses }) {
print {$in} $address, "\n";
print {$out} "Added: <$address>\n";
}
close $in;
close $out;
waitpid($pid, 0);
my $exit_status = $? >> 8;
print "Exit status: ", $exit_status, "/n" if $exit_status;
}
sub connect_to_list {
my $mailinglist = shift;
my $cmd = '/usr/lib/mailman/bin/add_members';
my @opts = ('--regular-members-file=-', '--welcome-msg=n', '--admin-notify=n');
my @args = ($cmd, @opts, $mailinglist);
my ($in, $out);
my $pid = open2($out, $in, @args);
return ($in, $out, $pid);
}
1;
=======
I believe the problem is within the module, and not the calling
program, which uses the module thusly:
use strict;
use warnings;
use MyModules;
my $results = extract_new_addresses(); # an arrayref to list of email addresses
my $return_val = MyModule::Mailman->add_subscribers('prospects', $results);
print "Return value: ", $return_val, "\n";
=======
I usually spot any howlers when I prepare these messages, but
nothing has jumped out at me. Any suggestions why I'm getting
the error would be gratefully received.
Justin.
--
Justin C, by the sea.
------------------------------
Date: Mon, 2 Jul 2012 16:47:58 +0100
From: Ben Morrow <ben@morrow.me.uk>
Subject: Re: IPC::Open2 reporting error in sys.excepthook
Message-Id: <ees8c9-nh9.ln1@anubis.morrow.me.uk>
Quoth Justin C <justin.1207@purestblue.com>:
>
> I can't see what I'm doing wrong here. My program is doing what
> I want it to do, using a module I wrote to interface with
> Mailman, but I'm getting:
>
> GLOB(0x2d86348)close failed in file object destructor:
> Error in sys.excepthook:
This is a Python error, which means it's coming from Mailman. I'm afraid
I don't know what it means.
> Original exception was:
>
> My program extracts recently added email addresses from a
> database of prospects and adds them to Mailman mailing list
> manager using the commandline Mailman tools:
>
> package MyModules::Mailman;
>
> use strict;
> use Carp;
> use IPC::Open2
>
> sub add_subscribers {
> my (undef, $mailinglist, $addresses) = @_;
>
> my ($in, $out, $pid) = connect_to_list($mailinglist);
>
> for my $address (@{ $addresses }) {
> print {$in} $address, "\n";
> print {$out} "Added: <$address>\n";
$out is for reading from Mailman. Why are you trying to write to it?
> }
> close $in;
> close $out;
> waitpid($pid, 0);
> my $exit_status = $? >> 8;
> print "Exit status: ", $exit_status, "/n" if $exit_status;
> }
> sub connect_to_list {
> my $mailinglist = shift;
> my $cmd = '/usr/lib/mailman/bin/add_members';
> my @opts = ('--regular-members-file=-', '--welcome-msg=n', '--admin-notify=n');
> my @args = ($cmd, @opts, $mailinglist);
>
> my ($in, $out);
> my $pid = open2($out, $in, @args);
>
> return ($in, $out, $pid);
> }
Ben
------------------------------
Date: Mon, 02 Jul 2012 18:57:49 +0100
From: Rainer Weikusat <rweikusat@mssgmbh.com>
Subject: Re: IPC::Open2 reporting error in sys.excepthook
Message-Id: <874npqkpuq.fsf@sapphire.mobileactivedefense.com>
Ben Morrow <ben@morrow.me.uk> writes:
> Quoth Justin C <justin.1207@purestblue.com>:
>>
>> I can't see what I'm doing wrong here. My program is doing what
>> I want it to do, using a module I wrote to interface with
>> Mailman, but I'm getting:
>>
>> GLOB(0x2d86348)close failed in file object destructor:
>> Error in sys.excepthook:
>
> This is a Python error, which means it's coming from Mailman. I'm afraid
> I don't know what it means.
According to http://bugs.python.org/issue4192, it means "don't close
your end of the pipe the client is supposed to write to before it has
terminated" (meaning, do
wait();
close(...);
not the other way round (explicit close can usually be omitted in
Perl which should solve the problem as well).
------------------------------
Date: Tue, 3 Jul 2012 13:06:56 +0100
From: Justin C <justin.1207@purestblue.com>
Subject: Re: IPC::Open2 reporting error in sys.excepthook
Message-Id: <0s3bc9-if4.ln1@zem.masonsmusic.co.uk>
On 2012-07-02, Rainer Weikusat <rweikusat@mssgmbh.com> wrote:
> Ben Morrow <ben@morrow.me.uk> writes:
>> Quoth Justin C <justin.1207@purestblue.com>:
>>>
>>> I can't see what I'm doing wrong here. My program is doing what
>>> I want it to do, using a module I wrote to interface with
>>> Mailman, but I'm getting:
>>>
>>> GLOB(0x2d86348)close failed in file object destructor:
>>> Error in sys.excepthook:
>>
>> This is a Python error, which means it's coming from Mailman. I'm afraid
>> I don't know what it means.
>
> According to http://bugs.python.org/issue4192, it means "don't close
> your end of the pipe the client is supposed to write to before it has
> terminated" (meaning, do
>
> wait();
> close(...);
>
> not the other way round (explicit close can usually be omitted in
> Perl which should solve the problem as well).
But if I don't 'close' my program doesn't end, the Mailman part
(IPC::Open2) sits there waiting for further input, but there
isn't any.
A little experimentation, it is only my closing the $out that
causes the problem so I've removed it and I'm leaving it to perl
to sort out.
Problem resolved. Thanks for the replies.
Justin.
--
Justin C, by the sea.
------------------------------
Date: Tue, 3 Jul 2012 12:54:37 +0100
From: Justin C <justin.1207@purestblue.com>
Subject: Re: IPC::Open2 reporting error in sys.excepthook
Message-Id: <t43bc9-if4.ln1@zem.masonsmusic.co.uk>
On 2012-07-02, Ben Morrow <ben@morrow.me.uk> wrote:
>
> Quoth Justin C <justin.1207@purestblue.com>:
>>
>> I can't see what I'm doing wrong here. My program is doing what
>> I want it to do, using a module I wrote to interface with
>> Mailman, but I'm getting:
>>
>> GLOB(0x2d86348)close failed in file object destructor:
>> Error in sys.excepthook:
>
> This is a Python error, which means it's coming from Mailman. I'm afraid
> I don't know what it means.
Thank you for the clue, I'll go and start looking elsewhere.
[snip]
>> print {$out} "Added: <$address>\n";
>
> $out is for reading from Mailman. Why are you trying to write to it?
I have no idea! The line is to tell me what I've done; why I'm trying
to print it there I have no idea.
Justin.
--
Justin C, by the sea.
------------------------------
Date: Tue, 03 Jul 2012 13:46:11 +0100
From: Rainer Weikusat <rweikusat@mssgmbh.com>
Subject: Re: IPC::Open2 reporting error in sys.excepthook
Message-Id: <87a9zhhv1o.fsf@sapphire.mobileactivedefense.com>
Justin C <justin.1207@purestblue.com> writes:
> On 2012-07-02, Rainer Weikusat <rweikusat@mssgmbh.com> wrote:
>> Ben Morrow <ben@morrow.me.uk> writes:
>>> Quoth Justin C <justin.1207@purestblue.com>:
>>>>
>>>> I can't see what I'm doing wrong here. My program is doing what
>>>> I want it to do, using a module I wrote to interface with
>>>> Mailman, but I'm getting:
>>>>
>>>> GLOB(0x2d86348)close failed in file object destructor:
>>>> Error in sys.excepthook:
>>>
>>> This is a Python error, which means it's coming from Mailman. I'm afraid
>>> I don't know what it means.
>>
>> According to http://bugs.python.org/issue4192, it means "don't close
>> your end of the pipe the client is supposed to write to before it has
>> terminated" (meaning, do
>>
>> wait();
>> close(...);
>>
>> not the other way round (explicit close can usually be omitted in
>> Perl which should solve the problem as well).
>
> But if I don't 'close' my program doesn't end, the Mailman part
> (IPC::Open2) sits there waiting for further input, but there
> isn't any.
>
> A little experimentation, it is only my closing the $out that
> causes the problem
That's the exact same thing I wrote ('don't close your end of the pipe
the client is supposed to write to before it has terminated') and also
stated in the text the link points to,
This happens because when flooder.py terminates, its stdout will be
closed, but another pipe end in receirver.py process is already closed, so
Python\sysmodule.c(1098): _check_and_flush (FILE *stream)
In this function, fflush() fails.
Most likely (I haven't tested this), the reason is that there's some
buffered output and when fflush tries to write that to 'a broken
pipe', the results is either a SIGPIPE signal or an EPIPE
error. Explicit close can usually be omitted in Perl, namely, when the
close is just being done because someone felt like being anal about
this for no particular reason, as in your code, and simply not closing
this filehandle for no purpose will solve the problem.
------------------------------
Date: Sun, 01 Jul 2012 12:09:27 -0400
From: Shmuel (Seymour J.) Metz <spamtrap@library.lspace.org.invalid>
Subject: Re: less code best code
Message-Id: <4ff07637$1$fuzhry+tra$mr2ice@news.patriot.net>
In <4varu7toc1nq24ocj4qugh329u5150vhco@4ax.com>, on 06/29/2012
at 06:31 AM, J³rgen Exner <jurgenex@hotmail.com> said:
>If you are talking about complex data structures then any
>programming language (well, maybe except assembler) provides
>means to construct those.
Actually, assembler does better than some other languages.
--
Shmuel (Seymour J.) Metz, SysProg and JOAT <http://patriot.net/~shmuel>
Unsolicited bulk E-mail subject to legal action. I reserve the
right to publicly post or ridicule any abusive E-mail. Reply to
domain Patriot dot net user shmuel+news to contact me. Do not
reply to spamtrap@library.lspace.org
------------------------------
Date: Mon, 02 Jul 2012 20:46:38 -0400
From: Bernie Cosell <bernie@fantasyfarm.com>
Subject: Re: Making CGI.pm forget
Message-Id: <c2g4v7d323m89ig1gbevou1korom3k93u5@library.airnews.net>
Xho Jingleheimerschmidt <xhoster@gmail.com> wrote:
} On 06/19/2012 08:51 AM, Bernie Cosell wrote:
} > One aspect of CGI.pm that I've never liked is that it tries to 'remember'
} > form variables from one page to the next --- and worse makes it hard to
} > change a value, preferring to ignore your attempts to change the value and
} > keep the old one unless you go to some bother.
}
} Is reading the documentation "some bother"?
}
} -nosticky
Well, to iterate:
1) I was already *using* nosticky, and my params still didn't get
unstuck, and
2) I also had done "new CGI("")" which the docs say should get me an
empty CGI object and *STILL* my params didn't get unstuck.
The only thing I found that works, and I'll admit it isn't much of a
bother, is that I needed to do a "Delete_all" after the "CGI("")" to
*really* get me an empty CGI object with no memory.
I do *NOT* understand why *either* (1) or (2) above weren't sufficient [as
I read the docs _either_ should have been].
/Bernie\
--
Bernie Cosell Fantasy Farm Fibers
bernie@fantasyfarm.com Pearisburg, VA
--> Too many people, too few sheep <--
------------------------------
Date: Sun, 01 Jul 2012 21:37:32 -0400
From: Shmuel (Seymour J.) Metz <spamtrap@library.lspace.org.invalid>
Subject: Re: question concerning pipes and large strings
Message-Id: <4ff0fb5c$4$fuzhry+tra$mr2ice@news.patriot.net>
In <87obob86q6.fsf@sapphire.mobileactivedefense.com>, on 06/22/2012
at 08:52 PM, Rainer Weikusat <rweikusat@mssgmbh.com> said:
>The state of certain sciences such as 'physics' or 'astronomy' has
>been 'good enough' for any even remotely practical purpose for
>something like a century or so.
Nonsense. We routinely use devices that would have been impossible
without a better understanding of Physics.
--
Shmuel (Seymour J.) Metz, SysProg and JOAT <http://patriot.net/~shmuel>
Unsolicited bulk E-mail subject to legal action. I reserve the
right to publicly post or ridicule any abusive E-mail. Reply to
domain Patriot dot net user shmuel+news to contact me. Do not
reply to spamtrap@library.lspace.org
------------------------------
Date: Mon, 2 Jul 2012 23:13:45 +0200
From: Martijn Lievaart <m@rtij.nl.invlalid>
Subject: Re: question concerning pipes and large strings
Message-Id: <9hf9c9-1lf.ln1@news.rtij.nl>
On Fri, 22 Jun 2012 18:29:07 +0100, Rainer Weikusat wrote:
> Martijn Lievaart <m@rtij.nl.invlalid> writes:
>> On Thu, 21 Jun 2012 23:11:55 +0100, Rainer Weikusat wrote:
>>
>>> But this is nevertheless sort-of a ghost discussion: Something
>>> 'complete' which has been written in C will doubtlessly outperform the
>>> pipeline easily.
>>
>> What is more valuable, your time or computer time?
>
> Depending on how you look at it, my time is either infinitely more
> valuable than computer time (every second of my finite life which has
> passed is gone forever and cannot be replaced) or infinitely less
> valuable (People will pay in order to use computer time to solve
> problems. Nobody will pay me for my lifetime just because it is
> theoretically valuable to me and if I just starved to death tomorrow,
> the world-at-a-large wouldn't take note of this loss).
>
> Seems you question doesn't really make sense ...
I think you just answered the question... :-)
But the context is of course one of efficiency. To put it to extremes,
should you put in a few hours work to shave of a few milliseconds
runtime? If you work in realtiome systems the answer might be a definate
yes. Otherwise it more often than not should be no. Unless your hobby is
making programs run as efficient as possible.
Remember the three laws of optimization: Don't, don't, don't. Or profile,
profile, profile.
Cases in point:
I was asked to speed up a ldap synchronization program. After looking it
over, I rewrote the algorithm from O(3) to O(2), resulting in a 300%
speedup. The result was still not fast enough, but lacking a clear point
of attack (programmers are often very bad at estimating what is or isn't
efficient, I was not going to fall for that trap), I profiled the
program. I found that 40% of the time, the program was converting strings
from utf8 to latin1 and vise versa. Caching the conversion resulted in
another 20% speedup. On a whim, as I know that graphics can take time and
I knew the profiler could not profile that graphics part, I changed the
progress bar to update only 1 in 16 steps. This resulted in another 20%
speedup, at which point the program was fast enough for our datasets
(which we knew would not grow).
A program I wrote was rather efficient at it's task, but it did affect
the efficiency of the database involved. Other tasks did not yet suffer
enough that users noticed, but monitoring indicated a much higher
database load. After talking it over with a colleague, he came up with a
much better algorithm. I found I could also cache some values at program
start. The resulting speedup was huge, from a minute runtime down to less
than a second. As this was a daemon, the runtime for a single pass was
not important at all, however, the database load was down to unnoticable
for monitoring.
I have to create a report every month. It takes some 2 minutes to run. I
can make it much more efficient, but it's just not worth my time.
M4
------------------------------
Date: 6 Apr 2001 21:33:47 GMT (Last modified)
From: Perl-Users-Request@ruby.oce.orst.edu (Perl-Users-Digest Admin)
Subject: Digest Administrivia (Last modified: 6 Apr 01)
Message-Id: <null>
Administrivia:
To submit articles to comp.lang.perl.announce, send your article to
clpa@perl.com.
Back issues are available via anonymous ftp from
ftp://cil-www.oce.orst.edu/pub/perl/old-digests.
#For other requests pertaining to the digest, send mail to
#perl-users-request@ruby.oce.orst.edu. Do not waste your time or mine
#sending perl questions to the -request address, I don't have time to
#answer them even if I did know the answer.
------------------------------
End of Perl-Users Digest V11 Issue 3729
***************************************
