[31775] in Perl-Users-Digest
Perl-Users Digest, Issue: 3038 Volume: 11
daemon@ATHENA.MIT.EDU (Perl-Users Digest)
Wed Jul 21 14:09:22 2010
Date: Wed, 21 Jul 2010 11:09:07 -0700 (PDT)
From: Perl-Users Digest <Perl-Users-Request@ruby.OCE.ORST.EDU>
To: Perl-Users@ruby.OCE.ORST.EDU (Perl-Users Digest)
Perl-Users Digest Wed, 21 Jul 2010 Volume: 11 Number: 3038
Today's topics:
data structures/query question <cartercc@gmail.com>
Re: FAQ 9.14 How do I make sure users can't enter value <justin.1007@purestblue.com>
Need help/advice to improve script <sopan.shewale@gmail.com>
Re: Need help/advice to improve script <m@rtij.nl.invlalid>
Re: Need help/advice to improve script <jurgenex@hotmail.com>
Re: Need help/advice to improve script <hjp-usenet2@hjp.at>
Re: Need help/advice to improve script <hjp-usenet2@hjp.at>
Re: Need help/advice to improve script <rvtol+usenet@xs4all.nl>
Re: Need help/advice to improve script <rvtol+usenet@xs4all.nl>
Stress testing process start <nospam-abuse@ilyaz.org>
Re: Stress testing process start <email@invalid.net>
Re: Stress testing process start <jimsgibson@gmail.com>
Digest Administrivia (Last modified: 6 Apr 01) (Perl-Users-Digest Admin)
----------------------------------------------------------------------
Date: Wed, 21 Jul 2010 07:33:52 -0700 (PDT)
From: ccc31807 <cartercc@gmail.com>
Subject: data structures/query question
Message-Id: <235f65df-d269-4c15-9ca0-f5a85e443c8b@g35g2000yqa.googlegroups.com>
I've solved this twice, but still am not happy with my solution. I can
post working code that meets the requirements, and I will do so. But
first, I'd like to listen to the wisdom of this august body.
I have reproduced a sample of the data file below. The top row
consists of position codes. The left column consists of function
codes. The cells consists of the number of people who have the
position identified in the top row that perform the function
identified in the left column.
The requirement is: return a list of all position codes where at least
one person with that position performs all the functions in the query.
A query can look like this:
----------------query (in.txt)----------------
53ProcStuR 15DataEntr 54ProspStu 02AcadAdvUg 48PrtcipOr 68Trnscrpt
69TrnfrCrd 27GrdProcA
Here is a sample run:
------------output-----------------
jobs1>perl crunch_data_2.plx
Enter the name of your data file (comma separated values): Consol.csv
Enter the name of the functions file (space separated values): in.txt
The functions you specified are: 53ProcStuR 15DataEntr 54ProspStu
02AcadAdvUg 48PrtcipOr 68Trnscrpt 69Trn frCrd 27GrdProcA
The positions that matched: fID 1BDD 1ADSS 1SC 1E 1SD 1ASD 1AA 1SSS
1ASC 1DSS
------------end output---------------------
As I said, I can and will post working code that meets the
requirements, but I don't want to poison the well before hand. I would
appreciate very much your best idea of how to do this.
Here is a sample of the data file -- but only the first few columns
and the first few rows because the file is much larger than this.
---------------data file (Consol.csv)-------------------------
fID,1AA,1AAD,1AD,1ADA,1ADB,1ADBFA,1ADBO,1ADFG,1ADFS,1ADMAS,1ADOF,1ADSS,
1AE,1AF,1AS,1AS2,1ASC,1ASD,1ASDSS,1ASST,1ATRD,1ATTD,1BD,1BDD,1BDR,1BDS,
1BM,1CA,1CLK1,1CLK3
15DataEntr,31,1,1,1,1,1,1,,,1,1,5,1,1,1,1,4,6,1,1,1,1,1,3,1,1,,1,1,1
54ProspStu,31,,1,1,,,,,,1,1,5,1,,,,4,5,1,1,,,1,6,1,,,,1,1
03ClericSup,23,,,1,1,1,1,,1,1,,3,1,1,1,1,2,4,,1,1,1,,1,1,1,,1,1,1
48PrtcipOr,27,,1,1,,1,,,1,1,1,4,1,1,,1,3,5,1,,1,,,4,,,,,,
53ProcStuR,36,,1,1,,,,,,1,1,3,1,1,,,4,3,,,,,1,2,1,,,,,
42MPlnRcrt,12,1,,,,,,1,,,,3,,,,,1,5,,1,1,,1,7,1,1,,,,1
52PrepStuF,22,,,1,1,,,,,1,1,2,,1,,1,3,4,,,,,,1,,,,,,
55RcvScanD,13,,,,,1,,,,1,,,,1,,1,1,2,,1,1,1,,,,,,1,1,1
24FileDevM,15,,,1,1,1,,,,1,1,4,,1,1,1,2,3,1,1,,,,3,,,,1,1,1
56RecepDut,12,,,,,,,,,1,,,,1,,,1,2,,1,1,1,,,,,,1,1,1
65TestProc,18,,,1,,,,1,,,1,3,1,,,,4,2,,,1,,,2,,,,,,1
04AdmssProc,18,,,1,,,,,,1,1,3,1,1,,,3,3,,,,,,2,,,,,1,
02AcadAdvUg,30,,,1,,,,1,,,1,4,1,1,,,4,3,,,,,,2,,,,,,
27GrdProcA,23,1,,1,1,,1,,1,1,,6,,1,1,,2,4,1,1,,,,2,1,,,,,
09CommInvPR,8,1,1,,,,,1,,1,,2,,,,,2,4,1,1,,,1,7,1,1,,,,1
17RetenStr,7,1,1,1,,,,,,1,,4,,,,,2,6,1,,,,1,6,1,1,,,,
16RecrtStr,7,1,,,,,,1,,1,,1,1,,,,1,5,,,,,1,7,1,1,,,,
18DocImagi,9,,,1,,,,,,,1,2,,1,,1,2,3,,1,,,,,,,,1,1,
68Trnscrpt,27,,,,,,,,,,1,4,1,,,,2,3,,,,,1,1,1,,,,,
57RecrdsMa,8,,,1,1,1,,,,1,1,4,,1,,1,2,3,1,,,,,1,,,,,,
01AcadAdvGr,23,,,1,,,,1,,1,,6,,,,,3,3,,,,,,1,,,,,,
07ClassCoor,5,,1,1,,,,,,,1,3,,,,,1,6,,1,,,,,,,,,,1
58ReprtPre,9,,,,1,1,1,,,1,,1,,1,,1,1,2,1,,,1,,2,,,,1,,
28GrdProce,17,,1,1,,1,,,,,1,6,,,,1,3,3,1,,,,,,,,,,,
45SupEqpPu,1,,,,1,1,1,,1,,1,,,1,1,1,2,2,,,,1,,,,1,,,,
49EndTrmPr,20,,1,1,,,,,,,1,4,,,,,3,3,,,,,,,,,,,,
20EnrLeadM,3,1,,,,,,1,,1,,,,,,,1,3,1,,,,1,6,1,1,,,,
33MailRoom,1,,,,,1,,,1,,,,,1,1,1,1,2,,1,,1,,,,1,,1,,1
51ClassSch,4,1,,1,,,,,,,,2,1,,,,2,2,,,1,,,,,,,,,
29ImplPoli,1,1,1,,,1,2,1,,1,1,5,,,,,1,3,1,,,,,2,,,,,,
35MktgBusR,4,1,,,,,,,,1,,1,,,,,1,2,,,,,,6,,1,,,,
69TrnfrCrd,26,,,,,,,,,,1,3,1,,,,2,1,,,,,,1,,,,,,
13StuSvcsT,7,,1,1,,1,,,,,1,8,1,,,,2,3,1,,,,,1,,,,,,
70TrblshtS,7,,,,1,1,,,1,1,1,1,,1,1,1,3,2,,,,,,1,,,,,,
06BldgMonSe,3,1,,,,1,,,,,1,,1,,,,1,4,,,,,,,,,2,,,1
10HRDocs,,1,,,1,1,2,,1,,1,3,,,,,1,1,1,,,1,,,,,,,,
47OvrseeSt,5,,1,1,,1,,1,,,1,4,1,,,,1,2,1,,,,,,,,,,,
11ComplRpt,2,1,,,1,1,1,,,1,,,,1,,1,,2,,,1,,,1,,,,1,,
62StaffSup,,1,,,,1,1,,1,,1,3,,,,,1,1,,,,,,2,,,,,,
------------------------------
Date: Wed, 21 Jul 2010 12:59:18 +0100
From: Justin C <justin.1007@purestblue.com>
Subject: Re: FAQ 9.14 How do I make sure users can't enter values into a form that cause my CGI script to do bad things?
Message-Id: <m14jh7-gav.ln1@zem.masonsmusic.co.uk>
On 2010-07-20, PerlFAQ Server <brian@theperlreview.com> wrote:
> This is an excerpt from the latest version perlfaq9.pod, which
> comes with the standard Perl distribution. These postings aim to
> reduce the number of repeated questions as well as allow the community
> to review and update the answers. The latest version of the complete
> perlfaq is at http://faq.perl.org .
>
> --------------------------------------------------------------------
>
> 9.14: How do I make sure users can't enter values into a form that cause my CGI script to do bad things?
>
> See the security references listed in the CGI Meta FAQ
>
> http://www.perl.org/CGI_MetaFAQ.html
That page redirects to:
<URL:http://brian-d-foy.cvs.sourceforge.net/viewvc/brian-d-foy/CGI_MetaFAQ/CGI_MetaFAQ.html>
and three out of the five links in the Security section fail to open.
Two 404s and one server that has gone away - this may be temporary.
Justin.
--
Justin C, by the sea.
------------------------------
Date: Wed, 21 Jul 2010 02:17:20 -0700 (PDT)
From: "sopan.shewale@gmail.com" <sopan.shewale@gmail.com>
Subject: Need help/advice to improve script
Message-Id: <d628c106-71f0-43ac-bd93-cbed68f6bd1f@w15g2000pro.googlegroups.com>
Hello,
I have Apache htpasswd format password store. The fields are ":"
separated and has more fields than standard format. I have script to
read the fields and load into Hash.
I need advice/help to improve the script. The current one works-but
not really good script. The script goes as follow:
--------------------------------
#!/usr/bin/perl
use strict;
use warnings;
my $data = {};
for (<DATA>) {
my $line = $_;
if ( defined $line ) {
if ( $line =~ /^(.*?):(.*?):(.*?):(.*?):(.*?)(?::(.*))?$/
|| $line =~ /^(.*?):(.*?):(.*?):(.*?)(?::(.*))?$/
|| $line =~ /^(.*?):(.*?):(.*?)(?::(.*))?$/
|| $line =~ /^(.*?):(.*?)(?::(.*))?$/ )
{
$data->{$1}->{pass} = $2;
$data->{$1}->{emails} = $3 || '';
$data->{$1}->{flag} =
( ( defined $4 ) && ( $4 == 0 ) ) ? 0 : ( $4 || '' );
$data->{$1}->{pass_change} = $5 || '';
$data->{$1}->{flag_change} = $6 || '';
}
}
}
use Data::Dumper;
print Data::Dumper->Dump( [$data] );
__DATA__
AllPresent:hjliEO35kCgwI:all@example.com:1:23232:24324
LastMissing:CyL92g3OKi.jM:lastmissing@example.com:1:2323
FlagZero:CyL92g3OKi.jM:flagzero@example.com:0:23232
OnlyFlag:ZuqpLZ7AxHBvw:onlyflag@example.com:0
RestMissing:ZuqpLZ7AxHBvw:missing@example.com
-------------------------
Please note - only first two fields are must fields, rest may be
missing from lines. first is - username, second is encrypted
password, third is email, fourth is flag to force user to change
password(0-pass change is not required, 1-user must change pass),
fifth- the last time password changed epoch time, sixth- time of
change of flag (some times flag is changed by admins of the system).
thanks in advance
------------------------------
Date: Wed, 21 Jul 2010 14:27:15 +0200
From: Martijn Lievaart <m@rtij.nl.invlalid>
Subject: Re: Need help/advice to improve script
Message-Id: <3m5jh7-i09.ln1@news.rtij.nl>
On Wed, 21 Jul 2010 02:17:20 -0700, sopan.shewale@gmail.com wrote:
> Hello,
>
> I have Apache htpasswd format password store. The fields are ":"
> separated and has more fields than standard format. I have script to
> read the fields and load into Hash.
>
> I need advice/help to improve the script. The current one works-but not
> really good script. The script goes as follow:
> --------------------------------
> #!/usr/bin/perl
> use strict;
> use warnings;
>
> my $data = {};
Simpler to use a hash variable, if needed you can always take a reference
to that.
>
> for (<DATA>) {
While (<...>) has magic, use it.
>
> my $line = $_;
> if ( defined $line ) {
> if ( $line =~ /^(.*?):(.*?):(.*?):(.*?):(.*?)(?::(.*))?$/
> || $line =~ /^(.*?):(.*?):(.*?):(.*?)(?::(.*))?$/ || $line
> =~ /^(.*?):(.*?):(.*?)(?::(.*))?$/ || $line =~
> /^(.*?):(.*?)(?::(.*))?$/ )
This is why split() was invented
> {
> $data->{$1}->{pass} = $2;
> $data->{$1}->{emails} = $3 || '';
> $data->{$1}->{flag} =
> ( ( defined $4 ) && ( $4 == 0 ) ) ? 0 : ( $4 || '' );
> $data->{$1}->{pass_change} = $5 || '';
> $data->{$1}->{flag_change} = $6 || '';
> }
> }
>
> }
>
#!/usr/bin/perl
use strict;
use warnings;
my %data;
while (my $line = <DATA>) {
# chop of newline
chomp $line;
# split into fields
my ($user, $pass, $email, $flag, $passchange, $flagchange) =
split /:/, $line, 6;
# stuff into hash, using a hash slice
@{$data{$user}}{qw/pass email flag passchange flagchange/} =
($pass, $email, $flag, $passchange, $flagchange);
}
use Data::Dumper;
print Data::Dumper->Dump( [\%data] );
__DATA__
AllPresent:hjliEO35kCgwI:all@example.com:1:23232:24324
LastMissing:CyL92g3OKi.jM:lastmissing@example.com:1:2323
FlagZero:CyL92g3OKi.jM:flagzero@example.com:0:23232
OnlyFlag:ZuqpLZ7AxHBvw:onlyflag@example.com:0
RestMissing:ZuqpLZ7AxHBvw:missing@example.com
The line with he hash slice can also be written as:
$data{$user}{pass} = $pass;
$data{$user}{email} = $email;
... etc ...
And obviously, it can even be shortened to:
# split and stuff into hash, using a hash slice
@{$data{$user}}{qw/pass email flag passchange flagchange/} =
split /:/, $line, 6;
HTH,
M4
------------------------------
Date: Wed, 21 Jul 2010 05:31:27 -0700
From: Jürgen Exner <jurgenex@hotmail.com>
Subject: Re: Need help/advice to improve script
Message-Id: <mapd46d0bajm9bbe2fd49ffl44p0v7fpmu@4ax.com>
"sopan.shewale@gmail.com" <sopan.shewale@gmail.com> wrote:
>Hello,
>
>I have Apache htpasswd format password store. The fields are ":"
>separated and has more fields than standard format. I have script to
>read the fields and load into Hash.
>
>I need advice/help to improve the script. The current one works-but
>not really good script. The script goes as follow:
>--------------------------------
>#!/usr/bin/perl
>use strict;
>use warnings;
Good!
>my $data = {};
This is confusing. Is $data a reference to an anonymous hash? The length
of an empty anonymous hash? Are you using curly brackets instead of
regular quotes? Is this a typo and you meant %data instead?
Whatever it is, such a non-standard use is very confusing and also
totally unnecessary because Perl will initialize variables to 'emtpy'
anyway.
>for (<DATA>) {
This doesn't make much sense. for() creates the whole argument list
first. So either just read the whole file into a array or use while() to
process it line by line.
> my $line = $_;
> if ( defined $line ) {
The test is unnecessary. If the line was read from the file then the
variable is defined.
> if ( $line =~ /^(.*?):(.*?):(.*?):(.*?):(.*?)(?::(.*))?$/
> || $line =~ /^(.*?):(.*?):(.*?):(.*?)(?::(.*))?$/
> || $line =~ /^(.*?):(.*?):(.*?)(?::(.*))?$/
> || $line =~ /^(.*?):(.*?)(?::(.*))?$/ )
OMG! If your data is colon-separated then just split() it at the colon
and check how many elements the result array has.
@fields = split /:/, $line;
if (@fields > 2 and @fields < 8) {
or something similar.
jue
------------------------------
Date: Wed, 21 Jul 2010 15:04:36 +0200
From: "Peter J. Holzer" <hjp-usenet2@hjp.at>
Subject: Re: Need help/advice to improve script
Message-Id: <slrni4ds35.kdr.hjp-usenet2@hrunkner.hjp.at>
On 2010-07-21 09:17, sopan.shewale@gmail.com <sopan.shewale@gmail.com> wrote:
> Hello,
>
> I have Apache htpasswd format password store. The fields are ":"
> separated and has more fields than standard format. I have script to
> read the fields and load into Hash.
>
> I need advice/help to improve the script. The current one works-but
> not really good script. The script goes as follow:
> --------------------------------
> #!/usr/bin/perl
> use strict;
> use warnings;
>
> my $data = {};
>
> for (<DATA>) {
> my $line = $_;
Change these two lines to
while (my $line = <DATA>) {
> if ( defined $line ) {
$line is always defined here so remove this line.
> if ( $line =~ /^(.*?):(.*?):(.*?):(.*?):(.*?)(?::(.*))?$/
> || $line =~ /^(.*?):(.*?):(.*?):(.*?)(?::(.*))?$/
> || $line =~ /^(.*?):(.*?):(.*?)(?::(.*))?$/
> || $line =~ /^(.*?):(.*?)(?::(.*))?$/ )
> {
That looks extremely convoluted. I didn't even try to understand what
this does. Each line is a colon-separated list of fields, isn't it?
So replace it with:
chomp($line);
my ($username, $pass, $emails, $flag, $pass_change, $flag_change)
= split(/:/, $line);
next unless $pass;
and then use $username, $pass, etc. instead of $1, $2, etc.
> $data->{$1}->{pass} = $2;
> $data->{$1}->{emails} = $3 || '';
> $data->{$1}->{flag} =
> ( ( defined $4 ) && ( $4 == 0 ) ) ? 0 : ( $4 || '' );
This could be written somewhat more readable as:
$data->{$username}->{flag} = defined $flag ? $flag : '';
or if you have Perl 5.10, as
$data->{$username}->{flag} = $flag // '';
And since this is a flag it should probably only be used in a boolean
context, so it doesn't matter whether the value is 0, '' or undef, so
I would just write
$data->{$username}->{flag} = $flag;
instead.
> $data->{$1}->{pass_change} = $5 || '';
> $data->{$1}->{flag_change} = $6 || '';
And these are numeric values. 0 may be a valid value, so replacing it
with '' may be an error. Even if it isn't, I'd prefer undef for invalid
values over '', I'd just write
$data->{$username}->{pass_change} = $pass_change;
$data->{$username}->{flag_change} = $flag_change;
here, too.
Finally, the long list of assignments bothers me. So I'd rewrite that to
$data->{$username}
= {
pass => $pass,
emails => $emails || '',
flag => $flag,
pass_change => $pass_change,
flag_change => $flag_change,
};
and the duplication of fieldnames could also be avoided:
my @field_names = qw(pass emails flag pass_change flag_change);
my ($username, @field_values) = split(/:/, $line);
@{ $data->{$username} }{@field_names} = @field_values;
if we can live with an undefined 'emails' field.
> }
> }
>
> }
>
> use Data::Dumper;
> print Data::Dumper->Dump( [$data] );
>
> __DATA__
> AllPresent:hjliEO35kCgwI:all@example.com:1:23232:24324
> LastMissing:CyL92g3OKi.jM:lastmissing@example.com:1:2323
> FlagZero:CyL92g3OKi.jM:flagzero@example.com:0:23232
> OnlyFlag:ZuqpLZ7AxHBvw:onlyflag@example.com:0
> RestMissing:ZuqpLZ7AxHBvw:missing@example.com
>
So my final script looks like this:
#!/usr/bin/perl
use strict;
use warnings;
my $data = {};
while (my $line = <DATA>) {
chomp $line;
my @field_names = qw(pass emails flag pass_change flag_change);
my ($username, @field_values) = split(/:/, $line);
next unless $username;
@{ $data->{$username} }{@field_names} = @field_values;
}
use Data::Dumper;
print Data::Dumper->Dump( [$data] );
__DATA__
AllPresent:hjliEO35kCgwI:all@example.com:1:23232:24324
LastMissing:CyL92g3OKi.jM:lastmissing@example.com:1:2323
FlagZero:CyL92g3OKi.jM:flagzero@example.com:0:23232
OnlyFlag:ZuqpLZ7AxHBvw:onlyflag@example.com:0
RestMissing:ZuqpLZ7AxHBvw:missing@example.com
hp
------------------------------
Date: Wed, 21 Jul 2010 15:31:16 +0200
From: "Peter J. Holzer" <hjp-usenet2@hjp.at>
Subject: Re: Need help/advice to improve script
Message-Id: <slrni4dtl4.kdr.hjp-usenet2@hrunkner.hjp.at>
On 2010-07-21 12:31, JĂĽrgen Exner <jurgenex@hotmail.com> wrote:
> "sopan.shewale@gmail.com" <sopan.shewale@gmail.com> wrote:
>>my $data = {};
>
> This is confusing.
I don't think so. It's perfectly clear and normal Perl.
> Is $data a reference to an anonymous hash?
Yes.
> The length
> of an empty anonymous hash?
No.
> Are you using curly brackets instead of
> regular quotes?
No. Of course he could have meant to write
my $data = q{};
but there is no reason to assume that.
> Is this a typo and you meant %data instead?
Then he would also have had to use parentheses instead of the braces.
So this is unlikely.
> Whatever it is, such a non-standard use is very confusing and also
> totally unnecessary because Perl will initialize variables to 'emtpy'
> anyway.
But “empty” (you mean “undef”) isn't the same as a reference to an empty
hash. It makes no difference in this program (the anonymous hash is
autovivified at the first access), but an explicit initialization like
this:
* Serves as a reminder that this variable is intended to be used as a
hashref (and not as an arrayref, string, number, or whatever)
* May catch some usage errors
* May prevent autovivification at the wrong place. For example,
my $data; func($data);
is not the same as
my $data = {}; func($data);
if func() accesses $data->{something}.
So it's sometimes necessary, often useful and otherwise harmless.
hp
------------------------------
Date: Wed, 21 Jul 2010 19:54:36 +0200
From: "Dr.Ruud" <rvtol+usenet@xs4all.nl>
Subject: Re: Need help/advice to improve script
Message-Id: <4c47345d$0$22939$e4fe514c@news.xs4all.nl>
Peter J. Holzer wrote:
> But “empty” (you mean “undef”) isn't the same as a reference to an empty
> hash. It makes no difference in this program (the anonymous hash is
> autovivified at the first access), but an explicit initialization like
> this:
>
> * Serves as a reminder that this variable is intended to be used as a
> hashref (and not as an arrayref, string, number, or whatever)
> * May catch some usage errors
> * May prevent autovivification at the wrong place. For example,
> my $data; func($data);
> is not the same as
> my $data = {}; func($data);
> if func() accesses $data->{something}.
>
> So it's sometimes necessary, often useful and otherwise harmless.
I also, and pretty often, use it in this way:
my $dfbb = $data{ foo }{ bar }{ baz } ||= {};
--
Ruud
------------------------------
Date: Wed, 21 Jul 2010 19:58:41 +0200
From: "Dr.Ruud" <rvtol+usenet@xs4all.nl>
Subject: Re: Need help/advice to improve script
Message-Id: <4c473552$0$22941$e4fe514c@news.xs4all.nl>
Peter J. Holzer wrote:
> my ($username, $pass, $emails, $flag, $pass_change, $flag_change)
> = split(/:/, $line);
Variant:
my %line;
@line{qw/ pass emails flag pass_change flag_change /} =
split /:/, $line;
--
Ruud
------------------------------
Date: Wed, 21 Jul 2010 09:48:38 +0000 (UTC)
From: Ilya Zakharevich <nospam-abuse@ilyaz.org>
Subject: Stress testing process start
Message-Id: <slrni4dgjm.u4u.nospam-abuse@powdermilk.math.berkeley.edu>
Some time ago I got ideas that the process start logic of Perl may be
flawed (at least the part implemented by me :-[ ;-). So I spend some
time, and made a primitive script to stress-test Perl in the "tricky"
situations. See ilyaz.org/software/tmp/check_redir.pl
The best-hoped-for results are: having only <123//456> and <> as
possible output strings (ones after the colon in the output), maybe
interspersed with some warnings about closed filehandles. I tested on
3 architectures OS/2, BSD, and RedHat. I see much more than this.
The problems which I see are:
a) $! may become "Inappropriate ioctl for device" during
closing (some of) STDERR, STDOUT, STDIN.
b) I may see output <<456>>, which means that STDERR was NOT redirected.
c) Shell may be called in
open GRANDKID, q(|-), "$^X -wl - 2>&1" or die 12;
while Perl has enough whistles to do it without shell ( as I assume
it is doing with q(-|) ).
d) I see "Unknown error 171324724" as content of $!.
e) I see $? being -1, or 256 when things end in exit(0).
f) the grandchild may block.
(What is most surprising is that BSD behaves differently than Linux...)
I suspected that most of the errors would be due to MY code in Perl;
one piece implementing shell-less 2>&1 redirection, and one reporting
the error of exec() to the fork()'s parent. These would be easy to
fix (one could "relocate" generated filehandles to be above 2 - and,
for best result, above $^F). But it looks like there are different
cans of worms present in the picture...
Can you experiment (just run the script without arguments) and report
if you see some OTHER problems than those mentioned?
Thanks,
Ilya
------------------------------
Date: Wed, 21 Jul 2010 14:08:44 GMT
From: QoS <email@invalid.net>
Subject: Re: Stress testing process start
Message-Id: <MbD1o.18921$Bh2.13458@newsfe04.iad>
Ilya Zakharevich <nospam-abuse@ilyaz.org> wrote in message-id: <slrni4dgjm.u4u.nospam-abuse@powdermilk.math.berkeley.edu>
>
> Some time ago I got ideas that the process start logic of Perl may be
> flawed (at least the part implemented by me :-[ ;-). So I spend some
> time, and made a primitive script to stress-test Perl in the "tricky"
> situations. See ilyaz.org/software/tmp/check_redir.pl
>
> The best-hoped-for results are: having only <123//456> and <> as
> possible output strings (ones after the colon in the output), maybe
> interspersed with some warnings about closed filehandles. I tested on
> 3 architectures OS/2, BSD, and RedHat. I see much more than this.
>
> The problems which I see are:
>
> a) $! may become "Inappropriate ioctl for device" during
> closing (some of) STDERR, STDOUT, STDIN.
>
> b) I may see output <<456>>, which means that STDERR was NOT redirected.
>
> c) Shell may be called in
> open GRANDKID, q(|-), "$^X -wl - 2>&1" or die 12;
> while Perl has enough whistles to do it without shell ( as I assume
> it is doing with q(-|) ).
>
> d) I see "Unknown error 171324724" as content of $!.
>
> e) I see $? being -1, or 256 when things end in exit(0).
>
> f) the grandchild may block.
>
> (What is most surprising is that BSD behaves differently than Linux...)
>
> I suspected that most of the errors would be due to MY code in Perl;
> one piece implementing shell-less 2>&1 redirection, and one reporting
> the error of exec() to the fork()'s parent. These would be easy to
> fix (one could "relocate" generated filehandles to be above 2 - and,
> for best result, above $^F). But it looks like there are different
> cans of worms present in the picture...
>
> Can you experiment (just run the script without arguments) and report
> if you see some OTHER problems than those mentioned?
>
> Thanks,
> Ilya
On Windows XP (This is perl, v5.8.8 built for MSWin32-x86-multi-thread):
List form of pipe open not implemented at C:\Documents and Settings\Admin\Deskto
p\tst.pl line 23.
Hth,
J
------------------------------
Date: Wed, 21 Jul 2010 10:13:07 -0700
From: Jim Gibson <jimsgibson@gmail.com>
Subject: Re: Stress testing process start
Message-Id: <210720101013075236%jimsgibson@gmail.com>
In article <slrni4dgjm.u4u.nospam-abuse@powdermilk.math.berkeley.edu>,
Ilya Zakharevich <nospam-abuse@ilyaz.org> wrote:
> Some time ago I got ideas that the process start logic of Perl may be
> flawed (at least the part implemented by me :-[ ;-). So I spend some
> time, and made a primitive script to stress-test Perl in the "tricky"
> situations. See ilyaz.org/software/tmp/check_redir.pl
>
> Can you experiment (just run the script without arguments) and report
> if you see some OTHER problems than those mentioned?
Mac OS X (10.6.4), Perl 5.10.1:
% perl check_redir.pl
clo_in=0 clo_out=0 clo_err=0 pipe=0 via_shell=0: <123//456>
clo_in=0 clo_out=0 clo_err=0 pipe=0 via_shell=1: <123//456>
clo_in=0 clo_out=0 clo_err=0 pipe=i via_shell=0: <123//456>
clo_in=0 clo_out=0 clo_err=0 pipe=i via_shell=1: <123//456>
clo_in=0 clo_out=0 clo_err=0 pipe=o via_shell=0: <123//456>
clo_in=0 clo_out=0 clo_err=0 pipe=o via_shell=1: <123//456>
--
clo_in=0 clo_out=0 clo_err=1 pipe=0 via_shell=0: <123//456>
clo_in=0 clo_out=0 clo_err=1 pipe=0 via_shell=1: <123//456>
clo_in=0 clo_out=0 clo_err=1 pipe=i via_shell=0: <123//456>
clo_in=0 clo_out=0 clo_err=1 pipe=i via_shell=1: <123//456>
clo_in=0 clo_out=0 clo_err=1 pipe=o via_shell=0: <123//456>
clo_in=0 clo_out=0 clo_err=1 pipe=o via_shell=1: <123//456>
--
clo_in=0 clo_out=1 clo_err=0 pipe=0 via_shell=0: <grandkid: $?=256,
Inappropriate ioctl for device//sh: 1: Bad file descriptor>
clo_in=0 clo_out=1 clo_err=0 pipe=0 via_shell=1: <grandkid: $?=256,
Inappropriate ioctl for device//sh: 1: Bad file descriptor>
clo_in=0 clo_out=1 clo_err=0 pipe=i via_shell=0: <grandkid: $?=256,
//sh: 1: Bad file descriptor>
clo_in=0 clo_out=1 clo_err=0 pipe=i via_shell=1: <grandkid: $?=256,
//sh: 1: Bad file descriptor>
clo_in=0 clo_out=1 clo_err=0 pipe=o via_shell=0: <print() on closed
filehandle STDOUT at - line 23, <GRANDKID> line 1.//print() on closed
filehandle STDOUT at - line 23, <GRANDKID> line 2.>
clo_in=0 clo_out=1 clo_err=0 pipe=o via_shell=1: <print() on closed
filehandle STDOUT at - line 23, <GRANDKID> line 1.//print() on closed
filehandle STDOUT at - line 23, <GRANDKID> line 2.>
--
clo_in=0 clo_out=1 clo_err=1 pipe=0 via_shell=0: <>
clo_in=0 clo_out=1 clo_err=1 pipe=0 via_shell=1: <>
clo_in=0 clo_out=1 clo_err=1 pipe=i via_shell=0: <>
clo_in=0 clo_out=1 clo_err=1 pipe=i via_shell=1: <>
clo_in=0 clo_out=1 clo_err=1 pipe=o via_shell=0: <>
clo_in=0 clo_out=1 clo_err=1 pipe=o via_shell=1: <>
--
clo_in=1 clo_out=0 clo_err=0 pipe=0 via_shell=0: <123//456>
clo_in=1 clo_out=0 clo_err=0 pipe=0 via_shell=1: <123//456>
clo_in=1 clo_out=0 clo_err=0 pipe=i via_shell=0: <123//456>
clo_in=1 clo_out=0 clo_err=0 pipe=i via_shell=1: <123//456>
clo_in=1 clo_out=0 clo_err=0 pipe=o via_shell=0: <123//456>
clo_in=1 clo_out=0 clo_err=0 pipe=o via_shell=1: <123//456>
--
clo_in=1 clo_out=0 clo_err=1 pipe=0 via_shell=0: <123//456>
clo_in=1 clo_out=0 clo_err=1 pipe=0 via_shell=1: <123//456>
clo_in=1 clo_out=0 clo_err=1 pipe=i via_shell=0: <123//456>
clo_in=1 clo_out=0 clo_err=1 pipe=i via_shell=1: <123//456>
clo_in=1 clo_out=0 clo_err=1 pipe=o via_shell=0: <123//456>
clo_in=1 clo_out=0 clo_err=1 pipe=o via_shell=1: <123//456>
--
clo_in=1 clo_out=1 clo_err=0 pipe=0 via_shell=0: <grandkid: $?=-1,
Unknown error: 171324724>
clo_in=1 clo_out=1 clo_err=0 pipe=0 via_shell=1: <grandkid: $?=-1,
Unknown error: 171324724>
clo_in=1 clo_out=1 clo_err=0 pipe=i via_shell=0: <grandkid: $?=256,
//sh: 1: Bad file descriptor>
clo_in=1 clo_out=1 clo_err=0 pipe=i via_shell=1: <grandkid: $?=256,
//sh: 1: Bad file descriptor>
clo_in=1 clo_out=1 clo_err=0 pipe=o via_shell=0: <print() on closed
filehandle STDOUT at - line 23, <GRANDKID> line 1.//print() on closed
filehandle STDOUT at - line 23, <GRANDKID> line 2.>
clo_in=1 clo_out=1 clo_err=0 pipe=o via_shell=1: <print() on closed
filehandle STDOUT at - line 23, <GRANDKID> line 1.//print() on closed
filehandle STDOUT at - line 23, <GRANDKID> line 2.>
--
clo_in=1 clo_out=1 clo_err=1 pipe=0 via_shell=0: <>
clo_in=1 clo_out=1 clo_err=1 pipe=0 via_shell=1: <>
clo_in=1 clo_out=1 clo_err=1 pipe=i via_shell=0: <>
clo_in=1 clo_out=1 clo_err=1 pipe=i via_shell=1: <>
clo_in=1 clo_out=1 clo_err=1 pipe=o via_shell=0: <>
clo_in=1 clo_out=1 clo_err=1 pipe=o via_shell=1: <>
--
--
Jim Gibson
------------------------------
Date: 6 Apr 2001 21:33:47 GMT (Last modified)
From: Perl-Users-Request@ruby.oce.orst.edu (Perl-Users-Digest Admin)
Subject: Digest Administrivia (Last modified: 6 Apr 01)
Message-Id: <null>
Administrivia:
To submit articles to comp.lang.perl.announce, send your article to
clpa@perl.com.
Back issues are available via anonymous ftp from
ftp://cil-www.oce.orst.edu/pub/perl/old-digests.
#For other requests pertaining to the digest, send mail to
#perl-users-request@ruby.oce.orst.edu. Do not waste your time or mine
#sending perl questions to the -request address, I don't have time to
#answer them even if I did know the answer.
------------------------------
End of Perl-Users Digest V11 Issue 3038
***************************************
