[32366] in Perl-Users-Digest
Perl-Users Digest, Issue: 3633 Volume: 11
daemon@ATHENA.MIT.EDU (Perl-Users Digest)
Wed Mar 7 18:09:46 2012
Date: Wed, 7 Mar 2012 15:09:08 -0800 (PST)
From: Perl-Users Digest <Perl-Users-Request@ruby.OCE.ORST.EDU>
To: Perl-Users@ruby.OCE.ORST.EDU (Perl-Users Digest)
Perl-Users Digest Wed, 7 Mar 2012 Volume: 11 Number: 3633
Today's topics:
Error building Math::Pari on AIX 7.1 <mwulfff@yahoo.de>
Re: Error building Math::Pari on AIX 7.1 <rweikusat@mssgmbh.com>
Help with dynamic regex <dave@invalid.invalid>
Re: Help with dynamic regex <rweikusat@mssgmbh.com>
Re: Help with dynamic regex <rweikusat@mssgmbh.com>
Re: Help with dynamic regex <dave@invalid.invalid>
Re: Help with dynamic regex <dave@invalid.invalid>
Re: Help with dynamic regex <rweikusat@mssgmbh.com>
Re: Help with dynamic regex <dave@invalid.invalid>
Re: Help with dynamic regex <ben@morrow.me.uk>
Re: Help with dynamic regex <rweikusat@mssgmbh.com>
Re: Help with dynamic regex <rweikusat@mssgmbh.com>
Re: Help with dynamic regex <ben@morrow.me.uk>
Re: Help with dynamic regex <rweikusat@mssgmbh.com>
Re: little quiz (Seymour J.)
Re: references (4567890) <rvtol+usenet@xs4all.nl>
Digest Administrivia (Last modified: 6 Apr 01) (Perl-Users-Digest Admin)
----------------------------------------------------------------------
Date: Wed, 7 Mar 2012 02:33:14 -0800 (PST)
From: Michael Wulff <mwulfff@yahoo.de>
Subject: Error building Math::Pari on AIX 7.1
Message-Id: <e5e32252-303f-4b5e-b029-ebbce1b02c96@q12g2000yqg.googlegroups.com>
Hello,
I am trying to compile Math::Pari in a Perl 5.14.2 environment on AIX
7.1 TL01 64 Bit using vac compiler v11.1.0.8. After some diffilculties
I finally managed to compile GP/PARI using the options
env LD=/usr/ccs/bin/ld ./Configure --static
and `make gp-sta'
(See also thread http://pari.math.u-bordeaux.fr/archives/pari-users-1202/msg00017.html
in pari users list.)
When buiding Math::Pari I first got an error that mpinl.h was not
found. I could fix this copying this header file into the build
directory:
cp /usr/local/include/pari/mpinl.h ../pari-2.5.1/src/headers
But now I run into the following error:
xlc_r -q64 -c -I ../../pari-2.5.1/src -I ../../pari-2.5.1/src/
headers -I ../../pari-2.5.1/src/graph -I . -D_ALL_SOURCE -
D_ANSI_C_SOURCE -D_POSIX_SOURCE -qmaxmem=-1 -qnoansialias -
DUSE_NATIVE_DLOPEN -DNEED_PTHREAD_INIT -qlanglvl=extended -q64 -
DUSE_64_BIT_ALL -q64 -O -DVERSION=\"\" -DXS_VERSION=\"\" -
Derr=pari_err -DDYNAMIC_PLOTTING -o polarit1.o ../../pari-2.5.1/src/
basemath/polarit1.c
xlc_r -q64 -c -I ../../pari-2.5.1/src -I ../../pari-2.5.1/src/
headers -I ../../pari-2.5.1/src/graph -I . -D_ALL_SOURCE -
D_ANSI_C_SOURCE -D_POSIX_SOURCE -qmaxmem=-1 -qnoansialias -
DUSE_NATIVE_DLOPEN -DNEED_PTHREAD_INIT -qlanglvl=extended -q64 -
DUSE_64_BIT_ALL -q64 -O -DVERSION=\"\" -DXS_VERSION=\"\" -
Derr=pari_err -DDYNAMIC_PLOTTING -o paricfg.o ../../pari-2.5.1/src/
language/paricfg.c
"../../pari-2.5.1/src/language/paricfg.c", line 21.31: 1506-221 (S)
Initializer must be a valid constant expression.
"../../pari-2.5.1/src/language/paricfg.c", line 22.33: 1506-221 (S)
Initializer must be a valid constant expression.
"../../pari-2.5.1/src/language/paricfg.c", line 24.34: 1506-221 (S)
Initializer must be a valid constant expression.
make: 1254-004 The error code from the last command is 1.
Stop.
make: 1254-004 The error code from the last command is 2.
Stop.
I get the same error when executing `perl Makefile.PL LIBPARI="-L/lib -
lpari"'
Any ideas?
Here are the outputs from perl Makefile.PL and perl -V:
# perl Makefile.PL
Filtered out versions too new...
Setting up Math::Pari with Perl 5.014002 on aix version 7.1.0.0;
cc=xlc_r -q64, gccversion=, cccdlflags=' ',
ccflags='-D_ALL_SOURCE -D_ANSI_C_SOURCE -D_POSIX_SOURCE -qmaxmem=-1
-qnoansialias -DUSE_NATIVE_DLOPEN -DNEED_PTHREAD_INIT -
qlanglvl=extended -q64 -DUSE_64_BIT_ALL -q64',
optimize='-O',
ld=ld, ldflags='-brtl -bdynamic -b64'.
Found GP/PARI build directory in ../pari-2.5.1
(integer-formatted version 2005001).
...Generating libPARI/paricfg.h ...
Scanning header files...
...ulong
...getrusage
...CLK_TCK not defined
Creating libPARI/paricfg.h...
sh: gp: not found
...Processor of family `port' detected
Checking if your kit is complete...
Looks good
...Processor of family `port' detected
...I will use portable assembler-less build
...Assembler is not GNU assembler
Writing Makefile for Math::PARI::libPARI
Writing MYMETA.yml
Writing Makefile for Math::Pari
Writing MYMETA.yml
# perl -V
Summary of my perl5 (revision 5 version 14 subversion 2)
configuration:
Platform:
osname=aix, osvers=7.1.0.0, archname=aix-thread-multi-64all
uname='aix pstest7 1 7 00f6cffd4c00 '
config_args='-d -Dcc=xlc_r -Duseshrplib -Duse64bitall -
Dusethreads'
hint=recommended, useposix=true, d_sigaction=define
useithreads=define, usemultiplicity=define
useperlio=define, d_sfio=undef, uselargefiles=define,
usesocks=undef
use64bitint=define, use64bitall=define, uselongdouble=undef
usemymalloc=n, bincompat5005=undef
Compiler:
cc='xlc_r -q64', ccflags ='-D_ALL_SOURCE -D_ANSI_C_SOURCE -
D_POSIX_SOURCE -qmaxmem=-1 -qnoansialias -DUSE_NATIVE_DLOPEN -
DNEED_PTHREAD_INIT -qlanglvl=extended -q64 -DUSE_64_BIT_ALL -q64',
optimize='-O',
cppflags='-D_ALL_SOURCE -D_ANSI_C_SOURCE -D_POSIX_SOURCE -
qmaxmem=-1 -qnoansialias -DUSE_NATIVE_DLOPEN -DNEED_PTHREAD_INIT -
qlanglvl=extended'
ccversion='11.1.0.8', gccversion='', gccosandvers=''
intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=87654321
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=8
ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=8
alignbytes=8, prototype=define
Linker and Libraries:
ld='ld', ldflags ='-brtl -bdynamic -b64'
libpth=/lib /usr/lib /usr/ccs/lib /usr/lib64
libs=-lbind -lnsl -ldbm -ldl -lld -lm -lcrypt -lpthreads -lc
perllibs=-lbind -lnsl -ldl -lld -lm -lcrypt -lpthreads -lc
libc=, so=a, useshrplib=true, libperl=libperl.a
gnulibc_version=''
Dynamic Linking:
dlsrc=dl_aix.xs, dlext=so, d_dlsymun=undef, ccdlflags=' -bE:/usr/
local/lib/perl5/5.14.2/aix-thread-multi-64all/CORE/perl.exp'
cccdlflags=' ', lddlflags='-b64 -bhalt:4 -G -bI:$(PERL_INC)/
perl.exp -bE:$(BASEEXT).exp -bnoentry -lpthreads -lc -lm'
Characteristics of this binary (from libperl):
Compile-time options: MULTIPLICITY PERL_DONT_CREATE_GVSV
PERL_IMPLICIT_CONTEXT PERL_MALLOC_WRAP
PERL_PRESERVE_IVUV USE_64_BIT_ALL
USE_64_BIT_INT
USE_ITHREADS USE_LARGE_FILES USE_PERLIO
USE_PERL_ATOF
USE_REENTRANT_API
Built under aix
Compiled at Jan 26 2012 11:11:04
@INC:
/usr/local/lib/perl5/site_perl/5.14.2/aix-thread-multi-64all
/usr/local/lib/perl5/site_perl/5.14.2
/usr/local/lib/perl5/5.14.2/aix-thread-multi-64all
/usr/local/lib/perl5/5.14.2
.
Regards,
Michael
------------------------------
Date: Wed, 07 Mar 2012 14:01:02 +0000
From: Rainer Weikusat <rweikusat@mssgmbh.com>
Subject: Re: Error building Math::Pari on AIX 7.1
Message-Id: <878vjczf1d.fsf@sapphire.mobileactivedefense.com>
Michael Wulff <mwulfff@yahoo.de> writes:
> I am trying to compile Math::Pari in a Perl 5.14.2 environment on AIX
> 7.1 TL01 64 Bit using vac compiler v11.1.0.8. After some diffilculties
> I finally managed to compile GP/PARI using the options
[...]
> I run into the following error:
>
> xlc_r -q64 -c -I ../../pari-2.5.1/src -I ../../pari-2.5.1/src/
> headers -I ../../pari-2.5.1/src/graph -I . -D_ALL_SOURCE -
> D_ANSI_C_SOURCE -D_POSIX_SOURCE -qmaxmem=-1 -qnoansialias -
> DUSE_NATIVE_DLOPEN -DNEED_PTHREAD_INIT -qlanglvl=extended -q64 -
> DUSE_64_BIT_ALL -q64 -O -DVERSION=\"\" -DXS_VERSION=\"\" -
> Derr=pari_err -DDYNAMIC_PLOTTING -o polarit1.o ../../pari-2.5.1/src/
> basemath/polarit1.c
> xlc_r -q64 -c -I ../../pari-2.5.1/src -I ../../pari-2.5.1/src/
> headers -I ../../pari-2.5.1/src/graph -I . -D_ALL_SOURCE -
> D_ANSI_C_SOURCE -D_POSIX_SOURCE -qmaxmem=-1 -qnoansialias -
> DUSE_NATIVE_DLOPEN -DNEED_PTHREAD_INIT -qlanglvl=extended -q64 -
> DUSE_64_BIT_ALL -q64 -O -DVERSION=\"\" -DXS_VERSION=\"\" -
> Derr=pari_err -DDYNAMIC_PLOTTING -o paricfg.o ../../pari-2.5.1/src/
> language/paricfg.c
> "../../pari-2.5.1/src/language/paricfg.c", line 21.31: 1506-221 (S)
> Initializer must be a valid constant expression.
> "../../pari-2.5.1/src/language/paricfg.c", line 22.33: 1506-221 (S)
> Initializer must be a valid constant expression.
> "../../pari-2.5.1/src/language/paricfg.c", line 24.34: 1506-221 (S)
> Initializer must be a valid constant expression.
You will need to have a look at the mentioned lines of this file in
order to determine what's wrong. As far as the message goes, this
should be a variable intialization using something the compiler you
are using does not consider to be a constant (C) expression but
without knowing that this non-expression happens to be, there's litte
which can be said to that.
------------------------------
Date: Wed, 7 Mar 2012 16:29:12 +0000 (UTC)
From: "Dave Saville" <dave@invalid.invalid>
Subject: Help with dynamic regex
Message-Id: <fV45K0OBJxbE-pn2-jjQ6nZUSugRO@localhost>
I have a general purpose CGI script to interogate various log files on
my server. Because sometimes I want to limit what I see I have an
optional field where I can type a regular expression.
If the field is blank then obviously nothing special needs to be done,
but if there is something then I need to build a regex. If the field
is just for example abc then I need m/abc/ generated - or something
that will do that. OTH if the field has /abc/i then I need m/abc/i and
so on.
Things are further complicated in that if there is a regular
expression then I want that matching text highlighted in the output.
What I have follows - but it does not always work. For example if I
have /o/g it does not highlight all the o's.
sub getlog
{
my $logfile = shift;
open LOG, "<$logfile" or die "Can't open $logfile $!";
local $regex = $regex;
if ( $regex )
{
if ( $regex =~ m/^(.)(.*)\1([giomsx]+)$/ )
{
my $body = $2;
my $mod = $3;
if ( $mod )
{
$regex = qr/(?$mod)$body/;
}
else
{
$regex = qr/$body/;
}
}
}
while ( <LOG> )
{
s/\</</gm;
s/\>/>/gm;
s/\xf8/°/gm;
if ( $regex )
{
push @log, $_ if s/($regex)/\<span
class="searchlight"\>$1\<\/span\>/;
}
else
{
push @log, $_;
}
}
close LOG;
}
TIA
--
Regards
Dave Saville
------------------------------
Date: Wed, 07 Mar 2012 17:37:58 +0000
From: Rainer Weikusat <rweikusat@mssgmbh.com>
Subject: Re: Help with dynamic regex
Message-Id: <87y5rc8g7d.fsf@sapphire.mobileactivedefense.com>
"Dave Saville" <dave@invalid.invalid> writes:
> I have a general purpose CGI script to interogate various log files on
> my server. Because sometimes I want to limit what I see I have an
> optional field where I can type a regular expression.
>
> If the field is blank then obviously nothing special needs to be done,
> but if there is something then I need to build a regex. If the field
> is just for example abc then I need m/abc/ generated - or something
> that will do that. OTH if the field has /abc/i then I need m/abc/i and
> so on.
>
> Things are further complicated in that if there is a regular
> expression then I want that matching text highlighted in the output.
> What I have follows - but it does not always work. For example if I
> have /o/g it does not highlight all the o's.
>
> sub getlog
> {
> my $logfile = shift;
> open LOG, "<$logfile" or die "Can't open $logfile $!";
>
> local $regex = $regex;
A general remark: Consider using my for local variables of
subroutines. It is generally faster and these variables are then
private to the subroutine which defines them while the local binding
will be visible to all subroutines invoked from this one. Also, using
a parameter to pass data into subroutines is usually sensible because
the subroutine then becomes independent of its environment and it is
easier to see what data flows into it (ie, without reading through all
of the code). Here, this would mean using something like
sub getlog
{
my ($logfile, $regex) = @_;
>
> if ( $regex )
> {
> if ( $regex =~ m/^(.)(.*)\1([giomsx]+)$/ )
> {
> my $body = $2;
> my $mod = $3;
>
> if ( $mod )
> {
> $regex = qr/(?$mod)$body/;
> }
> else
> {
> $regex = qr/$body/;
> }
> }
> }
>
> while ( <LOG> )
> {
> s/\</</gm;
> s/\>/>/gm;
> s/\xf8/°/gm;
>
> if ( $regex )
> {
> push @log, $_ if s/($regex)/\<span
> class="searchlight"\>$1\<\/span\>/;
> }
According to perlre(1) qr/// doesn't support the g modifier and if you
want 'global substituations' (in a particular line of input), you need
to apply that to s///. A possible way to achieve that would be to
create a subroutine which performs the desired modification on the input
line and invoke that for each line. Example how this could be done:
------------
my $r = '|g|g'; # set to regex
my $s;
if ($r =~ /(.)(.*)\1([giomsx]*)$/) {
$s = eval(sprintf('sub { $_[0] =~ s/(%s)/XX\1XX/%s; }',
$2, $3));
} else {
$s = sub { };
}
$a = 'gagbgcgd';
$s->($a);
print $a, "\n";
------------
NB: The created substitution routine modifies its argument since this
gets around the need to copy the complete log file.
------------------------------
Date: Wed, 07 Mar 2012 17:54:28 +0000
From: Rainer Weikusat <rweikusat@mssgmbh.com>
Subject: Re: Help with dynamic regex
Message-Id: <87pqco8ffv.fsf@sapphire.mobileactivedefense.com>
Rainer Weikusat <rweikusat@mssgmbh.com> writes:
[...]
> if ($r =~ /(.)(.*)\1([giomsx]*)$/) {
> $s = eval(sprintf('sub { $_[0] =~ s/(%s)/XX\1XX/%s; }',
> $2, $3));
Additional remark: Because this is based on interpolating a string
into some code, it will be possible to use this to execute arbitrary
code by providing suitable input to the script, eg,
my $r = '|a)//; print "Gotcha!\n"; s/(b|';
will print Gotcha! whenever the substituation routine is
executed. OTOH, since regexes support executing code anyway, this
might not be much of a concern.
------------------------------
Date: Wed, 7 Mar 2012 18:58:04 +0000 (UTC)
From: "Dave Saville" <dave@invalid.invalid>
Subject: Re: Help with dynamic regex
Message-Id: <fV45K0OBJxbE-pn2-V3rOFVhdlVZd@localhost>
On Wed, 7 Mar 2012 17:37:58 UTC, Rainer Weikusat
<rweikusat@mssgmbh.com> wrote:
<snip>
> A general remark: Consider using my for local variables of
> subroutines. It is generally faster and these variables are then
> private to the subroutine which defines them while the local binding
> will be visible to all subroutines invoked from this one. Also, using
> a parameter to pass data into subroutines is usually sensible because
> the subroutine then becomes independent of its environment and it is
> easier to see what data flows into it (ie, without reading through all
> of the code). Here, this would mean using something like
Noted.
<snip>
> According to perlre(1) qr/// doesn't support the g modifier and if you
That explains a lot :-)
> want 'global substituations' (in a particular line of input), you need
> to apply that to s///. A possible way to achieve that would be to
> create a subroutine which performs the desired modification on the input
> line and invoke that for each line. Example how this could be done:
>
> ------------
> my $r = '|g|g'; # set to regex
> my $s;
>
> if ($r =~ /(.)(.*)\1([giomsx]*)$/) {
> $s = eval(sprintf('sub { $_[0] =~ s/(%s)/XX\1XX/%s; }',
> $2, $3));
> } else {
> $s = sub { };
> }
>
> $a = 'gagbgcgd';
> $s->($a);
>
> print $a, "\n";
> ------------
Sorry you have lost me there. Apart from it not doing what I need if
the regex is just supplied as "abc" I do not understand that eval -
obviously it runs the sprintf but what is that sub all about or the $s
= sub{} ?
I am guessing that $s gets set to *something* that $s->($a) passes $a
to - but how does that work?
Thanks for the help.
--
Regards
Dave Saville
------------------------------
Date: Wed, 7 Mar 2012 18:59:27 +0000 (UTC)
From: "Dave Saville" <dave@invalid.invalid>
Subject: Re: Help with dynamic regex
Message-Id: <fV45K0OBJxbE-pn2-sfMdjDx0rhwv@localhost>
On Wed, 7 Mar 2012 17:54:28 UTC, Rainer Weikusat
<rweikusat@mssgmbh.com> wrote:
> Rainer Weikusat <rweikusat@mssgmbh.com> writes:
>
> [...]
>
> > if ($r =~ /(.)(.*)\1([giomsx]*)$/) {
> > $s = eval(sprintf('sub { $_[0] =~ s/(%s)/XX\1XX/%s; }',
> > $2, $3));
>
> Additional remark: Because this is based on interpolating a string
> into some code, it will be possible to use this to execute arbitrary
> code by providing suitable input to the script, eg,
>
> my $r = '|a)//; print "Gotcha!\n"; s/(b|';
>
> will print Gotcha! whenever the substituation routine is
> executed. OTOH, since regexes support executing code anyway, this
> might not be much of a concern.
No concern at all in this case as it's just me. :-)
--
Regards
Dave Saville
------------------------------
Date: Wed, 07 Mar 2012 19:13:13 +0000
From: Rainer Weikusat <rweikusat@mssgmbh.com>
Subject: Re: Help with dynamic regex
Message-Id: <87vcmg6x86.fsf@sapphire.mobileactivedefense.com>
"Dave Saville" <dave@invalid.invalid> writes:
> On Wed, 7 Mar 2012 17:37:58 UTC, Rainer Weikusat
[...]
>> ------------
>> my $r = '|g|g'; # set to regex
>> my $s;
>>
>> if ($r =~ /(.)(.*)\1([giomsx]*)$/) {
>> $s = eval(sprintf('sub { $_[0] =~ s/(%s)/XX\1XX/%s; }',
>> $2, $3));
>> } else {
>> $s = sub { };
>> }
>>
>> $a = 'gagbgcgd';
>> $s->($a);
>>
>> print $a, "\n";
>> ------------
>
> Sorry you have lost me there. Apart from it not doing what I need if
> the regex is just supplied as "abc"
It expects that all regexes start and end with a delimiter-character.
It might make more sense to use that in the function template, eg
sprintf('sub { $_[0] =~ s%s(%s)%s/XX\1XX%s%s; }',
$1, $2, $1, $1, $3);
since there has to be some kind of delimiter and it must not occur
unescaped in the 're text' itself (so the burden of picking something
suitable is on the user). Otherwise, some kind of escaping would need
to be added.
> I do not understand that eval -
> obviously it runs the sprintf but what is that sub all about or the $s
> = sub{} ?
>
sub { } just created an anonymous subroutine which does nothing and
returns a coderef pointing to that. The 'eval' compiles and runs the
code contained in the string passed to it and returns the result. In
this case, that's a references to an anonymous subroutine which does
the intended pattern substituation (subject to the limitations
mentioned elswhere).
------------------------------
Date: Wed, 7 Mar 2012 19:25:07 +0000 (UTC)
From: "Dave Saville" <dave@invalid.invalid>
Subject: Re: Help with dynamic regex
Message-Id: <fV45K0OBJxbE-pn2-SdBsyquzgjzx@localhost>
On Wed, 7 Mar 2012 19:13:13 UTC, Rainer Weikusat
<rweikusat@mssgmbh.com> wrote:
> "Dave Saville" <dave@invalid.invalid> writes:
> > On Wed, 7 Mar 2012 17:37:58 UTC, Rainer Weikusat
>
> [...]
>
>
> >> ------------
> >> my $r = '|g|g'; # set to regex
> >> my $s;
> >>
> >> if ($r =~ /(.)(.*)\1([giomsx]*)$/) {
> >> $s = eval(sprintf('sub { $_[0] =~ s/(%s)/XX\1XX/%s; }',
> >> $2, $3));
> >> } else {
> >> $s = sub { };
> >> }
> >>
> >> $a = 'gagbgcgd';
> >> $s->($a);
> >>
> >> print $a, "\n";
> >> ------------
> >
> > Sorry you have lost me there. Apart from it not doing what I need if
> > the regex is just supplied as "abc"
>
> It expects that all regexes start and end with a delimiter-character.
> It might make more sense to use that in the function template, eg
>
> sprintf('sub { $_[0] =~ s%s(%s)%s/XX\1XX%s%s; }',
> $1, $2, $1, $1, $3);
>
> since there has to be some kind of delimiter and it must not occur
> unescaped in the 're text' itself (so the burden of picking something
> suitable is on the user). Otherwise, some kind of escaping would need
> to be added.
>
> > I do not understand that eval -
> > obviously it runs the sprintf but what is that sub all about or the $s
> > = sub{} ?
> >
>
> sub { } just created an anonymous subroutine which does nothing and
> returns a coderef pointing to that. The 'eval' compiles and runs the
> code contained in the string passed to it and returns the result. In
> this case, that's a references to an anonymous subroutine which does
> the intended pattern substituation (subject to the limitations
> mentioned elswhere).
Thanks again - my brain hurts. :-)
--
Regards
Dave Saville
------------------------------
Date: Wed, 7 Mar 2012 20:20:09 +0000
From: Ben Morrow <ben@morrow.me.uk>
Subject: Re: Help with dynamic regex
Message-Id: <pgsk29-4dj1.ln1@anubis.morrow.me.uk>
Quoth "Dave Saville" <dave@invalid.invalid>:
> On Wed, 7 Mar 2012 17:54:28 UTC, Rainer Weikusat
> <rweikusat@mssgmbh.com> wrote:
> > Rainer Weikusat <rweikusat@mssgmbh.com> writes:
> >
> > > if ($r =~ /(.)(.*)\1([giomsx]*)$/) {
> > > $s = eval(sprintf('sub { $_[0] =~ s/(%s)/XX\1XX/%s; }',
> > > $2, $3));
> >
> > Additional remark: Because this is based on interpolating a string
> > into some code,
...it is a completely stupid thing to do in a CGI script. This is,
unforunately, a case which would have slipped by taint mode, but it's
still ridiculously dangerous.
> > it will be possible to use this to execute arbitrary
> > code by providing suitable input to the script, eg,
> >
> > my $r = '|a)//; print "Gotcha!\n"; s/(b|';
> >
> > will print Gotcha! whenever the substituation routine is
> > executed. OTOH, since regexes support executing code anyway, this
> > might not be much of a concern.
Regexes are protected against interpolated sections containing (?{}) and
the like. See the documentation for 'use re "eval"'.
> No concern at all in this case as it's just me. :-)
That is the *wrong* attitude. If it's a CGI script you *must* assume Bad
People will get their hands on it, no matter how 'internal' you think
the webserver serving it is. You should be using taint mode and writing
with security in mind.
Ben
------------------------------
Date: Wed, 07 Mar 2012 21:02:13 +0000
From: Rainer Weikusat <rweikusat@mssgmbh.com>
Subject: Re: Help with dynamic regex
Message-Id: <87r4x46s6i.fsf@sapphire.mobileactivedefense.com>
Ben Morrow <ben@morrow.me.uk> writes:
> Quoth "Dave Saville" <dave@invalid.invalid>:
>> On Wed, 7 Mar 2012 17:54:28 UTC, Rainer Weikusat
>> <rweikusat@mssgmbh.com> wrote:
>> > Rainer Weikusat <rweikusat@mssgmbh.com> writes:
>> >
>> > > if ($r =~ /(.)(.*)\1([giomsx]*)$/) {
>> > > $s = eval(sprintf('sub { $_[0] =~ s/(%s)/XX\1XX/%s; }',
>> > > $2, $3));
>> >
>> > Additional remark: Because this is based on interpolating a string
>> > into some code,
>
> ...it is a completely stupid thing to do in a CGI script.
Depending on the purpose of the script, ie, is it only going to be
used by people who control the machine, anyway, as was the case here,
the input valid needs to be sanitized in some suitable way in order to
prevent code execution. Since this might not be self-evident to
everyone (aka 'SQL injection?'), I wrote the additional posting
pointing this out. Consequently, you assertion that I wrote this
because I would be 'too stupid' to understand the consequence, while
presumably being your honest conviction, does not hold water.
[...]
>> > it will be possible to use this to execute arbitrary
>> > code by providing suitable input to the script, eg,
>> >
>> > my $r = '|a)//; print "Gotcha!\n"; s/(b|';
>> >
>> > will print Gotcha! whenever the substituation routine is
>> > executed. OTOH, since regexes support executing code anyway, this
>> > might not be much of a concern.
>
> Regexes are protected against interpolated sections containing (?{}) and
> the like. See the documentation for 'use re "eval"'.
This is not applicable here, cf
when "use re 'eval'" is in effect, a regex is allowed to
contain "(?{ ... })" zero-width assertions even if regular
expression contains variable
interpolation. That is normally disallowed, since it is a
potential security risk. Note that this pragma is ignored when
the regular expression is obtained from tainted data, i.e.
evaluation is always disallowed with tainted regular
expressions. See "(?{ code })"
in perlre.
For the purpose of this pragma, interpolation of precompiled
regular expressions (i.e., the result of "qr//") is not
considered variable interpolation.
>> No concern at all in this case as it's just me. :-)
>
> That is the *wrong* attitude.
Nope. That's the appropriate attitude: Security isn't about 'voodoo
dancing for the sake of it' but about preventing some kind of
privilege escalation from happening. And 'requiring privilege' in
order to be able to perform the operation at all is a completely valid
strategy to achieve that: It is conceptually identical to having a
'privileged' system account of some kind.
------------------------------
Date: Wed, 07 Mar 2012 21:41:16 +0000
From: Rainer Weikusat <rweikusat@mssgmbh.com>
Subject: Re: Help with dynamic regex
Message-Id: <87mx7s6qdf.fsf@sapphire.mobileactivedefense.com>
Rainer Weikusat <rweikusat@mssgmbh.com> writes:
> Ben Morrow <ben@morrow.me.uk> writes:
>> Quoth "Dave Saville" <dave@invalid.invalid>:
>>> On Wed, 7 Mar 2012 17:54:28 UTC, Rainer Weikusat
>>> <rweikusat@mssgmbh.com> wrote:
>>> > Rainer Weikusat <rweikusat@mssgmbh.com> writes:
>>> >
>>> > > if ($r =~ /(.)(.*)\1([giomsx]*)$/) {
>>> > > $s = eval(sprintf('sub { $_[0] =~ s/(%s)/XX\1XX/%s; }',
>>> > > $2, $3));
>>> >
>>> > Additional remark: Because this is based on interpolating a string
>>> > into some code,
>>
>> ...it is a completely stupid thing to do in a CGI script.
>
> Depending on the purpose of the script,
[...]
> the input valid needs to be sanitized in some suitable way in order to
> prevent code execution.
This is actually extremely easy to do since Perl already provides a
suitable hook into 'the regex compiler', namely, the qr//
operator. Taking this into account, my toy example script could be
rewritten as follows:
-----------------
my $r = '/g/g';
my ($s, $mod);
if ($r) {
if ($r =~ /(.)(.*)\1([giomsx]*)$/) {
$r = $2;
$mod = $3;
}
$s = eval(sprintf('sub { $_[0] =~ s/(%s)/XX\1XX/%s; }',
qr/$r/, $mod));
} else {
$s = sub { };
}
$a = 'gagbgcgd';
$s->($a);
print $a, "\n";
-----------------
Which should both restore the ability to omit the delimiters when
there are no modifiers and prevent insertion of arbitrary code into
the created substituation routine.
[...]
>> Regexes are protected against interpolated sections containing (?{}) and
>> the like. See the documentation for 'use re "eval"'.
>
> This is not applicable here, cf
[...]
> For the purpose of this pragma, interpolation of precompiled
> regular expressions (i.e., the result of "qr//") is not
> considered variable interpolation.
The input to qr// is considered variable interpolation, though,
meaning, ordinarily, (?{...}) won't work.
------------------------------
Date: Wed, 7 Mar 2012 22:25:46 +0000
From: Ben Morrow <ben@morrow.me.uk>
Subject: Re: Help with dynamic regex
Message-Id: <as3l29-24l1.ln1@anubis.morrow.me.uk>
Quoth Rainer Weikusat <rweikusat@mssgmbh.com>:
>
> my $r = '/g/g';
> my ($s, $mod);
>
> if ($r) {
> if ($r =~ /(.)(.*)\1([giomsx]*)$/) {
> $r = $2;
> $mod = $3;
> }
>
> $s = eval(sprintf('sub { $_[0] =~ s/(%s)/XX\1XX/%s; }',
> qr/$r/, $mod));
Clever. Of course, eval is lexically scoped within the surrounding
block, so there's no need to compile the regex twice:
my $qr = qr/$r/;
$s = eval sprintf 'sub { $_[0] =~ s/$qr/XX\1XX/%s; }', $mod;
I might even consider that safe enough to use, given that the
interpolated section is now entirely predictable, except that there's no
need for eval at all. The only modifers you are accounting for that
can't be put in a (?X) are /g and /o, and /o isn't useful anyway, so
there's really nothing wrong with
my $g = $mod =~ s/g//g;
if ($g) {
$a =~ s/(?$mod)$r/XX$1XX/g;
}
else {
$a =~ s/(?$mod)$r/XX$1XX/;
}
and it's a whole lot harder to get wrong. (There is also /c, but that
isn't useful in this case either.)
> Rainer Weikusat <rweikusat@mssgmbh.com> writes:
> > Ben Morrow <ben@morrow.me.uk> writes:
> >
> >> Regexes are protected against interpolated sections containing (?{}) and
> >> the like. See the documentation for 'use re "eval"'.
> >
> > This is not applicable here, cf
>
> [...]
>
> > For the purpose of this pragma, interpolation of precompiled
> > regular expressions (i.e., the result of "qr//") is not
> > considered variable interpolation.
>
> The input to qr// is considered variable interpolation, though,
> meaning, ordinarily, (?{...}) won't work.
...which was my point. Interpolating into a regex is safe (modulo DoS
attacks using everlasting patterns) in a way eval can never be.
Ben
------------------------------
Date: Wed, 07 Mar 2012 22:58:37 +0000
From: Rainer Weikusat <rweikusat@mssgmbh.com>
Subject: Re: Help with dynamic regex
Message-Id: <87ipig6msi.fsf@sapphire.mobileactivedefense.com>
Ben Morrow <ben@morrow.me.uk> writes:
> Quoth Rainer Weikusat <rweikusat@mssgmbh.com>:
>>
>> my $r = '/g/g';
>> my ($s, $mod);
>>
>> if ($r) {
>> if ($r =~ /(.)(.*)\1([giomsx]*)$/) {
>> $r = $2;
>> $mod = $3;
>> }
>>
>> $s = eval(sprintf('sub { $_[0] =~ s/(%s)/XX\1XX/%s; }',
>> qr/$r/, $mod));
>
> Clever. Of course, eval is lexically scoped within the surrounding
> block, so there's no need to compile the regex twice:
>
> my $qr = qr/$r/;
> $s = eval sprintf 'sub { $_[0] =~ s/$qr/XX\1XX/%s; }', $mod;
>
This is a regex with a variable dynamically interpolated into it, IOW,
the $qr is evaluated every time the sub is executed, as opposed to
once when it is compiled. Since this is not good for anything, it
should rather be avoided.
> I might even consider that safe enough to use, given that the
> interpolated section is now entirely predictable, except that there's no
> need for eval at all. The only modifers you are accounting for that
> can't be put in a (?X) are /g and /o, and /o isn't useful anyway, so
> there's really nothing wrong with
>
> my $g = $mod =~ s/g//g;
> if ($g) {
> $a =~ s/(?$mod)$r/XX$1XX/g;
> }
> else {
> $a =~ s/(?$mod)$r/XX$1XX/;
> }
>
> and it's a whole lot harder to get wrong.
It is more code (consequently, it is easier to get wrong) and it
requires a conditional test whose outcome is always either one way or
the other way for each line of input processed. And log files can
easily be huge. Not to mention that it is also prone to so-called
'update anomalies' since there are now two replacement expressions
which need to be kept in sync manually.
[...]
>> >> Regexes are protected against interpolated sections containing (?{}) and
>> >> the like. See the documentation for 'use re "eval"'.
>> >
>> > This is not applicable here, cf
>>
>> [...]
>>
>> > For the purpose of this pragma, interpolation of precompiled
>> > regular expressions (i.e., the result of "qr//") is not
>> > considered variable interpolation.
>>
>> The input to qr// is considered variable interpolation, though,
>> meaning, ordinarily, (?{...}) won't work.
>
> ...which was my point. Interpolating into a regex is safe (modulo DoS
> attacks using everlasting patterns) in a way eval can never be.
There's no problem with eval per se, just with the string which is
actually evaluated. Consequently, there is some (possibly absent)
input validation/ sanitization code and this code can either be
correct, then, evaluating the input is 'safe;', or incorrect and then, it
isn't provided 'evaluating user input' isn't intentionally done in
order to enable users to execute whatever Perl code might be useful
for them. I've done this at least once in the past, with a script
which was not intended to be a security boundary, where users of the
script could use an arbitrary Perl expression in order to provide 'a
size argument'. The idea was that it might be convenient to use a Perl
expression calculating the actual size instead of having to do that in
some separate way before invoking the script.
------------------------------
Date: Wed, 07 Mar 2012 06:15:53 -0500
From: Shmuel (Seymour J.) Metz <spamtrap@library.lspace.org.invalid>
Subject: Re: little quiz
Message-Id: <4f574369$15$fuzhry+tra$mr2ice@news.patriot.net>
In <4f56a849$0$1383$4fafbaef@reader2.news.tin.it>, on 03/07/2012
at 01:14 AM, Kiuhnm <kiuhnm03.4t.yahoo.it> said:
>It goes without saying that I was talking of Perl.
No. You were responding to Ben Morrow's "Having global subs is a
feature, not a bug. Having *only* global subs is a bug; AIUI Perl 6
has proper lexically-scoped named subs." If you were talking of Perl
then you were engaging in circular reasoning; it's not a bug because
that's how it is. "But named sub are always global." was not
responsive to his comment if you were talking only about Perl.
--
Shmuel (Seymour J.) Metz, SysProg and JOAT <http://patriot.net/~shmuel>
Unsolicited bulk E-mail subject to legal action. I reserve the
right to publicly post or ridicule any abusive E-mail. Reply to
domain Patriot dot net user shmuel+news to contact me. Do not
reply to spamtrap@library.lspace.org
------------------------------
Date: Wed, 07 Mar 2012 22:11:31 +0100
From: "Dr.Ruud" <rvtol+usenet@xs4all.nl>
Subject: Re: references (4567890)
Message-Id: <4f57cf03$0$6915$e4fe514c@news2.news.xs4all.nl>
On 2012-03-06 18:49, Tim McDaniel wrote:
> Perl doesn't have matrices per se.
On the CPAN there is PDL::Matrix, Math::Cephes::Matrix.
> Also, I don't necessarily get to choose the data structures. In this
> case, it was a SQL interface that returned a ref to an array, where
> each array element was a ref to a hash of the form COLUMNNAME =>
> VALUE. If I want some other representation, then I have to haul out
> map, foreach, or whatever to restructure it.
Probably DBI's $dbh->selectall_arrayref( $sql, { Slice => {} } ).
A while-loop on $sth->fetchrow_arrayref is often more effective,
consider $sth->bind_columns( \( @row{ @{ $sth->{NAME} } } ) );
--
Ruud
------------------------------
Date: 6 Apr 2001 21:33:47 GMT (Last modified)
From: Perl-Users-Request@ruby.oce.orst.edu (Perl-Users-Digest Admin)
Subject: Digest Administrivia (Last modified: 6 Apr 01)
Message-Id: <null>
Administrivia:
To submit articles to comp.lang.perl.announce, send your article to
clpa@perl.com.
Back issues are available via anonymous ftp from
ftp://cil-www.oce.orst.edu/pub/perl/old-digests.
#For other requests pertaining to the digest, send mail to
#perl-users-request@ruby.oce.orst.edu. Do not waste your time or mine
#sending perl questions to the -request address, I don't have time to
#answer them even if I did know the answer.
------------------------------
End of Perl-Users Digest V11 Issue 3633
***************************************
