[32662] in Perl-Users-Digest
Perl-Users Digest, Issue: 3938 Volume: 11
daemon@ATHENA.MIT.EDU (Perl-Users Digest)
Fri May 3 00:09:25 2013
Date: Thu, 2 May 2013 21:09:05 -0700 (PDT)
From: Perl-Users Digest <Perl-Users-Request@ruby.OCE.ORST.EDU>
To: Perl-Users@ruby.OCE.ORST.EDU (Perl-Users Digest)
Perl-Users Digest Thu, 2 May 2013 Volume: 11 Number: 3938
Today's topics:
Re: nice parallel file reading <uri@stemsystems.com>
Re: nice parallel file reading <jurgenex@hotmail.com>
using $1 in a substitution replacement with variable in imaginarywave@gmail.com
Re: using $1 in a substitution replacement with variabl <peter@makholm.net>
Re: using $1 in a substitution replacement with variabl <derykus@gmail.com>
Re: using $1 in a substitution replacement with variabl <ben@morrow.me.uk>
Re: using $1 in a substitution replacement with variabl <derykus@gmail.com>
Re: using $1 in a substitution replacement with variabl <rweikusat@mssgmbh.com>
Digest Administrivia (Last modified: 6 Apr 01) (Perl-Users-Digest Admin)
----------------------------------------------------------------------
Date: Wed, 01 May 2013 01:04:25 -0400
From: Uri Guttman <uri@stemsystems.com>
Subject: Re: nice parallel file reading
Message-Id: <87hain47fq.fsf@stemsystems.com>
>>>>> "JE" == Jürgen Exner <jurgenex@hotmail.com> writes:
JE> "George Mpouras"
JE> <nospam.gravitalsun.antispam@spamno.hotmail.anispam.com.nospam> wrote:
>> # there was a problem with the code at my initial post
>> # Here is corrected, of how to read files like round-robin
>> # using an iterator
JE> While this might be mildly interesting as an academic exercise I wonder
JE> if there is any actual non-contrived application where you would have to
JE> read multiple files synchronously line-by-line and at the same time the
JE> files are too large to just load them into a variable and then process
JE> their content.
not as true today but merge sorting did this very thing in the olden
days. there are probably some similar problems today.
uri
------------------------------
Date: Wed, 01 May 2013 07:10:45 -0700
From: Jürgen Exner <jurgenex@hotmail.com>
Subject: Re: nice parallel file reading
Message-Id: <mb82o81r9lkd69p0jdtrk3i8vm33lcbecm@4ax.com>
Uri Guttman <uri@stemsystems.com> wrote:
>>>>>> "JE" == Jürgen Exner <jurgenex@hotmail.com> writes:
>
> JE> "George Mpouras"
> JE> <nospam.gravitalsun.antispam@spamno.hotmail.anispam.com.nospam> wrote:
> >> # there was a problem with the code at my initial post
> >> # Here is corrected, of how to read files like round-robin
> >> # using an iterator
>
> JE> While this might be mildly interesting as an academic exercise I wonder
> JE> if there is any actual non-contrived application where you would have to
> JE> read multiple files synchronously line-by-line and at the same time the
> JE> files are too large to just load them into a variable and then process
> JE> their content.
>
>not as true today but merge sorting did this very thing in the olden
>days. there are probably some similar problems today.
As I mentioned in a differen message merge sort does not read
_synchronously_, i.e. round robin, from the files but for each
line/value it depends upon which file currently has the lowest
line/value and this can very well be the same file again and again for
many lines/values.
jue
------------------------------
Date: Wed, 1 May 2013 12:12:06 -0700 (PDT)
From: imaginarywave@gmail.com
Subject: using $1 in a substitution replacement with variable interpolation
Message-Id: <34dc5921-af61-4bf6-9778-97984b47c73a@googlegroups.com>
I am trying to use variable interpolation in a replacement string including $1, $2,...
However, I can't get it to expand $1 into the replacement. I eventually will have the
$pattern and $replacement variables be read from a configuration file, but even
setting them manually doesn't work.
In the example script, you can see that the $1 (which should be 'DEF') is not
expanded in $new_name, but it is in $new_name2.
Adding an 'e' flag to the substitution doesn't help.
How do I fix this?
Matt
EXAMPLE CODE:
#!/usr/local/bin/perl
use strict;
my $old_name = 'ABC_DEF_GHI';
my $pattern = 'ABC_(...)_GHI';
my $replacement = 'CBA_${1}_IHG';
# using variables - doesn't work
my $new_name = $old_name;
$new_name =~ s|$pattern|$replacement|;
printf("%s --> %s\n", $old_name, $new_name);
# not using variables - does work
my $new_name2 = $old_name;
$new_name2 =~ s|ABC_(...)_GHI|CBA_${1}_IHG|;
printf("%s --> %s\n", $old_name, $new_name2);
OUTPUT:
ABC_DEF_GHI --> CBA_${1}_IHG
ABC_DEF_GHI --> CBA_DEF_IHG
------------------------------
Date: Wed, 01 May 2013 21:32:38 +0200
From: Peter Makholm <peter@makholm.net>
Subject: Re: using $1 in a substitution replacement with variable interpolation
Message-Id: <87vc72a42x.fsf@vps1.hacking.dk>
imaginarywave@gmail.com writes:
> my $pattern = 'ABC_(...)_GHI';
> my $replacement = 'CBA_${1}_IHG';
>
> # using variables - doesn't work
> my $new_name = $old_name;
> $new_name =~ s|$pattern|$replacement|;
It works to use s/$pattern/qq("$replacement")/ee.
First we do regular interpolation. With /e the replacement is trated as
perl code and therefore not interpolated:
-> s/ABC_(...)_GHI/qq("$replacement")/ee
Given the match we execute once:
-> s/ABC_(...)_GHI/"CBA_${1}_IHG"/e
And once more:
-> s/ABC_(...)_GHI/CBA_DEF_IHG/
Reaching the wanted result.
//Makholm
------------------------------
Date: Wed, 01 May 2013 16:30:57 -0700
From: Charles DeRykus <derykus@gmail.com>
Subject: Re: using $1 in a substitution replacement with variable interpolation
Message-Id: <kls8js$3dl$1@speranza.aioe.org>
On 5/1/2013 12:12 PM, imaginarywave@gmail.com wrote:
> I am trying to use variable interpolation in a replacement string including $1, $2,...
> However, I can't get it to expand $1 into the replacement. I eventually will have the
> $pattern and $replacement variables be read from a configuration file, but even
> setting them manually doesn't work.
>
> In the example script, you can see that the $1 (which should be 'DEF') is not
> expanded in $new_name, but it is in $new_name2.
>
> Adding an 'e' flag to the substitution doesn't help.
>
> ...
>
> #!/usr/local/bin/perl
> use strict;
>
> my $old_name = 'ABC_DEF_GHI';
>
> my $pattern = 'ABC_(...)_GHI';
> my $replacement = 'CBA_${1}_IHG';
>
> # using variables - doesn't work
> my $new_name = $old_name;
> $new_name =~ s|$pattern|$replacement|;
>
> printf("%s --> %s\n", $old_name, $new_name);
>
Perhaps, depending on input string complexity, you could just do this
and avoid a string eval:
(my $new_name=$old_name) = "${^PREMATCH}CBA_${1}_IHG${^POSTMATCH}"
if $old_name =~ m/ABC_(...)_GHI/p;
--
Charles DeRykus
------------------------------
Date: Thu, 2 May 2013 00:27:48 +0100
From: Ben Morrow <ben@morrow.me.uk>
Subject: Re: using $1 in a substitution replacement with variable interpolation
Message-Id: <k0l85a-bbh1.ln1@anubis.morrow.me.uk>
Quoth Peter Makholm <peter@makholm.net>:
> imaginarywave@gmail.com writes:
>
> > my $pattern = 'ABC_(...)_GHI';
> > my $replacement = 'CBA_${1}_IHG';
> >
> > # using variables - doesn't work
> > my $new_name = $old_name;
> > $new_name =~ s|$pattern|$replacement|;
>
> It works to use s/$pattern/qq("$replacement")/ee.
You do realise you just called 'eval' on user-supplied input, don't you?
This is extremely unsafe.
This is basically the same problem as the FAQ 'How can I expand
variables in text strings?', but it's made a bit more complicated by the
substitution. What you need to do is 'unwrap' the substitution, so you
can get in between the 'find' and 'replace' parts.
A normal substitution, $var =~ s/$pat/$rep/, is equivalent to this:
if ($var =~ /$pat/) {
substr $var, $-[0], $+[0] - $-[0], $rep;
}
To see why this is so, you need to read about @- and @+ in perlvar; you
also need to know what 4-arg substr does. If the substitution was an
s///g, this becomes a bit more complicated:
pos($var) = 0;
while ($var =~ /$pat/g) {
substr $var, $-[0], $+[0] - $-[0], $rep;
pos($var) = $-[0] + length($rep);
}
because we have to make sure the //g keeps looking at new bits of the
string. (This is what 'pos' does.)
So, now that we know this, we can apply the FAQ answer between 'find'
and 'replace'. This is made slightly more complicated by the fact that
@- and @+ are globals, so we need to copy them before we do any more
pattern matches.
my $var = "foo A fooo B fo";
my $pat = "f(o*)";
my $rep = 'X$1Y';
# begin at the beginning
pos($var) = 0;
while ($var =~ /$pat/g) {
# copy the globals, since we're about to do another match
my @beg = @-;
my @len = map $+[$_] - $-[$_], 0..$#-;
# construct the substituted string
(my $sub = $rep) =~
# you can allow more variables here if you want to
s/\$([1-9][0-9]?)/substr $var, $beg[$1], $len[$1]/ge;
# Note that the replacement above is with *one* /e, which is
# safe. You need two, that is, /ee, to invoke string eval.
# do the replacement
substr $var, $beg[0], $len[0], $sub;
# reset pos to carry on
pos($var) = $beg[0] + length($sub);
}
(This is the /g case, because that's more interesting.)
I think you could probably also do this with an s///ge which either had a
*really* complicated replacement part or which called a sub to build the
replacement string, though tbh I'm not entirely sure what happens if you
change the string you're working on during an s///ge.
Ben
------------------------------
Date: Wed, 01 May 2013 16:35:39 -0700
From: Charles DeRykus <derykus@gmail.com>
Subject: Re: using $1 in a substitution replacement with variable interpolation
Message-Id: <kls8sl$3dl$2@speranza.aioe.org>
On 5/1/2013 4:30 PM, Charles DeRykus wrote:
>> ...
>>
>
> Perhaps, depending on input string complexity, you could just do this
> and avoid a string eval:
>
> (my $new_name=$old_name) = "${^PREMATCH}CBA_${1}_IHG${^POSTMATCH}"
> if $old_name =~ m/ABC_(...)_GHI/p;
>
Or, simply:
my $new_name = "${^PREMATCH}CBA_${1}_IHG${^POSTMATCH}"
if $old_name =~ m/ABC_(...)_GHI/p;
--
Charles DeRykus
------------------------------
Date: Thu, 02 May 2013 12:23:59 +0100
From: Rainer Weikusat <rweikusat@mssgmbh.com>
Subject: Re: using $1 in a substitution replacement with variable interpolation
Message-Id: <8738u5hbg0.fsf@sapphire.mobileactivedefense.com>
imaginarywave@gmail.com writes:
[...]
> #!/usr/local/bin/perl
> use strict;
>
> my $old_name = 'ABC_DEF_GHI';
>
> my $pattern = 'ABC_(...)_GHI';
> my $replacement = 'CBA_${1}_IHG';
>
> # using variables - doesn't work
> my $new_name = $old_name;
> $new_name =~ s|$pattern|$replacement|;
>
> printf("%s --> %s\n", $old_name, $new_name);
Using s/// in this way is a little weird. What you are really doing is
to apply a 'pattern' regex to some string in order to extract certain
parts out of it and then, you interpolate these parts into a template
string. I would use something like this for that:
-----------------
#!/usr/local/bin/perl
use strict;
sub interpolate
{
my ($in, $pat, $out) = @_;
my @matches;
@matches = $in =~ /$pat/;
$out =~ s/\$\{([0-9]+)\}/$matches[$1 - 1]/g;
return $out;
}
my $old_name = 'ABC_DEF_GHI';
my $pattern = 'ABC_(...)_(...)';
my $template = '${2}_${1}_IHG';
my $new_name = interpolate($old_name, $pattern, $template);
printf("%s --> %s\n", $old_name, $new_name);
------------------------------
Date: 6 Apr 2001 21:33:47 GMT (Last modified)
From: Perl-Users-Request@ruby.oce.orst.edu (Perl-Users-Digest Admin)
Subject: Digest Administrivia (Last modified: 6 Apr 01)
Message-Id: <null>
Administrivia:
To submit articles to comp.lang.perl.announce, send your article to
clpa@perl.com.
Back issues are available via anonymous ftp from
ftp://cil-www.oce.orst.edu/pub/perl/old-digests.
#For other requests pertaining to the digest, send mail to
#perl-users-request@ruby.oce.orst.edu. Do not waste your time or mine
#sending perl questions to the -request address, I don't have time to
#answer them even if I did know the answer.
------------------------------
End of Perl-Users Digest V11 Issue 3938
***************************************
