[26266] in Perl-Users-Digest
Perl-Users Digest, Issue: 8449 Volume: 10
daemon@ATHENA.MIT.EDU (Perl-Users Digest)
Fri Sep 23 21:05:42 2005
Date: Fri, 23 Sep 2005 18:05:04 -0700 (PDT)
From: Perl-Users Digest <Perl-Users-Request@ruby.OCE.ORST.EDU>
To: Perl-Users@ruby.OCE.ORST.EDU (Perl-Users Digest)
Perl-Users Digest Fri, 23 Sep 2005 Volume: 10 Number: 8449
Today's topics:
Re: new how-to book about tit-fucking <misterlister169@gmail.com>
Re: new how-to book about tit-fucking <rxdxv@talk21.com>
Re: regex, number of matches <john@castleamber.com>
Re: regex, number of matches <someone@example.com>
Re: regex, number of matches <rvtol+news@isolution.nl>
Re: regex, number of matches <rvtol+news@isolution.nl>
Re: regex, number of matches <abigail@abigail.nl>
Re: regex, number of matches <john@castleamber.com>
Re: regex, number of matches <rvtol+news@isolution.nl>
Re: regex, number of matches <john@castleamber.com>
Re: regex, number of matches <rvtol+news@isolution.nl>
Re: regex, number of matches <rvtol+news@isolution.nl>
Re: regex, number of matches <rvtol+news@isolution.nl>
Re: regex, number of matches <abigail@abigail.nl>
Re: regex, number of matches <abigail@abigail.nl>
Re: Script for migrating HTML tree into a single direct <jgibson@mail.arc.nasa.gov>
Digest Administrivia (Last modified: 6 Apr 01) (Perl-Users-Digest Admin)
----------------------------------------------------------------------
Date: Fri, 23 Sep 2005 23:31:01 +0100
From: Lister <misterlister169@gmail.com>
Subject: Re: new how-to book about tit-fucking
Message-Id: <5e09j1lsirldoh4noo65olgurpepbtatpl@4ax.com>
On Fri, 27 May 2005 21:50:46 +0200 (CEST), "Agent 69"
<69-no-spam@69.69.69.69.invalid> wrote:
Titfucking for dummies?
--
.sig for rent
Apply within
------------------------------
Date: 23 Sep 2005 19:33:02 -0400
From: Robert de Vincy <rxdxv@talk21.com>
Subject: Re: new how-to book about tit-fucking
Message-Id: <Xns96DB61F32EB8rdv@216.128.74.6>
Lister did write:
> From: Lister <misterlister169@gmail.com>
I refuse to believe there are at least 168 other "misterlister"
gmail addresses.
--
BdeV
"I really should change my sig. It's out of date now."
-- Chris Young, <3C35B341.MD-1.4.16.unsatisfactory@bigfoot.com>
------------------------------
Date: 23 Sep 2005 18:06:47 GMT
From: John Bokma <john@castleamber.com>
Subject: Re: regex, number of matches
Message-Id: <Xns96DA854A97EE9castleamber@130.133.1.4>
"Dr.Ruud" <rvtol+news@isolution.nl> wrote:
> What I found hard to get, is the role of the '()' in the wc-words-line:
>
> $ret[1] += () = /\S+/g; # words
it forces list context :-)
> After a while, I understood it as an anonymous array that is filled with
> the matches, after which its length is used to increase the words-count.
>
> The creating and filling of () seemed like a waste of cpu-cycles,
Maybe it's optimized away?
> so I
> tried to find another way of counting the number of matches.
>
> Destructive variant:
>
> $ret[1] += s/\S+//g; # words
>
> I settled for:
>
> $ret[1] += 1 while /\S+/g; # words
Did you benchmark those?
--
John Small Perl scripts: http://johnbokma.com/perl/
Perl programmer available: http://castleamber.com/
Happy Customers: http://castleamber.com/testimonials.html
------------------------------
Date: Fri, 23 Sep 2005 19:04:19 GMT
From: "John W. Krahn" <someone@example.com>
Subject: Re: regex, number of matches
Message-Id: <TkYYe.189655$wr.164665@clgrps12>
Dr.Ruud wrote:
>
> What I found hard to get, is the role of the '()' in the wc-words-line:
>
> $ret[1] += () = /\S+/g; # words
>
> After a while, I understood it as an anonymous array that is filled with
> the matches, after which its length is used to increase the words-count.
If it had been an anonymous array it would have been:
$ret[1] += @{[ /\S+/g ]}; # words
John
--
use Perl;
program
fulfillment
------------------------------
Date: Fri, 23 Sep 2005 22:23:01 +0200
From: "Dr.Ruud" <rvtol+news@isolution.nl>
Subject: Re: regex, number of matches
Message-Id: <dh1va8.1mk.1@news.isolution.nl>
John Bokma schreef:
> Dr.Ruud:
>> What I found hard to get, is the role of the '()' in
>> $ret[1] += () = /\S+/g; # words
>
> it forces list context :-)
Yes, I'm starting to get that.
>> The creating and filling of () seemed like a waste of cpu-cycles,
>
> Maybe it's optimized away?
Well, maybe compare it to:
$ret[1] += (@tmp = /\S+/g); # words
but if tmp is not used afterwards, that also can be optimized.
>> Destructive variant:
>>
>> $ret[1] += s/\S+//g; # words
>>
>> I settled for:
>>
>> $ret[1] += 1 while /\S+/g; # words
>
>
> Did you benchmark those?
No, I'm not in a hurry (yet).
A new option for scalar mode would be the cleanest:
$ret[1] += /\S+/t; # words
--
Affijn, Ruud
"Gewoon is een tijger."
------------------------------
Date: Fri, 23 Sep 2005 22:31:31 +0200
From: "Dr.Ruud" <rvtol+news@isolution.nl>
Subject: Re: regex, number of matches
Message-Id: <dh1vov.1mk.1@news.isolution.nl>
Brian Wakem:
> The fastest way is to substitute a match with itself.
OK. I had tested that, but I hated the looks, because it doesn't explain
itself enough.
The notion that the 'fake substitution' operates 'at the C level' is
very convincing.
Did you also benchmark "s!\Q$kw\E!$&!g" ?
(You pay a price using &, see man perlre: $& is not so costly.)
--
Affijn, Ruud
"Gewoon is een tijger."
------------------------------
Date: 23 Sep 2005 21:50:47 GMT
From: Abigail <abigail@abigail.nl>
Subject: Re: regex, number of matches
Message-Id: <slrndj8u5n.8d9.abigail@alexandra.abigail.nl>
John Bokma (john@castleamber.com) wrote on MMMMCDVI September MCMXCIII in
<URL:news:Xns96DA854A97EE9castleamber@130.133.1.4>:
|| "Dr.Ruud" <rvtol+news@isolution.nl> wrote:
||
|| > What I found hard to get, is the role of the '()' in the wc-words-line:
|| >
|| > $ret[1] += () = /\S+/g; # words
||
|| it forces list context :-)
||
|| > After a while, I understood it as an anonymous array that is filled with
|| > the matches, after which its length is used to increase the words-count.
|| >
|| > The creating and filling of () seemed like a waste of cpu-cycles,
||
|| Maybe it's optimized away?
Yeah, one would hope. The benchmark below suggests it isn't.
||
|| > so I
|| > tried to find another way of counting the number of matches.
|| >
|| > Destructive variant:
|| >
|| > $ret[1] += s/\S+//g; # words
|| >
|| > I settled for:
|| >
|| > $ret[1] += 1 while /\S+/g; # words
||
||
|| Did you benchmark those?
I did. I was surprised the while variant was the clear winner - assigning
to an empty list is hardly faster than assigning to an array.
The results:
Rate sub list2 list while
sub 2437/s -- -26% -34% -62%
list2 3288/s 35% -- -11% -49%
list 3690/s 51% 12% -- -43%
while 6457/s 165% 96% 75% --
The above results are for perl5.8.7. For 5.8.0 till 5.8.6, the substitution
is faster, and only slightly slower than any of the list context solutions.
Results from 5.8.0:
Rate sub list2 list while
sub 3258/s -- -4% -5% -52%
list2 3381/s 4% -- -1% -50%
list 3429/s 5% 1% -- -50%
while 6826/s 109% 102% 99% --
5.6.0 is actually faster than 5.8.x. Here the substitution solution is
again slower than any of the list context solutions, but the substitution
is actually faster in 5.6.0 than any of the list context solutions is in
5.8.0:
Rate sub list2 list while
sub 3864/s -- -15% -18% -52%
list2 4525/s 17% -- -4% -43%
list 4696/s 22% 4% -- -41%
while 8000/s 107% 77% 70% --
And the code:
#!/usr/bin/perl
use strict;
use warnings;
no warnings 'syntax';
use Benchmark qw 'cmpthese';
our @data = <DATA>;
our ($list, $list2, $while, $sub);
cmpthese -1 => {
list => '$list = 0; $list += () = /\S+/g for @data;',
list2 => '$list2 = 0; $list2 += (my @t = /\S+/g) for @data;',
while => '$while = 0; do {$while ++ while /\S+/g} for @data;',
sub => '$sub = 0; $sub += s/(\S+)/$1/g for @data;',
};
die "\$list = $list; \$list2 = $list2; \$while = $while; \$sub = $sub\n"
unless $list == $list2 && $list == $while && $list == $sub;
__DATA__
"What happens next?" asked Twoflower.
Hrun screwed a finger in his ear and inspected it absently.
"Oh,", he said, "I expect in a minute the door will be
flung back and I'll be dragged off to some sort of temple
arena where I'll fight maybe a couple of giant spiders
and an eight-foot slave from the jungles of Klatch and then
I'll rescue some kind of a princess from the altar and then
I'll kill off a few guards or whatever and then this girl
will show me the secret passage out of the place and we'll
liberate a couple of horses and escape with the treasure."
Hrun leaned his head back on his hands and looked at the
ceiling, whistling tunelessly.
"All that?" said Twoflower.
"Usually."
--
my $qr = qr/^.+?(;).+?\1|;Just another Perl Hacker;|;.+$/;
$qr =~ s/$qr//g;
print $qr, "\n";
------------------------------
Date: 23 Sep 2005 22:00:33 GMT
From: John Bokma <john@castleamber.com>
Subject: Re: regex, number of matches
Message-Id: <Xns96DAACEC1B6BDcastleamber@130.133.1.4>
"Dr.Ruud" <rvtol+news@isolution.nl> wrote:
> John Bokma schreef:
>> Did you benchmark those?
>
> No, I'm not in a hurry (yet).
:-) Problem with benchmarking is what today seems to be a bad choice, can
be a better choice tomorrow. An example: map in a void context was some
time ago an expensive operation. IIRC it has been optimized (note that I
don't say that "we" should all use map in a void context now :-) )
> A new option for scalar mode would be the cleanest:
>
> $ret[1] += /\S+/t; # words
and t means? (Tellen :-D). The questions are: how often is this going to be
used, and; is a new option really needed, or can we get away with what's
available now and some documentation?
--
John Small Perl scripts: http://johnbokma.com/perl/
Perl programmer available: http://castleamber.com/
Happy Customers: http://castleamber.com/testimonials.html
------------------------------
Date: Sat, 24 Sep 2005 01:23:48 +0200
From: "Dr.Ruud" <rvtol+news@isolution.nl>
Subject: Re: regex, number of matches
Message-Id: <dh2a7t.14c.1@news.isolution.nl>
Abigail:
> I was surprised the while variant was the clear winner -
> assigning to an empty list is hardly faster than assigning to an
> array.
That is what I expected.
> And the code:
Thanks for that too.
How about:
sub2 => '$sub2 = 0; $sub2 += s/\S+/$&/g for @data;'
And how about the o-option (pre-compile), or doesn't that go with g?
--
Affijn, Ruud
"Gewoon is een tijger."
------------------------------
Date: 23 Sep 2005 23:49:17 GMT
From: John Bokma <john@castleamber.com>
Subject: Re: regex, number of matches
Message-Id: <Xns96DABF5B53AFcastleamber@130.133.1.4>
"Dr.Ruud" <rvtol+news@isolution.nl> wrote:
> sub2 => '$sub2 = 0; $sub2 += s/\S+/$&/g for @data;'
>
> And how about the o-option (pre-compile), or doesn't that go with g?
o is IIRC only useful in some rare cases when you use a variabele in the
regexp. Since your s/// is constant, I think it's compiled, optimized, etc.
at the compile stage of your script, but again IIRC.
--
John Small Perl scripts: http://johnbokma.com/perl/
Perl programmer available: http://castleamber.com/
Happy Customers: http://castleamber.com/testimonials.html
------------------------------
Date: Sat, 24 Sep 2005 01:45:59 +0200
From: "Dr.Ruud" <rvtol+news@isolution.nl>
Subject: Re: regex, number of matches
Message-Id: <dh2b5i.1ec.1@news.isolution.nl>
Brian Wakem:
> Dr.Ruud:
>
>
>> $ret[1] += () = /\S+/g; # words
>>
>>
>> Destructive variant:
>>
>> $ret[1] += s/\S+//g; # words
>>
>> I settled for:
>>
>> $ret[1] += 1 while /\S+/g; # words
>>
>> Is there a better/nicer/smarter/directer way to return the number of
>> matches from a regex?
>
>
> This was covered a few week ago in a thread titled 'Space (\s) count
> problem'.
>
> The fastest way is to substitute a match with itself.
As Abigail showed, there will be a difference between
(1) s/$kw/$kw/g (add \Q and \E where needed)
and
(2) s/\S+/$&/g
and
(3) s/\S+//g
The loops of both (1) and (3) are more 'constant' so will need less
cycles than (2).
(is my guess)
--
Affijn, Ruud
"Gewoon is een tijger."
------------------------------
Date: Sat, 24 Sep 2005 01:52:57 +0200
From: "Dr.Ruud" <rvtol+news@isolution.nl>
Subject: Re: regex, number of matches
Message-Id: <dh2bic.cs.1@news.isolution.nl>
John Bokma:
> Dr.Ruud:
>> A new option for scalar mode would be the cleanest:
>>
>> $ret[1] += /\S+/t; # words
>
> and t means? (Tellen :-D).
>>> See also http://dev.perl.org/perl6/rfc/110.html
s/t/=/ if parseable
--
Affijn, Ruud
"Gewoon is een tijger."
------------------------------
Date: Sat, 24 Sep 2005 02:01:46 +0200
From: "Dr.Ruud" <rvtol+news@isolution.nl>
Subject: Re: regex, number of matches
Message-Id: <dh2c40.1ak.1@news.isolution.nl>
Abigail schreef:
> [benchmark]
> And the code:
There is a problem with the benchmark, because perlre(1) says:
WARNING: Once Perl sees that you need one of $&, $`, or $' anywhere in
the program, it has to provide them for every pattern match. This may
substantially slow your program. Perl uses the same mechanism to pro-
duce $1, $2, etc, so you also pay a price for each pattern that con-
tains capturing parentheses. (To avoid this cost while retaining the
grouping behaviour, use the extended regular expression "(?: ... )"
instead.) But if you never use $&, $` or $', then patterns without
capturing parentheses will not be penalized. So avoid $&, $', and $`
if you can, but if you can't (and some algorithms really appreciate
them), once you've used them once, use them at will, because you've
already paid the price. As of 5.005, $& is not so costly as the other
two.
--
Affijn, Ruud
"Gewoon is een tijger."
------------------------------
Date: 24 Sep 2005 01:02:09 GMT
From: Abigail <abigail@abigail.nl>
Subject: Re: regex, number of matches
Message-Id: <slrndj99ch.8d9.abigail@alexandra.abigail.nl>
Dr.Ruud (rvtol+news@isolution.nl) wrote on MMMMCDVI September MCMXCIII in
<URL:news:dh2a7t.14c.1@news.isolution.nl>:
?? Abigail:
??
?? > I was surprised the while variant was the clear winner -
?? > assigning to an empty list is hardly faster than assigning to an
?? > array.
??
?? That is what I expected.
??
??
?? > And the code:
??
?? Thanks for that too.
??
?? How about:
??
?? sub2 => '$sub2 = 0; $sub2 += s/\S+/$&/g for @data;'
??
?? And how about the o-option (pre-compile), or doesn't that go with g?
/o doesn't make any difference, as your regexp is doesn't change:
Rate sub sub_o list2_o list2 list list_o while_o while
sub 2461/s -- -0% -26% -28% -29% -29% -62% -62%
sub_o 2466/s 0% -- -26% -27% -29% -29% -62% -62%
list2_o 3338/s 36% 35% -- -2% -4% -4% -48% -49%
list2 3398/s 38% 38% 2% -- -3% -3% -47% -48%
list 3490/s 42% 42% 5% 3% -- 0% -46% -46%
list_o 3490/s 42% 42% 5% 3% 0% -- -46% -46%
while_o 6458/s 162% 162% 93% 90% 85% 85% -- -1%
while 6516/s 165% 164% 95% 92% 87% 87% 1% --
which is the same test I did before, except that the _o cases use /o in
their regexes.
Here are the results with your 'sub2' case:
Rate sub sub2 list2 list while
sub 2465/s -- -6% -28% -29% -45%
sub2 2619/s 6% -- -24% -25% -41%
list2 3446/s 40% 32% -- -1% -22%
list 3479/s 41% 33% 1% -- -22%
while 4443/s 80% 70% 29% 28% --
Note that the cases that are run after 'sub2' will be skewed - now that
the compiler has seen $& any subsequent regex will carry a penalty. Judging
from the numbers above, 'while' was run after 'sub2', and has a much lower
run rate than in earlier benchmarks. But despite that, the 'while' solution
is still faster.
Abigail
--
perl -MLWP::UserAgent -MHTML::TreeBuilder -MHTML::FormatText -wle'print +(
HTML::FormatText -> new -> format (HTML::TreeBuilder -> new -> parse (
LWP::UserAgent -> new -> request (HTTP::Request -> new ("GET",
"http://work.ucsd.edu:5141/cgi-bin/http_webster?isindex=perl")) -> content))
=~ /(.*\))[-\s]+Addition/s) [0]'
------------------------------
Date: 24 Sep 2005 01:03:59 GMT
From: Abigail <abigail@abigail.nl>
Subject: Re: regex, number of matches
Message-Id: <slrndj99fv.8d9.abigail@alexandra.abigail.nl>
Dr.Ruud (rvtol+news@isolution.nl) wrote on MMMMCDVII September MCMXCIII
in <URL:news:dh2c40.1ak.1@news.isolution.nl>:
__ Abigail schreef:
__
__ > [benchmark]
__ > And the code:
__
__ There is a problem with the benchmark, because perlre(1) says:
__
__ WARNING: Once Perl sees that you need one of $&, $`, or $' anywhere in
__ the program, it has to provide them for every pattern match. This may
__ substantially slow your program. Perl uses the same mechanism to pro-
__ duce $1, $2, etc, so you also pay a price for each pattern that con-
__ tains capturing parentheses. (To avoid this cost while retaining the
__ grouping behaviour, use the extended regular expression "(?: ... )"
__ instead.) But if you never use $&, $` or $', then patterns without
__ capturing parentheses will not be penalized. So avoid $&, $', and $`
__ if you can, but if you can't (and some algorithms really appreciate
__ them), once you've used them once, use them at will, because you've
__ already paid the price. As of 5.005, $& is not so costly as the other
__ two.
I'm fully aware of the penalty associated with $& and friends.
But why does that cause a problem with the benchmark?
Abigail
--
perl -wle 'print "Prime" if (1 x shift) !~ /^1?$|^(11+?)\1+$/'
------------------------------
Date: Fri, 23 Sep 2005 11:10:46 -0700
From: Jim Gibson <jgibson@mail.arc.nasa.gov>
Subject: Re: Script for migrating HTML tree into a single directory ?
Message-Id: <230920051110467923%jgibson@mail.arc.nasa.gov>
In article <1127464898.c7410a3d1121836799ad50eab6e90102@teranews>, Pan
Am <panam@nospam.com> wrote:
> My Web hosting service does not support multiple directories...
> Can anyone suggest a Unix script that traverses a HTML tree and
> produces a working "single directory" version of the same?
The File::Find module will traverse a directory tree and return the
names of all of the files found in that directory.
Posted Via Usenet.com Premium Usenet Newsgroup Services
----------------------------------------------------------
** SPEED ** RETENTION ** COMPLETION ** ANONYMITY **
----------------------------------------------------------
http://www.usenet.com
------------------------------
Date: 6 Apr 2001 21:33:47 GMT (Last modified)
From: Perl-Users-Request@ruby.oce.orst.edu (Perl-Users-Digest Admin)
Subject: Digest Administrivia (Last modified: 6 Apr 01)
Message-Id: <null>
Administrivia:
#The Perl-Users Digest is a retransmission of the USENET newsgroup
#comp.lang.perl.misc. For subscription or unsubscription requests, send
#the single line:
#
# subscribe perl-users
#or:
# unsubscribe perl-users
#
#to almanac@ruby.oce.orst.edu.
NOTE: due to the current flood of worm email banging on ruby, the smtp
server on ruby has been shut off until further notice.
To submit articles to comp.lang.perl.announce, send your article to
clpa@perl.com.
#To request back copies (available for a week or so), send your request
#to almanac@ruby.oce.orst.edu with the command "send perl-users x.y",
#where x is the volume number and y is the issue number.
#For other requests pertaining to the digest, send mail to
#perl-users-request@ruby.oce.orst.edu. Do not waste your time or mine
#sending perl questions to the -request address, I don't have time to
#answer them even if I did know the answer.
------------------------------
End of Perl-Users Digest V10 Issue 8449
***************************************
