[26267] in Perl-Users-Digest
Perl-Users Digest, Issue: 8450 Volume: 10
daemon@ATHENA.MIT.EDU (Perl-Users Digest)
Sat Sep 24 06:05:35 2005
Date: Sat, 24 Sep 2005 03:05:05 -0700 (PDT)
From: Perl-Users Digest <Perl-Users-Request@ruby.OCE.ORST.EDU>
To: Perl-Users@ruby.OCE.ORST.EDU (Perl-Users Digest)
Perl-Users Digest Sat, 24 Sep 2005 Volume: 10 Number: 8450
Today's topics:
Re: regex, number of matches <abigail@abigail.nl>
Re: regex, number of matches <rvtol+news@isolution.nl>
Re: regex, number of matches <rvtol+news@isolution.nl>
Website scraper <dvh@dvhdvhdvh.dvh>
Re: Website scraper <steve@uptime.org.uk>
Digest Administrivia (Last modified: 6 Apr 01) (Perl-Users-Digest Admin)
----------------------------------------------------------------------
Date: 24 Sep 2005 01:06:21 GMT
From: Abigail <abigail@abigail.nl>
Subject: Re: regex, number of matches
Message-Id: <slrndj99kd.8d9.abigail@alexandra.abigail.nl>
Dr.Ruud (rvtol+news@isolution.nl) wrote on MMMMCDVI September MCMXCIII in
<URL:news:dh2b5i.1ec.1@news.isolution.nl>:
{}
{} As Abigail showed, there will be a difference between
{}
{} (1) s/$kw/$kw/g (add \Q and \E where needed)
{}
{} and
{}
{} (2) s/\S+/$&/g
{}
{} and
{}
{} (3) s/\S+//g
{}
{}
{} The loops of both (1) and (3) are more 'constant' so will need less
{} cycles than (2).
I did not show that, and I do not know what you mean by (1) and (3) being
"more constant" and hence needing less cycles.
Abigail
--
INIT {print "Perl " }
END {print "Hacker\n"}
CHECK {print "another "}
BEGIN {print "Just " }
------------------------------
Date: Sat, 24 Sep 2005 10:49:34 +0200
From: "Dr.Ruud" <rvtol+news@isolution.nl>
Subject: Re: regex, number of matches
Message-Id: <dh3b5a.gg.1@news.isolution.nl>
Abigail schreef:
> Dr.Ruud:
> {} As Abigail showed, there will be a difference between
> {}
> {} (1) s/$kw/$kw/g (add \Q and \E where needed)
> {}
> {} and
> {}
> {} (2) s/\S+/$&/g
> {}
> {} and
> {}
> {} (3) s/\S+//g
> {}
> {}
> {} The loops of both (1) and (3) are more 'constant' so will need
> less {} cycles than (2).
>
>
> I did not show that,
You're right. I first had 2 cases in there, and than added one and
changed another and left the top line as is.
> and I do not know what you mean by (1) and (3)
> being "more constant" and hence needing less cycles.
How little the regex changes for each iteration.
The "$&" part varies in each iteration, but an optimization for that is
feasible, since the search- and the replace-part can be detected as
equal. If that optimization kicks in with (2), than (1) < (2) < (3) in
"what needs to be done".
Case (3) is very much like (2), only the input gets changed, so that
will be the slowest.
Case (1) deals with a different situation: a keyword-count in stead of a
word-count.
--
Affijn, Ruud
"Gewoon is een tijger."
------------------------------
Date: Sat, 24 Sep 2005 11:14:34 +0200
From: "Dr.Ruud" <rvtol+news@isolution.nl>
Subject: Re: regex, number of matches
Message-Id: <dh3dfb.194.1@news.isolution.nl>
Abigail schreef:
> Dr.Ruud:
>> There is a problem with the benchmark, because [of what] perlre(1)
says
> I'm fully aware of the penalty associated with $& and friends.
> But why does that cause a problem with the benchmark?
My mistake again. I wrongly read the text about the "price for each
pattern that contains capturing parentheses" as also penalizing other
pattern matches.
I find it hard to think of a reason why the first use of $& should harm
all other pattern matches. And then why ()/$1 doesn't. Because they can
be handled in about the same way. I haven't looked into the Perl-source
yet, this is as good a reason as any.
I like to see the benchmark without the ()/$1 test, to see if the
remaining cases behave about the same. That might need absolute scores
next to the percentages, so running on a system that doesn't do much
else. I'll try to arrange that later today or tomorrow.
--
Affijn, Ruud
"Gewoon is een tijger."
------------------------------
Date: Sat, 24 Sep 2005 08:59:29 +0000 (UTC)
From: "DVH" <dvh@dvhdvhdvh.dvh>
Subject: Website scraper
Message-Id: <dh34hg$d0i$1@nwrdmz01.dmz.ncs.ea.ibs-infra.bt.com>
Hi,
I've been working through a Perl tutorial on using HTML::TokeParser, and
trying to adapt the example script it gives.
http://www.perl.com/pub/a/2001/11/15/creatingrss.html
The script is meant to scrape headlines from the BBC website and put them
into an RSS feed. It looks for CSS tags, then extracts the text nearby. I've
modified it because the tags in the example don't match the tags on the site
any more, but the script is still sticking at a certain point.
I *think* it's sticking here:
$headline = $stream->get_trimmed_text('/b') \
if ($tag->[1]{class} =~ /^h[12]$/);
I don't understand what that backslash is doing at the end of the first
line. And I don't see where the loop following the "if" in the second line
actually begins - shouldn't it begin with a curly bracket?
Any advice gratefully received.
DVH.
------------------------------
Date: Sat, 24 Sep 2005 10:41:19 +0100
From: Stephen Hildrey <steve@uptime.org.uk>
Subject: Re: Website scraper
Message-Id: <1127554879.68454.0@dyke.uk.clara.net>
DVH wrote:
> I've been working through a Perl tutorial on using HTML::TokeParser, and
> trying to adapt the example script it gives.
> http://www.perl.com/pub/a/2001/11/15/creatingrss.html
Note: that article was written in 2001. Screen-scrapers are notoriously
fragile - they often break in response to even the slightest change in
the target.
> The script is meant to scrape headlines from the BBC website and put them
> into an RSS feed. It looks for CSS tags, then extracts the text nearby. I've
> modified it because the tags in the example don't match the tags on the site
> any more, but the script is still sticking at a certain point.
As above, IIRC the BBC has changed their news page since 2001.
> I *think* it's sticking here:
>
> $headline = $stream->get_trimmed_text('/b') \
> if ($tag->[1]{class} =~ /^h[12]$/);
>
> I don't understand what that backslash is doing at the end of the first
> line.
I think the author got mixed up between Perl and shell scripting - where
'\' is used to continue across newlines. That line should be:
$headline = $stream->get_trimmed_text('/b')
if ($tag->[1]{class} =~ /^h[12]$/);
> And I don't see where the loop following the "if" in the second line
> actually begins - shouldn't it begin with a curly bracket?
It's an example of Perl's "statement if (cond)" syntax. So, just as you
can say:
if (foo) { bar; }
you can also say:
bar if (foo);
Consequently, the above scraper code is *exactly* the same as:
if ($tag->[1]{class} =~ /^h[12]$/)
{
$headline = $stream->get_trimmed_text('/b');
}
It's just a matter of preference and readability.
HTH,
Steve
--
Stephen Hildrey
E-mail: steve@uptime.org.uk / Tel: +442071931337
Jabber: steve@jabber.earth.li / MSN: foo@hotmail.co.uk
------------------------------
Date: 6 Apr 2001 21:33:47 GMT (Last modified)
From: Perl-Users-Request@ruby.oce.orst.edu (Perl-Users-Digest Admin)
Subject: Digest Administrivia (Last modified: 6 Apr 01)
Message-Id: <null>
Administrivia:
#The Perl-Users Digest is a retransmission of the USENET newsgroup
#comp.lang.perl.misc. For subscription or unsubscription requests, send
#the single line:
#
# subscribe perl-users
#or:
# unsubscribe perl-users
#
#to almanac@ruby.oce.orst.edu.
NOTE: due to the current flood of worm email banging on ruby, the smtp
server on ruby has been shut off until further notice.
To submit articles to comp.lang.perl.announce, send your article to
clpa@perl.com.
#To request back copies (available for a week or so), send your request
#to almanac@ruby.oce.orst.edu with the command "send perl-users x.y",
#where x is the volume number and y is the issue number.
#For other requests pertaining to the digest, send mail to
#perl-users-request@ruby.oce.orst.edu. Do not waste your time or mine
#sending perl questions to the -request address, I don't have time to
#answer them even if I did know the answer.
------------------------------
End of Perl-Users Digest V10 Issue 8450
***************************************
