[26168] in Perl-Users-Digest
Perl-Users Digest, Issue: 8357 Volume: 10
daemon@ATHENA.MIT.EDU (Perl-Users Digest)
Thu Aug 25 14:05:18 2005
Date: Thu, 25 Aug 2005 11:05:06 -0700 (PDT)
From: Perl-Users Digest <Perl-Users-Request@ruby.OCE.ORST.EDU>
To: Perl-Users@ruby.OCE.ORST.EDU (Perl-Users Digest)
Perl-Users Digest Thu, 25 Aug 2005 Volume: 10 Number: 8357
Today's topics:
Re: Field matching <jgibson@mail.arc.nasa.gov>
Re: Field matching <mail@nomail.no.com>
Re: Field matching <mmaaggggoott@vv.ppll>
Re: Field matching <Jeff.Stampes@xilinx.com>
Re: Is there any performance benefit to... xhoster@gmail.com
Re: Is there any performance benefit to... xhoster@gmail.com
Re: Is there any performance benefit to... <nomail@hursley.ibm.com>
Re: Is there any performance benefit to... xhoster@gmail.com
Digest Administrivia (Last modified: 6 Apr 01) (Perl-Users-Digest Admin)
----------------------------------------------------------------------
Date: Thu, 25 Aug 2005 08:19:38 -0700
From: Jim Gibson <jgibson@mail.arc.nasa.gov>
Subject: Re: Field matching
Message-Id: <250820050819385194%jgibson@mail.arc.nasa.gov>
In article <430DCB06.82BC1BA9@nomail.no.com>, Geezer From The Freezer
<mail@nomail.no.com> wrote:
> Whats the best way at using field manipulation in perl (much like
> awk where you can use: awk '$2 == 3 && $3 == 9{print}' )
>
> Assume I don't know how many fields are in each line of the input file too
Use the split function to break up a line into tokens separated by
white-space (the default) or any other separator, which you can define
with a regular expression (see 'perldoc -f split' for more info). For
example:
while(<DATA>) {
my @f = split;
print if @f[1] == 3 && $f[2] == 9;
}
Note that Perl arrays start with index [0] while awk fields are $1, $2,
etc., hence the different numbering of the fields. The number of fields
found will be @f in scalar context (e.g., 'scalar @f' or 'my $nf =
@f'), and the highest index is given by $#f = (scalar @f - 1).
----== Posted via Newsfeeds.Com - Unlimited-Uncensored-Secure Usenet News==----
http://www.newsfeeds.com The #1 Newsgroup Service in the World! >100,000 Newsgroups
---= East/West-Coast Server Farms - Total Privacy via Encryption =---
------------------------------
Date: Thu, 25 Aug 2005 16:41:30 +0100
From: Geezer From The Freezer <mail@nomail.no.com>
Subject: Re: Field matching
Message-Id: <430DE6AA.E8E9FD5@nomail.no.com>
Jim Gibson wrote:
>
> In article <430DCB06.82BC1BA9@nomail.no.com>, Geezer From The Freezer
> <mail@nomail.no.com> wrote:
>
> > Whats the best way at using field manipulation in perl (much like
> > awk where you can use: awk '$2 == 3 && $3 == 9{print}' )
> >
> > Assume I don't know how many fields are in each line of the input file too
>
> Use the split function to break up a line into tokens separated by
> white-space (the default) or any other separator, which you can define
> with a regular expression (see 'perldoc -f split' for more info). For
> example:
>
> while(<DATA>) {
> my @f = split;
> print if @f[1] == 3 && $f[2] == 9;
> }
Jim,
Ok I'll give that a whirl!! Thanks!
------------------------------
Date: Thu, 25 Aug 2005 17:54:45 +0200
From: /maggot/ <mmaaggggoott@vv.ppll>
Subject: Re: Field matching
Message-Id: <dekpcm$p7k$1@zeus.man.szczecin.pl>
Geezer From The Freezer wrote:
> Whats the best way at using field manipulation in perl (much like
> awk where you can use: awk '$2 == 3 && $3 == 9{print}' )
The -a (autosplit) commandline option is what you are looking for.
man perlrun
------------------------------
Date: Thu, 25 Aug 2005 09:54:48 -0600
From: Jeff Stampes <Jeff.Stampes@xilinx.com>
Subject: Re: Field matching
Message-Id: <dekpk8$1jo1@xco-news.xilinx.com>
Jim Gibson wrote:
> In article <430DCB06.82BC1BA9@nomail.no.com>, Geezer From The Freezer
> <mail@nomail.no.com> wrote:
>
>
>>Whats the best way at using field manipulation in perl (much like
>>awk where you can use: awk '$2 == 3 && $3 == 9{print}' )
>>
>>Assume I don't know how many fields are in each line of the input file too
>
>
> Use the split function to break up a line into tokens separated by
> white-space (the default) or any other separator, which you can define
> with a regular expression (see 'perldoc -f split' for more info). For
> example:
>
> while(<DATA>) {
> my @f = split;
> print if @f[1] == 3 && $f[2] == 9;
> }
Make it easier and learn about perl's command line switches. If you're
used to awk (which I'm not), you can apparently do awk-like things:
perl -lane 'print if $F[1] == 3 && $F[2] == 9;' MyDataFile
tmp > perl -lane 'print if $F[1] == 3 && $F[2] == 9;' MyDataFile
Field1 3 9
tmp > cat MyDataFile
Field1 Field2 Field3
Quazi Plitch Plisk
Field1 3 9
Foo Bar Field3
Field1 Field2 Field3
My personal mnemonic for remembering -lane as the magic set of command
line switches: "perl can do what awk can...it's not lame, it's lane".
See 'perldoc perlrun' for more information.
~Jeff
------------------------------
Date: 25 Aug 2005 15:32:48 GMT
From: xhoster@gmail.com
Subject: Re: Is there any performance benefit to...
Message-Id: <20050825113248.967$2n@newsreader.com>
Derek Fountain <nomail@hursley.ibm.com> wrote:
> >
> > Why don't you try running it without the line and seeing if there's a
> > benefit? Sheesh.
>
> Because, as I said in the part you conveniently chopped out of the quote
> above, this code takes many hours - maybe days - to run.
Hours? who cares. Days? Well, then run it on a smaller sub-set of the
problem. Sheesh.
Xho
--
-------------------- http://NewsReader.Com/ --------------------
Usenet Newsgroup Service $9.95/Month 30GB
------------------------------
Date: 25 Aug 2005 15:56:20 GMT
From: xhoster@gmail.com
Subject: Re: Is there any performance benefit to...
Message-Id: <20050825115620.484$SY@newsreader.com>
Derek Fountain <nomail@hursley.ibm.com> wrote:
> I've inherited a piece of code that does something like this:
>
> sub func {
> my %hash = ();
>
> ...lots of code that populates and uses the hash
Does any of the code pass a reference to that hash to other lexical scopes,
who might then hang onto the reference? Does any of the code make closures
upon the hash which are passed out of the lexical scope?
>
> %hash = ();
> }
>
> There are hundreds of these functions and they are called millions of
> times in a procedure which takes hours, sometimes days, to complete. The
> guy who wrote it was obviously concerned about performance.
>
> The question pertains to that resetting of the hash at the end of
> function. Does it do anything that exiting the function doesn't do?
It is possible it does, depending on your answers the questions above.
> In
> Java-land I've seen things like that to force the garbage collector to
> jump in, but in Perl-land won't it just get optimised away?
As far as I can tell, perl does very little optimization of this
nature. (In fact in this case it can't safely optimize it aways, because
AFAIK it does no flow analysis, so it doesn't know if references to the
hash have been passed out of the sub or not.)
> The hash
> doesn't get that big, as far as I can see, and there's nothing unusual
> in any of the hash processing code.
>
> Normally I'd just remove the line (or ignore it) but since it appears in
> every one of these functions, and since removing a useful optimisation
> might add an hour or two to my runtime, I thought I'd ask.
Benchmark it and see. If you don't trust your own benchmark to adequately
reflect the reality of your program, then you surely shouldn't trust the
say-so of people who have never actually seen the code.
Xho
--
-------------------- http://NewsReader.Com/ --------------------
Usenet Newsgroup Service $9.95/Month 30GB
------------------------------
Date: Thu, 25 Aug 2005 17:49:34 +0100
From: Derek Fountain <nomail@hursley.ibm.com>
Subject: Re: Is there any performance benefit to...
Message-Id: <430df635$0$69780$892e7fe2@authen.white.readfreenews.net>
> Hours? who cares. Days? Well, then run it on a smaller sub-set of the
> problem. Sheesh.
Maybe I've been in PHP-land too long, but when I used to frequent this
newsgroup a couple of years ago I found it invariably helpful. People
who knew the answers to on-topic questions would offer advice, and those
who didn't know the answers would keep quiet. People who wanted to learn
things about Perl would ask questions, and no one got shot down for
asking. Asking and learning always used to be the point of the newgroup.
Seems things have changed.
For the record, I'm working on a cluster of 256 dual and quad processor
Intel machines, with several thousand hard drives - well over a million
quids worth of hardware. It takes 4 of us working fulltime to drive this
thing, and running a testcase on it involves hours or days of
preparation. Changing the data to a smaller subset, then reconfiguring
the cluster to expect a new data set would take weeks of work. I guess
the smart-arses who like to say "just try it" don't have a whole lot of
experience of computing problems on this scale.
Trust me, sometimes it is simpler to just ask.
------------------------------
Date: 25 Aug 2005 17:17:58 GMT
From: xhoster@gmail.com
Subject: Re: Is there any performance benefit to...
Message-Id: <20050825131758.319$G3@newsreader.com>
Derek Fountain <nomail@hursley.ibm.com> wrote:
> > Hours? who cares. Days? Well, then run it on a smaller sub-set of
> > the problem. Sheesh.
>
> Maybe I've been in PHP-land too long, but when I used to frequent this
> newsgroup a couple of years ago I found it invariably helpful. People
> who knew the answers to on-topic questions would offer advice, and those
> who didn't know the answers would keep quiet. People who wanted to learn
> things about Perl would ask questions, and no one got shot down for
> asking. Asking and learning always used to be the point of the newgroup.
> Seems things have changed.
>
> For the record, I'm working on a cluster of 256 dual and quad processor
> Intel machines, with several thousand hard drives - well over a million
> quids worth of hardware.
Cool. If it is worth doing all that, then it is worth doing right, is it
not?
> It takes 4 of us working fulltime to drive this
> thing, and running a testcase on it involves hours or days of
> preparation.
Frankly, I don't find this very impressive. I have a set-up only slightly
smaller than that, and it doesn't take 4 people working full time to drive
it and it doesn't take me hours or days of preparation to run a test case.
> Changing the data to a smaller subset, then reconfiguring
> the cluster to expect a new data set would take weeks of work.
Then you are probably doing it wrong. Maybe we could help you. If you let
us.
> I guess
> the smart-arses who like to say "just try it" don't have a whole lot of
> experience of computing problems on this scale.
You guess wrong.
BTW, I notice you haven't answered the questions which I asked in
an effort to help you help me to help you. I guess pissing and moaning is
more important to you than actually getting the answer you claim you want.
>
> Trust me, sometimes it is simpler to just ask.
Since you don't seem to want to listen to the answers you get, it would be
even simpler to just do nothing.
In the mean time, I've run a benchmark showing that including the %hash=()
at the end of the subroutine is slightly but reliably slower. On my
computer. With my verion of Perl. With my "filler" code in the subroutine.
In the milieu of competing CPU and memory demands that exists on my system.
It didn't take me a week of preparation time to do this. If you think this
result transfers to your situation, then bon apetite.
Xho
--
-------------------- http://NewsReader.Com/ --------------------
Usenet Newsgroup Service $9.95/Month 30GB
------------------------------
Date: 6 Apr 2001 21:33:47 GMT (Last modified)
From: Perl-Users-Request@ruby.oce.orst.edu (Perl-Users-Digest Admin)
Subject: Digest Administrivia (Last modified: 6 Apr 01)
Message-Id: <null>
Administrivia:
#The Perl-Users Digest is a retransmission of the USENET newsgroup
#comp.lang.perl.misc. For subscription or unsubscription requests, send
#the single line:
#
# subscribe perl-users
#or:
# unsubscribe perl-users
#
#to almanac@ruby.oce.orst.edu.
NOTE: due to the current flood of worm email banging on ruby, the smtp
server on ruby has been shut off until further notice.
To submit articles to comp.lang.perl.announce, send your article to
clpa@perl.com.
#To request back copies (available for a week or so), send your request
#to almanac@ruby.oce.orst.edu with the command "send perl-users x.y",
#where x is the volume number and y is the issue number.
#For other requests pertaining to the digest, send mail to
#perl-users-request@ruby.oce.orst.edu. Do not waste your time or mine
#sending perl questions to the -request address, I don't have time to
#answer them even if I did know the answer.
------------------------------
End of Perl-Users Digest V10 Issue 8357
***************************************
