[26141] in Perl-Users-Digest
Perl-Users Digest, Issue: 8332 Volume: 10
daemon@ATHENA.MIT.EDU (Perl-Users Digest)
Thu Aug 18 14:05:18 2005
Date: Thu, 18 Aug 2005 11:05:05 -0700 (PDT)
From: Perl-Users Digest <Perl-Users-Request@ruby.OCE.ORST.EDU>
To: Perl-Users@ruby.OCE.ORST.EDU (Perl-Users Digest)
Perl-Users Digest Thu, 18 Aug 2005 Volume: 10 Number: 8332
Today's topics:
Re: [Socket]What is the bast solution? <scobloke2@infotop.co.uk>
Re: coding style and performance <1usa@llenroc.ude.invalid>
Re: coding style and performance <1usa@llenroc.ude.invalid>
Re: coding style and performance <nobull@mail.com>
Re: Organizing data for readability and efficiency xhoster@gmail.com
Re: Organizing data for readability and efficiency <Mark.Seger@hp.com>
Re: Organizing data for readability and efficiency xhoster@gmail.com
Digest Administrivia (Last modified: 6 Apr 01) (Perl-Users-Digest Admin)
----------------------------------------------------------------------
Date: Thu, 18 Aug 2005 15:05:06 +0000 (UTC)
From: Ian Wilson <scobloke2@infotop.co.uk>
Subject: Re: [Socket]What is the bast solution?
Message-Id: <de2831$f4c$1@nwrdmz01.dmz.ncs.ea.ibs-infra.bt.com>
sonet wrote:
> noneblocking socket
> io::poll
> thread
> prefork
>
> If i will handle a chat server that have 1000+ connection in the same time.
> What is the bast solution? Or apache (mod_perl2 preconnectionHander)...?
>
>
>
>
The discussion here might be of interest:
http://www.acme.com/software/thttpd/notes.html#nbio
This isn't a Perl answer but this doesn't seem to be a Perl question.
------------------------------
Date: Thu, 18 Aug 2005 15:53:49 GMT
From: "A. Sinan Unur" <1usa@llenroc.ude.invalid>
Subject: Re: coding style and performance
Message-Id: <Xns96B660B97B56Easu1cornelledu@127.0.0.1>
ngoc <ngoc@yahoo.com> wrote in news:43047c9d$1@news.broadpark.no:
> Hi
> Look at some code I see
> for my $element (@array) {
> ..do something..
> }
> I think is not performance attractive because of
> the interpreter will create new variable (my $element) for every loop
> step.
>
> I prefer
>
> my $element;
> for $element (@array) {
> ..do something..
> }
> The interpreter will NOT create new variable for every loop step. It
> just updates current one.
>
> What I think is right or wrong?
You do realize that the two snippets above are not the same. In general,
you should declare variables in the smallest applicable scope.
To compare the speed, you can use the Benchmark module:
perldoc Benchmark
On the other hand, I am not sure if you could detect a speed difference
between the two methods, and even if you could, it would be tiny, and
even then, there is no reason, a priori, your preferred method is going
to be faster.
Sinan
--
A. Sinan Unur <1usa@llenroc.ude.invalid>
(reverse each component and remove .invalid for email address)
comp.lang.perl.misc guidelines on the WWW:
http://mail.augustmail.com/~tadmc/clpmisc/clpmisc_guidelines.html
------------------------------
Date: Thu, 18 Aug 2005 15:53:49 GMT
From: "A. Sinan Unur" <1usa@llenroc.ude.invalid>
Subject: Re: coding style and performance
Message-Id: <Xns96B6610E8950Casu1cornelledu@127.0.0.1>
ngoc <ngoc@yahoo.com> wrote in news:43047c9d$1@news.broadpark.no:
> Hi
> Look at some code I see
> for my $element (@array) {
> ..do something..
> }
> I think is not performance attractive because of
> the interpreter will create new variable (my $element) for every loop
> step.
>
> I prefer
>
> my $element;
> for $element (@array) {
> ..do something..
> }
> The interpreter will NOT create new variable for every loop step. It
> just updates current one.
>
> What I think is right or wrong?
You do realize that the two snippets above are not the same. In general,
you should declare variables in the smallest applicable scope.
To compare the speed, you can use the Benchmark module:
perldoc Benchmark
On the other hand, I am not sure if you could detect a speed difference
between the two methods, and even if you could, it would be tiny, and
even then, there is no reason, a priori, your preferred method is going
to be faster.
Sinan
--
A. Sinan Unur <1usa@llenroc.ude.invalid>
(reverse each component and remove .invalid for email address)
comp.lang.perl.misc guidelines on the WWW:
http://mail.augustmail.com/~tadmc/clpmisc/clpmisc_guidelines.html
------------------------------
Date: Thu, 18 Aug 2005 18:31:26 +0100
From: Brian McCauley <nobull@mail.com>
Subject: Re: coding style and performance
Message-Id: <de2gle$7jl$1@redhat2.bham.ac.uk>
ngoc wrote:
> Villy Kruse wrote:
>
>> On Thu, 18 Aug 2005 14:18:38 +0200,
>> ngoc <ngoc@yahoo.com> wrote:
>>
>>
>>
>>> I prefer
>>>
>>> my $element;
>>> for $element (@array) {
>>> ..do something..
>>> }
>>> The interpreter will NOT create new variable for every loop step. It
>>> just updates current one.
>>
>> Then again, the loop won't use the $element you create before entering
>> the loop, it will use a new loop variable.
> I have tested what you suggest and It is as you say. But I still do not
> understand the logic.
> If the loop create a new variable with the same name as my own variable,
> So there are two variables with the same name.
> 1. My own variable will not conflict with loop's variable, because my
> variable is 'global' and loop variable is 'local' (scoping).
> 2. If a new variable is created without a 'my' in front of it, why 'use
> strict;' do not react?
History mostly. I've been saying for a long time that perl should at
least emit a warning when it infers the missing my() in a for statement.
Note: that $element is only implicitly declared as a new lexical
variable if there's currently a lexical $element in scope. If $element
is a package variable then the is different.
use strict;
use warnings;
our($element) = 'outside';
sub print_element {
print "$element\n";
}
print_element; # prints outside
for $element ( 'inside' ) {
print_element; # prints inside
}
print_element; # prints outside
__END__
If you change our() to my() it prints 'outside' three times.
------------------------------
Date: 18 Aug 2005 15:46:54 GMT
From: xhoster@gmail.com
Subject: Re: Organizing data for readability and efficiency
Message-Id: <20050818114654.762$sC@newsreader.com>
Mark Seger <Mark.Seger@hp.com> wrote:
> I know this can be a very long, complex topic and am hoping by using a
> specific example can keep things more focused, but I also suspect not.
> 8-)
>
> Anyhow I've put together a script for generating a variety of plots
> using perl/Tk from a set of tables. Each table describes what a plot
> looks like, for example its title, Y-acess limits, the variables it can
> plot, etc.
>
> To get things going quickly, I stored each in a array named for what the
> data is and indexed by the plot number. So $title[0] is the title of
> the first plot, $ymax[2] is upper bound of the yaxis of the 3rd, etc.
> While easy to understand there are too many arrays to pass around to
> various routines and I have therefore made them all globals.
>
> What I'm now looking for are alternatives data organizations that could
> allow me to store all the information in a single data structure that
> could be passed to different routines but still be easily readable as
> well as efficient.
Efficient with regards to what? Quick access time? Low memory usage?
Less typing?
...
> Another alternative would be to have a hash of arrays such that each
> hash entry could itself be an array (I'm not entirely sure of the
> terminology here having only recently been reading about this technique)
> and then I could refer to the data elements as something like
> $plot{'title'}->[2] to refer to the title of the 3rd plot and could
> simply pass around a reference to %plot. I like this organization but
> also suspect there are a variety of purmutations of this methodology and
> how one might do this and maintain good performance.
I'd prefer to always have the hashref $plot, rather than the hash %plot.
If the main program has a hash, but always passes refs to that hash to the
subs, then you have to keep changing your coding habit from dereferencing
to nondereferencing depending on which part of the code you are in. Just
make the main program use a ref so that it is on the same footing as subs
are.
Assuming the plots form some kind of natural progression such that
numbering them from 0 to N is the right thing to do, and thus the array is
a good representation, I'd turn it inside out, $plot->[2]{'title'}. It
seems like you are more likely to want all the things for one plot grouped
together, than you are to want the titles across all the plots grouped
together.
If the plots do not form some kind of natural progression such that
numbering them from 0 to N is the right thing to do; I might instead use a
hash of hashes, pulling the title out from being a value in the lower level
hash to being a key in the upper level one. Actually, I might just
duplicate the title, rather than pull it out from the lower level.
$plot->{'Time versus Money'}{'Y_MAX'}=8;
$plot->{'Time versus Money'}{'title'}='Time versus Money';
But then again, maybe not. If your code doens't care what the title is,
and only wants a list of plots and the title of each is nobody' business
except the plots, then leaving them in an array makes sense.
> In any event, as I said in the beginning I'd be interested in people's
> thoughts on this topic and how they might choose to organize things for
> ease of maintenance (my code tends to be more verbose for clarity) as
> well as performance.
I'm having a hard time seeing how these decisions will have much effect on
performance. Are there going to be millions of plots?
Xho
--
-------------------- http://NewsReader.Com/ --------------------
Usenet Newsgroup Service $9.95/Month 30GB
------------------------------
Date: Thu, 18 Aug 2005 12:35:50 -0400
From: Mark Seger <Mark.Seger@hp.com>
To: xhoster@gmail.com
Subject: Re: Organizing data for readability and efficiency
Message-Id: <4304B8E6.6040709@hp.com>
>>In any event, as I said in the beginning I'd be interested in people's
>>thoughts on this topic and how they might choose to organize things for
>>ease of maintenance (my code tends to be more verbose for clarity) as
>>well as performance.
>
>
> I'm having a hard time seeing how these decisions will have much effect on
> performance. Are there going to be millions of plots?
Clearly static information like plot titles, variable definitions, etc
are accessed once and I don't care a whole lot about efficiency.
But while there won't be millions of plots but there could be millions
of data points. Thse plots have to do with system performance
monitoring and I typically collect performance data every 10 seconds and
then plot it, that's almost 10,000 data points per line per plot per
day. Some plots have 4 or more lines in them so now we're up to maybe
50K calculations for one plot. Now lets say you have 1/2 dozen plots
and as many as dozens or even hundreds of systems. If I want to know
what the maximum y-value is for a data point, I want to store that
number in a way that makes access to it reasonably efficient.
But whether this particular application needs to access the data a lot
vs a little, I think the question is still a valid one as to how
efficient is it to access data stored in various structue like hashes of
arrays, etc.
-mark
------------------------------
Date: 18 Aug 2005 17:41:42 GMT
From: xhoster@gmail.com
Subject: Re: Organizing data for readability and efficiency
Message-Id: <20050818134142.294$6T@newsreader.com>
Mark Seger <Mark.Seger@hp.com> wrote:
> >>In any event, as I said in the beginning I'd be interested in people's
> >>thoughts on this topic and how they might choose to organize things for
> >>ease of maintenance (my code tends to be more verbose for clarity) as
> >>well as performance.
> >
> >
> > I'm having a hard time seeing how these decisions will have much effect
> > on performance. Are there going to be millions of plots?
>
> Clearly static information like plot titles, variable definitions, etc
> are accessed once and I don't care a whole lot about efficiency.
>
> But while there won't be millions of plots but there could be millions
> of data points. Thse plots have to do with system performance
> monitoring and I typically collect performance data every 10 seconds and
> then plot it, that's almost 10,000 data points per line per plot per
> day. Some plots have 4 or more lines in them so now we're up to maybe
> 50K calculations for one plot. Now lets say you have 1/2 dozen plots
> and as many as dozens or even hundreds of systems. If I want to know
> what the maximum y-value is for a data point, I want to store that
> number in a way that makes access to it reasonably efficient.
You still have to actually plot this data, no? It seems to me that that
is going to be dominate anything else you are going to be doing with
the data, so the efficiency of all this other stuff is still moot, in my
mind.
In any case, when you have nested loops which process nested data
structures, you can always materialize values in between the loops. I do
it all the time, although usually for ease of typing/reading the code
rather than out of performance concerns.
foreach my $i ( 0..$#$plots) {
my $y_max=$plots->[$i]{'Y-max'};
foreach my $y_value ( @{$plots->[$i]{'Y-value-list'}} ) {
die "Error in $i" if $y_value > $y_max;
## die "Error in $i" if $y_value > $plots->[$i]{'Y-max'};
};
};
> But whether this particular application needs to access the data a lot
> vs a little, I think the question is still a valid one as to how
> efficient is it to access data stored in various structue like hashes of
> arrays, etc.
You seem to want minutely detailed answers to rather vague and expansive
questions. If you are so concerned about micro-optimization of your code,
then start with real code and test, test, test.
If you are considering making the choice between arrays and hashes based on
the performance of single look-ups into each (rather than by whether the
nature of your data best fits an array or best fits a hash), then I say you
are engaging in micro-optimization. Perhaps this is the nub of your
question, so let me repeat. The difference between array access and hash
access is small enough that if you are concerned about it, only actual
tests on actual data will be satifactory.
Xho
--
-------------------- http://NewsReader.Com/ --------------------
Usenet Newsgroup Service $9.95/Month 30GB
------------------------------
Date: 6 Apr 2001 21:33:47 GMT (Last modified)
From: Perl-Users-Request@ruby.oce.orst.edu (Perl-Users-Digest Admin)
Subject: Digest Administrivia (Last modified: 6 Apr 01)
Message-Id: <null>
Administrivia:
#The Perl-Users Digest is a retransmission of the USENET newsgroup
#comp.lang.perl.misc. For subscription or unsubscription requests, send
#the single line:
#
# subscribe perl-users
#or:
# unsubscribe perl-users
#
#to almanac@ruby.oce.orst.edu.
NOTE: due to the current flood of worm email banging on ruby, the smtp
server on ruby has been shut off until further notice.
To submit articles to comp.lang.perl.announce, send your article to
clpa@perl.com.
#To request back copies (available for a week or so), send your request
#to almanac@ruby.oce.orst.edu with the command "send perl-users x.y",
#where x is the volume number and y is the issue number.
#For other requests pertaining to the digest, send mail to
#perl-users-request@ruby.oce.orst.edu. Do not waste your time or mine
#sending perl questions to the -request address, I don't have time to
#answer them even if I did know the answer.
------------------------------
End of Perl-Users Digest V10 Issue 8332
***************************************
