[23813] in Perl-Users-Digest
Perl-Users Digest, Issue: 6016 Volume: 10
daemon@ATHENA.MIT.EDU (Perl-Users Digest)
Thu Jan 29 20:16:18 2004
Date: Thu, 29 Jan 2004 17:15:48 -0800 (PST)
From: Perl-Users Digest <Perl-Users-Request@ruby.OCE.ORST.EDU>
To: Perl-Users@ruby.OCE.ORST.EDU (Perl-Users Digest)
Perl-Users Digest Thu, 29 Jan 2004 Volume: 10 Number: 6016
Today's topics:
Re: Is there a more efficient way to do this ? (Malcolm Dew-Jones)
Re: Is there a more efficient way to do this ? <dragnet@internalysis.com>
Re: Is there a more efficient way to do this ? <usenet@morrow.me.uk>
Re: Is there a more efficient way to do this ? <glex_nospam@qwest.invalid>
Re: Is there a more efficient way to do this ? <dragnet@internalysis.com>
Re: Is there a more efficient way to do this ? <uri@stemsystems.com>
Re: Is there a more efficient way to do this ? <dragnet@internalysis.com>
Re: Is there a more efficient way to do this ? <dragnet@internalysis.com>
Re: Is there a more efficient way to do this ? <uri@stemsystems.com>
Re: Is there a more efficient way to do this ? <dragnet@internalysis.com>
Re: Is there a more efficient way to do this ? <syscjm@gwu.edu>
Re: Is there a more efficient way to do this ? <dragnet@internalysis.com>
Digest Administrivia (Last modified: 6 Apr 01) (Perl-Users-Digest Admin)
----------------------------------------------------------------------
Date: 27 Jan 2004 14:29:58 -0800
From: yf110@vtn1.victoria.tc.ca (Malcolm Dew-Jones)
Subject: Re: Is there a more efficient way to do this ?
Message-Id: <4016e666@news.victoria.tc.ca>
Marc Bissonnette (dragnet@internalysis.com) wrote:
: Sorry for the vague subject line -
: I am displaying a table of data from a projects db; Projects that are
: active appear on top, with completed tasks appearing on the bottom;
: My ( relevant portion of ) code looks like this:
: ($row[8] is the date completed field, in yyyymmdd - it will only contain 8
: digits if it is completed, otherwise NULL for active and '1' for 'on hold')
OT: that looks bad. NULL is fine for unfinished, that's what it's for -
there's no date yet - but using 1 to indicate 'on hold' is overloading the
meaning of the field. One could imagine wanting to know _when_ a project
was put on hold. (Of course this data layout may not be up to you.)
------------------------------
Date: Wed, 28 Jan 2004 00:15:21 GMT
From: Marc Bissonnette <dragnet@internalysis.com>
Subject: Re: Is there a more efficient way to do this ?
Message-Id: <Xns947DC3E522DBCdragnetinternalysisc@207.35.177.134>
yf110@vtn1.victoria.tc.ca (Malcolm Dew-Jones) wrote in
news:4016e666@news.victoria.tc.ca:
> Marc Bissonnette (dragnet@internalysis.com) wrote:
>: Sorry for the vague subject line -
>
>: I am displaying a table of data from a projects db; Projects that are
>: active appear on top, with completed tasks appearing on the bottom;
>
>: My ( relevant portion of ) code looks like this:
>: ($row[8] is the date completed field, in yyyymmdd - it will only
>: contain 8 digits if it is completed, otherwise NULL for active and
>: '1' for 'on hold')
>
> OT: that looks bad. NULL is fine for unfinished, that's what it's for
> - there's no date yet - but using 1 to indicate 'on hold' is
> overloading the meaning of the field. One could imagine wanting to
> know _when_ a project was put on hold. (Of course this data layout
> may not be up to you.)
Well, I've got a field set up for 'date_assigned', so I figured simply re-
using the 'date_completed' field for "on hold" or "active" would save an
extra column;
You do bring up a good point, though - given that status of any project can
be changed (brought back to active from 'completed' or 'on hold'), I think
I'll add a field that holds a date value for 'date_last_status_change',
which would let the users know the 'whens' of all three status
possibilities.
Thanks for the insight :)
--
Marc Bissonnette
CGI / Database / Web Management Tools: http://www.internalysis.com
Something To Sell? Looking To Buy? http://www.whitewaterclassifieds.ca
Looking for a new ISP? http://www.canadianisp.com
------------------------------
Date: Wed, 28 Jan 2004 02:21:12 +0000 (UTC)
From: Ben Morrow <usenet@morrow.me.uk>
Subject: Re: Is there a more efficient way to do this ?
Message-Id: <bv76ao$2bv$1@wisteria.csv.warwick.ac.uk>
Marc Bissonnette <dragnet@internalysis.com> wrote:
> yf110@vtn1.victoria.tc.ca (Malcolm Dew-Jones) wrote in
> news:4016e666@news.victoria.tc.ca:
> > OT: that looks bad. NULL is fine for unfinished, that's what it's for
> > - there's no date yet - but using 1 to indicate 'on hold' is
> > overloading the meaning of the field. One could imagine wanting to
> > know _when_ a project was put on hold. (Of course this data layout
> > may not be up to you.)
>
> Well, I've got a field set up for 'date_assigned', so I figured simply re-
> using the 'date_completed' field for "on hold" or "active" would save an
> extra column;
>
> You do bring up a good point, though - given that status of any project can
> be changed (brought back to active from 'completed' or 'on hold'), I think
> I'll add a field that holds a date value for 'date_last_status_change',
> which would let the users know the 'whens' of all three status
> possibilities.
Well... what I'd do is have a sub-entity 'status_changes', with a
record giving the date of and new status after each change. Then you
can simply get the latest record for a given project to find the
current status, and the oldest to find when the project was
initiated. This may be more history than you need to keep, though.
Ben
--
We do not stop playing because we grow old;
we grow old because we stop playing.
ben@morrow.me.uk
------------------------------
Date: Tue, 27 Jan 2004 12:47:23 -0600
From: "J. Gleixner" <glex_nospam@qwest.invalid>
Subject: Re: Is there a more efficient way to do this ?
Message-Id: <0nyRb.45$dw4.48030@news.uswest.net>
Marc Bissonnette wrote:
> Sorry for the vague subject line -
>
> I am displaying a table of data from a projects db; Projects that are
> active appear on top, with completed tasks appearing on the bottom;
>
> My ( relevant portion of ) code looks like this:
> ($row[8] is the date completed field, in yyyymmdd - it will only contain 8
> digits if it is completed, otherwise NULL for active and '1' for 'on hold')
>
> # ... set up DB query, start loop from DB...
>
> while (@row=$sth->fetchrow()) {
> if (length $row[8] == 8) {
> $completeddata .= "$row[0]<BR>\n";
> } else {
> print "$row[0]<BR>\n";
> }
> }
> if ($completeddata) {
> print "Completed Items:<BR>\n";
> print $completeddata;
> }
>
> This code works - it displays everything in the order and in the place
> desired, but I'm wondering if the above is a kludge, or really the best way
> to accomplish displaying completed items second (other than executing two
> separate queries on the DB - Which is equally feasible, but wondering if I
> need to do that, or if the above is more efficient)
>
Why query for 9+ fields and only use the first and the 9th ([0,8])?
Unless 8 was mis-typed and should have been 0. Complete waste of time to
query for more values than you use.
Sort on your date completed field in your SQL, then you only need the
field for $row[0] in your select clause. You'd also need a simple way
to print your "Completed Items" line once.
------------------------------
Date: Tue, 27 Jan 2004 18:58:48 GMT
From: Marc Bissonnette <dragnet@internalysis.com>
Subject: Re: Is there a more efficient way to do this ?
Message-Id: <Xns947D8E39BF799dragnetinternalysisc@207.35.177.135>
"J. Gleixner" <glex_nospam@qwest.invalid> wrote in
news:0nyRb.45$dw4.48030@news.uswest.net:
> Marc Bissonnette wrote:
>> Sorry for the vague subject line -
>>
>> I am displaying a table of data from a projects db; Projects that are
>> active appear on top, with completed tasks appearing on the bottom;
>>
>> My ( relevant portion of ) code looks like this:
>> ($row[8] is the date completed field, in yyyymmdd - it will only
>> contain 8 digits if it is completed, otherwise NULL for active and
>> '1' for 'on hold')
>>
>> # ... set up DB query, start loop from DB...
>>
>> while (@row=$sth->fetchrow()) {
>> if (length $row[8] == 8) {
>> $completeddata .= "$row[0]<BR>\n";
>> } else {
>> print "$row[0]<BR>\n";
>> }
>> }
>> if ($completeddata) {
>> print "Completed Items:<BR>\n";
>> print $completeddata;
>> }
>>
>> This code works - it displays everything in the order and in the
>> place desired, but I'm wondering if the above is a kludge, or really
>> the best way to accomplish displaying completed items second (other
>> than executing two separate queries on the DB - Which is equally
>> feasible, but wondering if I need to do that, or if the above is more
>> efficient)
>>
> Why query for 9+ fields and only use the first and the 9th ([0,8])?
> Unless 8 was mis-typed and should have been 0. Complete waste of time
> to query for more values than you use.
>
> Sort on your date completed field in your SQL, then you only need the
> field for $row[0] in your select clause. You'd also need a simple way
> to print your "Completed Items" line once.
Sorry, I only used the single field in my example code for brevity - in
the production code, all nine fields are indeed used, but I didn't want
to waste people's space/time with code (display and formatting) that
wasn't relevant to the problem.
The sort on date is perfect - I should have thought of that - thank you!
For printing 'Completed Items' only once, I think this would work (and
reduces the example code above by 36%!):
# query sorted by date for the following
while (@row=$sth->fetchrow()) {
if (length $row[8] == 8 && $completedheader != 1) {
print "Completed Items:<BR>\n";
$completedheader=1;
}
print "$row[0]<BR>\n";
}
Thank you - this works *much* better than what I had before!
--
Marc Bissonnette
CGI / Database / Web Management Tools: http://www.internalysis.com
Something To Sell? Looking To Buy? http://www.whitewaterclassifieds.ca
Looking for a new ISP? http://www.canadianisp.com
------------------------------
Date: Tue, 27 Jan 2004 19:20:05 GMT
From: Uri Guttman <uri@stemsystems.com>
Subject: Re: Is there a more efficient way to do this ?
Message-Id: <x7znc95hcq.fsf@mail.sysarch.com>
>>>>> "MB" == Marc Bissonnette <dragnet@internalysis.com> writes:
MB> For printing 'Completed Items' only once, I think this would work (and
MB> reduces the example code above by 36%!):
MB> # query sorted by date for the following
MB> while (@row=$sth->fetchrow()) {
MB> if (length $row[8] == 8 && $completedheader != 1) {
MB> print "Completed Items:<BR>\n";
MB> $completedheader=1;
MB> }
MB> print "$row[0]<BR>\n";
MB> }
MB> Thank you - this works *much* better than what I had before!
someone else mentioned buffering and printing. i suggest you always
buffer your output and print later. it is generally faster (each print
call is slow) and you can control where the output goes (maybe you want
the output in mail or a file as well). this is summarized in my rule,
print rarely, print late.
uri
--
Uri Guttman ------ uri@stemsystems.com -------- http://www.stemsystems.com
--Perl Consulting, Stem Development, Systems Architecture, Design and Coding-
Search or Offer Perl Jobs ---------------------------- http://jobs.perl.org
------------------------------
Date: Wed, 28 Jan 2004 02:27:49 GMT
From: Marc Bissonnette <dragnet@internalysis.com>
Subject: Re: Is there a more efficient way to do this ?
Message-Id: <Xns947DDA5AF8B42dragnetinternalysisc@207.35.177.134>
Ben Morrow <usenet@morrow.me.uk> wrote in
news:bv76ao$2bv$1@wisteria.csv.warwick.ac.uk:
>
> Marc Bissonnette <dragnet@internalysis.com> wrote:
>> yf110@vtn1.victoria.tc.ca (Malcolm Dew-Jones) wrote in
>> news:4016e666@news.victoria.tc.ca:
>> > OT: that looks bad. NULL is fine for unfinished, that's what it's
>> > for - there's no date yet - but using 1 to indicate 'on hold' is
>> > overloading the meaning of the field. One could imagine wanting to
>> > know _when_ a project was put on hold. (Of course this data layout
>> > may not be up to you.)
>>
>> Well, I've got a field set up for 'date_assigned', so I figured
>> simply re- using the 'date_completed' field for "on hold" or "active"
>> would save an extra column;
>>
>> You do bring up a good point, though - given that status of any
>> project can be changed (brought back to active from 'completed' or
>> 'on hold'), I think I'll add a field that holds a date value for
>> 'date_last_status_change', which would let the users know the 'whens'
>> of all three status possibilities.
>
> Well... what I'd do is have a sub-entity 'status_changes', with a
> record giving the date of and new status after each change. Then you
> can simply get the latest record for a given project to find the
> current status, and the oldest to find when the project was
> initiated. This may be more history than you need to keep, though.
At this point, yeah, it's a little more than I need to keep for it's
current use. I'd like to take the whole project / task manager and package
it up as a commercial tool to make a few bucks on it down the road, which
is when I will end up adding features aimed at a much broader audience than
the current group of colleagues using it right now.
--
Marc Bissonnette
CGI / Database / Web Management Tools: http://www.internalysis.com
Something To Sell? Looking To Buy? http://www.whitewaterclassifieds.ca
Looking for a new ISP? http://www.canadianisp.com
------------------------------
Date: Tue, 27 Jan 2004 19:23:50 GMT
From: Marc Bissonnette <dragnet@internalysis.com>
Subject: Re: Is there a more efficient way to do this ?
Message-Id: <Xns947D927831060dragnetinternalysisc@207.35.177.135>
Uri Guttman <uri@stemsystems.com> wrote in
news:x7znc95hcq.fsf@mail.sysarch.com:
>>>>>> "MB" == Marc Bissonnette <dragnet@internalysis.com> writes:
>
> MB> For printing 'Completed Items' only once, I think this would
> work (and MB> reduces the example code above by 36%!):
>
> MB> # query sorted by date for the following
> MB> while (@row=$sth->fetchrow()) {
> MB> if (length $row[8] == 8 && $completedheader != 1) {
> MB> print "Completed Items:<BR>\n";
> MB> $completedheader=1;
> MB> }
> MB> print "$row[0]<BR>\n";
> MB> }
>
> MB> Thank you - this works *much* better than what I had before!
>
> someone else mentioned buffering and printing. i suggest you always
> buffer your output and print later. it is generally faster (each print
> call is slow) and you can control where the output goes (maybe you
> want the output in mail or a file as well). this is summarized in my
> rule, print rarely, print late.
I can definitely see the logic in this, but what about the worry about
using up too many memory resources ? If you're outputting hundreds or
thousands of lines, would this not affect the other users on the machine
(on a shared webserver, for example) ?
--
Marc Bissonnette
CGI / Database / Web Management Tools: http://www.internalysis.com
Something To Sell? Looking To Buy? http://www.whitewaterclassifieds.ca
Looking for a new ISP? http://www.canadianisp.com
------------------------------
Date: Tue, 27 Jan 2004 19:59:19 GMT
From: Uri Guttman <uri@stemsystems.com>
Subject: Re: Is there a more efficient way to do this ?
Message-Id: <x7smi15fjd.fsf@mail.sysarch.com>
>>>>> "MB" == Marc Bissonnette <dragnet@internalysis.com> writes:
>> someone else mentioned buffering and printing. i suggest you always
>> buffer your output and print later. it is generally faster (each print
>> call is slow) and you can control where the output goes (maybe you
>> want the output in mail or a file as well). this is summarized in my
>> rule, print rarely, print late.
MB> I can definitely see the logic in this, but what about the worry about
MB> using up too many memory resources ? If you're outputting hundreds or
MB> thousands of lines, would this not affect the other users on the machine
MB> (on a shared webserver, for example) ?
well, then you have the classic memory/speed tradeoff. these days memory
is cheap. perl has always leaned towards the speed side of that tradeoff
and uses more memory to get more speed. but note that all your output
will be buffered in stdio and also in the pipe to the server (assuming
true forked cgi scripts) and in the pipe to the browser. so there is
plenty of memory usage all around. finally, when you print a large chunk
and it is going up a pipe, it may just block if it fills up all the
pipe's buffer space. then your script will wait until the server reads
it and frees up that space. so your best bet is to try out various
combinations but i doubt any web page would be spitting out so much text
as to cause serious problems here. who outputs thousands of lines on a
dynamic web page?
uri
--
Uri Guttman ------ uri@stemsystems.com -------- http://www.stemsystems.com
--Perl Consulting, Stem Development, Systems Architecture, Design and Coding-
Search or Offer Perl Jobs ---------------------------- http://jobs.perl.org
------------------------------
Date: Tue, 27 Jan 2004 21:09:42 GMT
From: Marc Bissonnette <dragnet@internalysis.com>
Subject: Re: Is there a more efficient way to do this ?
Message-Id: <Xns947DA46C215Edragnetinternalysisc@207.35.177.134>
Uri Guttman <uri@stemsystems.com> wrote in
news:x7smi15fjd.fsf@mail.sysarch.com:
>>>>>> "MB" == Marc Bissonnette <dragnet@internalysis.com> writes:
>
> >> someone else mentioned buffering and printing. i suggest you
> >> always buffer your output and print later. it is generally faster
> >> (each print call is slow) and you can control where the output
> >> goes (maybe you want the output in mail or a file as well). this
> >> is summarized in my rule, print rarely, print late.
>
> MB> I can definitely see the logic in this, but what about the worry
> about MB> using up too many memory resources ? If you're outputting
> hundreds or MB> thousands of lines, would this not affect the other
> users on the machine MB> (on a shared webserver, for example) ?
>
> well, then you have the classic memory/speed tradeoff. these days
> memory is cheap. perl has always leaned towards the speed side of that
> tradeoff and uses more memory to get more speed. but note that all
> your output will be buffered in stdio and also in the pipe to the
> server (assuming true forked cgi scripts) and in the pipe to the
> browser. so there is plenty of memory usage all around. finally, when
> you print a large chunk and it is going up a pipe, it may just block
> if it fills up all the pipe's buffer space. then your script will wait
> until the server reads it and frees up that space. so your best bet is
> to try out various combinations but i doubt any web page would be
> spitting out so much text as to cause serious problems here. who
> outputs thousands of lines on a dynamic web page?
Admittedly, the cases are few and I tend to buffer everything at least
until any and all transforms and formatting are complete before spitting
out to the client;
There are some cases, though, where a whole ton of data is being spit out
like that: survey data with a few tens of thousands of lines; ordering
histories for online stores, etc. (Obviously, that's not the most
efficient way of doing things, but when a client *insists* on keeping the
whole thing available for viewing and you wanna get paid....)
I think I will take a look at my coding in general, though, to do a shift
towards buffering the bulk of it before printing - you're right in that
it does leave your options open for multiple outputs, which is always a
good thing :)
--
Marc Bissonnette
CGI / Database / Web Management Tools: http://www.internalysis.com
Something To Sell? Looking To Buy? http://www.whitewaterclassifieds.ca
Looking for a new ISP? http://www.canadianisp.com
------------------------------
Date: Wed, 28 Jan 2004 16:29:58 -0500
From: Chris Mattern <syscjm@gwu.edu>
Subject: Re: Is there a more efficient way to do this ?
Message-Id: <401829D6.30909@gwu.edu>
Marc Bissonnette wrote:
> Uri Guttman <uri@stemsystems.com> wrote in
> news:x7znc95hcq.fsf@mail.sysarch.com:
>
>>
>>someone else mentioned buffering and printing. i suggest you always
>>buffer your output and print later. it is generally faster (each print
>>call is slow) and you can control where the output goes (maybe you
>>want the output in mail or a file as well). this is summarized in my
>>rule, print rarely, print late.
>
>
> I can definitely see the logic in this, but what about the worry about
> using up too many memory resources ? If you're outputting hundreds or
> thousands of lines, would this not affect the other users on the machine
> (on a shared webserver, for example) ?
>
A line is 80 bytes. Thousands of lines is therefore hundreds of kilobytes.
If you have hundreds of users, all of whom are getting thousands of lines
at once, that is tens of megabytes. My PC has *five hundred* megabytes.
A serious web server will have gigabytes. Your memory usage from print
buffering is not likely to be an issue.
Chris Mattern
------------------------------
Date: Wed, 28 Jan 2004 22:27:37 GMT
From: Marc Bissonnette <dragnet@internalysis.com>
Subject: Re: Is there a more efficient way to do this ?
Message-Id: <Xns947EB1A2FE012dragnetinternalysisc@207.35.177.135>
Chris Mattern <syscjm@gwu.edu> wrote in news:401829D6.30909@gwu.edu:
> Marc Bissonnette wrote:
>> Uri Guttman <uri@stemsystems.com> wrote in
>> news:x7znc95hcq.fsf@mail.sysarch.com:
>>
>>>
>>>someone else mentioned buffering and printing. i suggest you always
>>>buffer your output and print later. it is generally faster (each
>>>print call is slow) and you can control where the output goes (maybe
>>>you want the output in mail or a file as well). this is summarized in
>>>my rule, print rarely, print late.
>>
>>
>> I can definitely see the logic in this, but what about the worry
>> about using up too many memory resources ? If you're outputting
>> hundreds or thousands of lines, would this not affect the other users
>> on the machine (on a shared webserver, for example) ?
>>
> A line is 80 bytes. Thousands of lines is therefore hundreds of
> kilobytes. If you have hundreds of users, all of whom are getting
> thousands of lines at once, that is tens of megabytes. My PC has
> *five hundred* megabytes. A serious web server will have gigabytes.
> Your memory usage from print buffering is not likely to be an issue.
Sorry, I should have been more specific - this particular application spits
out 'lines' of data to a web browser - usually for import into either Excel
or some other DB, so the byte length is usually quite long.
Still, you're right - it's a smart thing to estimate the average line
length and compare that to a worst/max case scenario for output and base
the judgement on buffering on that number .
--
Marc Bissonnette
CGI / Database / Web Management Tools: http://www.internalysis.com
Something To Sell? Looking To Buy? http://www.whitewaterclassifieds.ca
Looking for a new ISP? http://www.canadianisp.com
------------------------------
Date: 6 Apr 2001 21:33:47 GMT (Last modified)
From: Perl-Users-Request@ruby.oce.orst.edu (Perl-Users-Digest Admin)
Subject: Digest Administrivia (Last modified: 6 Apr 01)
Message-Id: <null>
Administrivia:
#The Perl-Users Digest is a retransmission of the USENET newsgroup
#comp.lang.perl.misc. For subscription or unsubscription requests, send
#the single line:
#
# subscribe perl-users
#or:
# unsubscribe perl-users
#
#to almanac@ruby.oce.orst.edu.
NOTE: due to the current flood of worm email banging on ruby, the smtp
server on ruby has been shut off until further notice.
To submit articles to comp.lang.perl.announce, send your article to
clpa@perl.com.
#To request back copies (available for a week or so), send your request
#to almanac@ruby.oce.orst.edu with the command "send perl-users x.y",
#where x is the volume number and y is the issue number.
#For other requests pertaining to the digest, send mail to
#perl-users-request@ruby.oce.orst.edu. Do not waste your time or mine
#sending perl questions to the -request address, I don't have time to
#answer them even if I did know the answer.
------------------------------
End of Perl-Users Digest V10 Issue 6016
***************************************
