[19476] in Perl-Users-Digest
Perl-Users Digest, Issue: 1671 Volume: 10
daemon@ATHENA.MIT.EDU (Perl-Users Digest)
Fri Aug 31 18:05:56 2001
Date: Fri, 31 Aug 2001 15:05:12 -0700 (PDT)
From: Perl-Users Digest <Perl-Users-Request@ruby.OCE.ORST.EDU>
To: Perl-Users@ruby.OCE.ORST.EDU (Perl-Users Digest)
Message-Id: <999295511-v10-i1671@ruby.oce.orst.edu>
Content-Type: text
Perl-Users Digest Fri, 31 Aug 2001 Volume: 10 Number: 1671
Today's topics:
Re: A better way (Hardy Merrill)
Re: can this perl script be more elegant/shorter ? (Tim)
Re: can this perl script be more elegant/shorter ? <ren@tivoli.com>
Re: can this perl script be more elegant/shorter ? (Tad McClellan)
Re: can this perl script be more elegant/shorter ? <krahnj@acm.org>
Re: Dangerous Perl Script? (Malcolm Dew-Jones)
Re: Dangerous Perl Script? <Doug.King@abh.siemens.com>
Re: Dangerous Perl Script? (Malcolm Dew-Jones)
Re: Environment variables and AOL (G.A.D.Miles)
Re: File::Find not recursing on Win32 Perl 5.001 <ren@tivoli.com>
Re: formatted ouput <jprok@snet.net>
How to copy files in Perl and more <radiotito@yahoo.com>
Re: How to copy files in Perl and more <jurgenex@hotmail.com>
Re: Java mucks up split <cberry@cinenet.net>
Re: Long on Sun? (Mark Jason Dominus)
Re: Long on Sun? (Mark Jason Dominus)
Re: Long on Sun? <jwilson2000@home.com>
Re: Long on Sun? (Mark Jason Dominus)
Re: Long on Sun? <ren@tivoli.com>
Re: Long on Sun? <comdog@panix.com>
Re: multithreaded LWP (Michel Dalle)
Re: multithreaded LWP <citykid@nospam.edu>
Re: multithreaded LWP (Rocco Caputo)
Re: Net::FTP question (David Krainess)
Re: Perl 5.6.1 and PerlCC (B module) bombs using tie. <djberge@uswest.com>
Perl Discussion (Hal Lesesne)
Re: Perl in Non-IT Work Environments <melorama@nospam.gov>
Digest Administrivia (Last modified: 6 Apr 01) (Perl-Users-Digest Admin)
----------------------------------------------------------------------
Date: 31 Aug 2001 12:57:56 -0700
From: merrill@missioncriticallinux.com (Hardy Merrill)
Subject: Re: A better way
Message-Id: <fdfedf4b.0108311157.471d22bd@posting.google.com>
"Jonathan Clover" <jclover@nati.org> wrote in message news:<tov8vu1bmc4maf@corp.supernews.com>...
> I have this code to insert a new row into a database and I would like to
> make the code, that's focused on(!) more concise, but am unsure on how.
> Ideas anyone.
> Jonathan Clover
> NATI
>
> P.S. @question will always and only contain 10 values
>
> my $dbh = DBI->connect("DBI:CSV:f_dir=$database_dir")
> or diehtml("Cannot connect: " . $DBI::errstr);
>
> ! $dbh->do( "insert into test_ans values
> ('$test','$questions[0]','$questions[1]','$questions[2]','$questions[3]','$q
> uestions[4]','$questions[5]','$questions[6]','$questions[7]','$questions[8]'
> ,'$questions[9]')" )
> ! or diehtml("Cannot do: " . $dbh->errstr());
>
> $dbh->disconnect();
I haven't used CSV, but assuming CSV can use placeholders, I would
suggest using placeholders. On *nix read up on placeholders by doing
"perldoc DBI" and searching(using the slash "/" operator) for
"placeholder". You should also read the DBD::CSV perldocs to see what
DBD::CSV supports.
Using placeholders your code would looks something like this:
my $sth = $dbh->prepare(qq{
insert into test_ans
values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
});
$rows_affected = $sth->execute($test, @questions[0..9]);
BTW, to use placeholders you'll need to change the "do" to a "prepare
and execute". Also, read up on error checking/trapping and
RaiseError(perldoc DBI).
------------------------------
Date: 31 Aug 2001 11:50:20 -0700
From: tvn007@hotmail.com (Tim)
Subject: Re: can this perl script be more elegant/shorter ?
Message-Id: <21724be2.0108311050.6bf86adf@posting.google.com>
Hi John,
Thanks for your post.
I tried to run your code . However,
I encountered syntax error on the line below:
> print OUTFILE $chain[$_][$i] for 1 .. 7;
Could you please again help ?
Thanks
"John W. Krahn" <krahnj@acm.org> wrote in message news:<3B8F4227.BB73F149@acm.org>...
> "John W. Krahn" wrote:
> >
> > Tim wrote:
> > >
> > > Hi,
> > >
> > > Could some one help me to make this script more elegant and shorter ?
> > >
> > > Here is my PERL script:
> > >
> > > [snip]
> >
> > This should get you started.
> >
> > [snip]
>
> Even shorter.
>
>
> #!/usr/local/bin/perl5 -w
> use strict;
>
> $/ = ';';
> my $input1 = 'chaintest.scan';
> my $output2 = 'OUT.out';
> my $ct_scanout;
> my @chain;
>
> open INFILE, "< $input1" or die "cannot open $input1: $!";
> open OUTFILE, "> $output2" or die "cannot open $output2: $!";
>
> while ( <INFILE> ) {
> next unless /CHAIN_TEST/ .. /^SCAN_CELLS/;
> $ct_scanout = 0 if /apply\s*"grp[0-9]_load"/;
> $ct_scanout = 1 if /apply\s*"grp[0-9]_unload"/;
>
> if ( /chain\s+"chain([0-9])"\s*=\s*"([^"]*)"/ ) {
> my $chain_number = $1;
> $_ = $2;
>
> tr/\\\n\t //d;
> if ( $ct_scanout ) {
> tr/01/LH/;
> }
> else {
> tr/X/0/;
> }
>
> @{ $chain[ $chain_number ] } = split //;
> if ( $chain_number == 7 ) {
> for ( my $i = 0; $i < @{ $chain[ $chain_number ] }; $i++ ) {
> print OUTFILE "\n(", $ct_scanout ? 'ct_so ' : 'ct_si ';
> print OUTFILE $chain[$_][$i] for 1 .. 7;
> print OUTFILE ' )';
> }
> }
> }
> }
>
> __END__
>
>
>
> John
------------------------------
Date: 31 Aug 2001 15:01:45 -0500
From: Ren Maddox <ren@tivoli.com>
Subject: Re: can this perl script be more elegant/shorter ?
Message-Id: <m3d75ct2vq.fsf@dhcp9-161.support.tivoli.com>
On 31 Aug 2001, tvn007@hotmail.com wrote:
> I encountered syntax error on the line below:
>
>"John W. Krahn" <krahnj@acm.org> wrote in message news:<3B8F4227.BB73F149@acm.org>...
>
>> print OUTFILE $chain[$_][$i] for 1 .. 7;
Older versions of Perl did not support the "for" statement modifier.
Looks like you probably have one of those. Seems like that was added
with 5.005, IIRC.
--
Ren Maddox
ren@tivoli.com
------------------------------
Date: Fri, 31 Aug 2001 15:46:50 -0400
From: tadmc@augustmail.com (Tad McClellan)
Subject: Re: can this perl script be more elegant/shorter ?
Message-Id: <slrn9ovqda.4vd.tadmc@tadmc26.august.net>
Tim <tvn007@hotmail.com> wrote:
>I tried to run your code . However,
>I encountered syntax error on the line below:
Errr, was there an error message?
Error messages are for helping solve errors. If you want us to
help, we need to know exactly what the message said.
>> print OUTFILE $chain[$_][$i] for 1 .. 7;
There is no syntax error in that line, your problem is elsewhere.
[snip backwards-quoted text]
--
Tad McClellan SGML consulting
tadmc@augustmail.com Perl programming
Fort Worth, Texas
------------------------------
Date: Fri, 31 Aug 2001 21:54:23 GMT
From: "John W. Krahn" <krahnj@acm.org>
Subject: Re: can this perl script be more elegant/shorter ?
Message-Id: <3B900804.90B885EF@acm.org>
Tim wrote:
>
> Hi John,
>
> Thanks for your post.
>
> I tried to run your code . However,
> I encountered syntax error on the line below:
>
> > print OUTFILE $chain[$_][$i] for 1 .. 7;
>
> Could you please again help ?
[Please don't top-post.]
As Ren mentioned you must be running an older version of perl so change
that line to:
for ( 1 .. 7 ) { print OUTFILE $chain[$_][$i] }
John
--
use Perl;
program
fulfillment
------------------------------
Date: 31 Aug 2001 11:58:43 -0800
From: yf110@vtn1.victoria.tc.ca (Malcolm Dew-Jones)
Subject: Re: Dangerous Perl Script?
Message-Id: <3b8fde63@news.victoria.tc.ca>
Mark Jason Dominus (mjd@plover.com) wrote:
: This means you are letting the remote web user run a shell command, as
: if they had typed it at the terminal.
: If you want to let anyone in the entire world run commands on your
: computer, then why do you have passwords? It would be simpler to just
: have a public guest account with no password.
That depends on the server security configuration.
If the server restricts access to this script to a single IP address,
which you own, then you are controlling access to the script, and it
does not present the danger you say it does.
Also, the script runs as the web account user. If that account has very
limited access then the script has the same limited access, in particular
it does not necessarily have the ability view or list arbitrary files, or
to write or delete files to which it does have access. (Rather like
anonymous ftp, which could in theory allow someone to rm /*, but in
practise never does).
I wouldn't install the script though. Any such script should provide some
security of its own, such as a password, or at least checking that the
remote user had to do a login to access it.
Also, any such script should only be in place temporarily, installed while
you use it and then removed.
------------------------------
Date: Fri, 31 Aug 2001 15:13:09 -0400
From: Doug King <Doug.King@abh.siemens.com>
Subject: Re: Dangerous Perl Script?
Message-Id: <3B8FE1C5.1284AE48@abh.siemens.com>
Malcolm Dew-Jones wrote:
>
> Mark Jason Dominus (mjd@plover.com) wrote:
>
> : This means you are letting the remote web user run a shell command, as
> : if they had typed it at the terminal.
>
> : If you want to let anyone in the entire world run commands on your
> : computer, then why do you have passwords? It would be simpler to just
> : have a public guest account with no password.
>
> That depends on the server security configuration.
>
> If the server restricts access to this script to a single IP address,
> which you own, then you are controlling access to the script, and it
> does not present the danger you say it does.
IP addresses can be faked, so I would not rely on that for security
<snip>
------------------------------
Date: 31 Aug 2001 13:33:12 -0800
From: yf110@vtn1.victoria.tc.ca (Malcolm Dew-Jones)
Subject: Re: Dangerous Perl Script?
Message-Id: <3b8ff488@news.victoria.tc.ca>
Doug King (Doug.King@abh.siemens.com) wrote:
: Malcolm Dew-Jones wrote:
: >
: > Mark Jason Dominus (mjd@plover.com) wrote:
: >
: > : This means you are letting the remote web user run a shell command, as
: > : if they had typed it at the terminal.
: >
: > : If you want to let anyone in the entire world run commands on your
: > : computer, then why do you have passwords? It would be simpler to just
: > : have a public guest account with no password.
: >
: > That depends on the server security configuration.
: >
: > If the server restricts access to this script to a single IP address,
: > which you own, then you are controlling access to the script, and it
: > does not present the danger you say it does.
: IP addresses can be faked, so I would not rely on that for security
Fine, so suppose you run a dual server using two ports, and the
non-standard port is blocked. Only the second server allows access to
various items.
Or you could be using https to ensure yourself of the users authenticity,
or a VPN, or ...
The point is, the security or lack thereof also depends very much on the
server security configuration. You can make such a script secure if you
wish.
------------------------------
Date: Fri, 31 Aug 2001 17:11:10 GMT
From: usad1@gadnet.com (G.A.D.Miles)
Subject: Re: Environment variables and AOL
Message-Id: <3b8fc42b.199552185@news.newsguy.com>
>And just in case you still are not 100% clear on this, allow me to
>reiterate that this has nothing whatever, in the least, to do with
>Perl in any way shape or form.
Since it's a Perl program I'm working on, and since so much CGI is in
Perl, I thought at least someone here may have some sound advice. And
this time, unlike last, I've got it. So thanks all.
------------------------------
Date: 31 Aug 2001 12:36:49 -0500
From: Ren Maddox <ren@tivoli.com>
Subject: Re: File::Find not recursing on Win32 Perl 5.001
Message-Id: <m3pu9ct9la.fsf@dhcp9-161.support.tivoli.com>
On Thu, 30 Aug 2001, bart.lateur@skynet.be wrote:
> Phil Hibbs wrote:
>
>>> What version of perl is it? I tried it on 5.6.1 on 2000 Pro.
>>
>>It's 5.001 as per the subject line. And no, I can't upgrade it, much
>>though I'd like to.
>
> You still using Win3.1, or NT3.51? If no, why not?
>
> Funny how people find it natural to upgrade their soft if it's from
> Microsoft, but not from anything else.
I know of plenty of businesses that still have Win95 (Rev B) systems.
Not Win3.1 or NT3.51, but still about as old as Perl 5.001.
--
Ren Maddox
ren@tivoli.com
------------------------------
Date: Fri, 31 Aug 2001 17:30:31 -0400
From: john prokopek <jprok@snet.net>
Subject: Re: formatted ouput
Message-Id: <3B9001F7.6DD63184@snet.net>
exactly.
oh well. just a thought.
Bernard El-Hagin wrote:
>
> On 29 Aug 2001 13:49:05 GMT, Anno Siegel <anno4000@lublin.zrz.tu-berlin.de>
> wrote:
> >According to jprok <john.prokopek@ubsw.com>:
> >> can "formats" be used for input as well as output?
> >
> >Formats are an output-only affair. So much so that I can't begin to
> >guess what you even mean by "using formats for input".
>
> I think he wants something like scanf(3) in C.
>
> Cheers,
> Bernard
> --
> perl -l54e's yyw q q tvmrx "h\ywx ersxliv zivp legoiv"qiy;y #a-zA-Z#d-gu-z#
> chefghijklmnopqrstuvwxyzcJab-def-uPwxyzc;s j j s u u s t t s r r s
> ppevalpereeteueje'
--
John D. Prokopek
jprokopek@snet.net
------------------------------
Date: Fri, 31 Aug 2001 23:41:23 +0200
From: Valentin 30IR976 <radiotito@yahoo.com>
Subject: How to copy files in Perl and more
Message-Id: <3B900483.AF88D73D@yahoo.com>
How could I copy one file to a directory? I uses ActiveState Perl under
NT
I have tried:
$mydirectory="c:\midir";
`copy myfile.txt $mydirectory`;
and it doesnt work...
Are there any other function like copy to copy files?
And also another question...
I would like to execute the command "dir" of NT, from my Perl program. I
have written
`dir`;
but doesnt work.
But if I write:
$command=`dir`;
print "$command";
it works...
Whats going on?
Thanks in advance from newbie Valentin
------------------------------
Date: Fri, 31 Aug 2001 15:01:36 -0700
From: "Jürgen Exner" <jurgenex@hotmail.com>
Subject: Re: How to copy files in Perl and more
Message-Id: <3b90094a$1@news.microsoft.com>
"Valentin 30IR976" <radiotito@yahoo.com> wrote in message
news:3B900483.AF88D73D@yahoo.com...
> How could I copy one file to a directory? I uses ActiveState Perl under
> NT
perldoc -m File::copy
------------------------------
Date: Fri, 31 Aug 2001 18:17:45 -0000
From: Craig Berry <cberry@cinenet.net>
Subject: Re: Java mucks up split
Message-Id: <Xns910E72E8A837Ecberrycinenetnet1@207.126.101.92>
trewth_seeker@yahoo.com (Trewth Seeker) wrote in
news:d690a633.0108302226.117c0f1e@posting.google.com:
> abigail@foad.org (Abigail) wrote in message
> news:<slrn9op2u6.sp9.abigail@alexandra.xs4all.nl>...
>> Is Java's split also "broken" because Java doesn't have a $* variable?
>
> Are you a sophist for pay or just for pleasure?
No strawman is safe when Abigail goes to battle. :)
--
Craig Berry <http://www.cinenet.net/~cberry/>
"That which is now known, was once only imagined." - William Blake
------------------------------
Date: Fri, 31 Aug 2001 18:27:30 GMT
From: mjd@plover.com (Mark Jason Dominus)
Subject: Re: Long on Sun?
Message-Id: <3b8fd712.2d61$e4@news.op.net>
In article <12Qj7.22393$MK5.14623469@news1.sttln1.wa.home.com>,
Jeff Wilson <jwilson2000@home.com> wrote:
>Does anyone know if packing/unpacking a long on a Sun Solaris machine uses
>32 or 64 bits?
Perl knows. Perhaps you could ask Perl.
perl -le '$p = pack "L", 0; print length 8*$p'
--
@P=split//,".URRUU\c8R";@d=split//,"\nrekcah xinU / lreP rehtona tsuJ";sub p{
@p{"r$p","u$p"}=(P,P);pipe"r$p","u$p";++$p;($q*=2)+=$f=!fork;map{$P=$P[$f^ord
($p{$_})&6];$p{$_}=/ ^$P/ix?$P:close$_}keys%p}p;p;p;p;p;map{$p{$_}=~/^[P.]/&&
close$_}%p;wait until$?;map{/^r/&&<$_>}%p;$_=$d[$q];sleep rand(2)if/\S/;print
------------------------------
Date: Fri, 31 Aug 2001 18:35:02 GMT
From: mjd@plover.com (Mark Jason Dominus)
Subject: Re: Long on Sun?
Message-Id: <3b8fd8d5.2d89$22d@news.op.net>
In article <3b8fd712.2d61$e4@news.op.net>,
Mark Jason Dominus <mjd@plover.com> wrote:
>In article <12Qj7.22393$MK5.14623469@news1.sttln1.wa.home.com>,
>Jeff Wilson <jwilson2000@home.com> wrote:
>>Does anyone know if packing/unpacking a long on a Sun Solaris machine uses
>>32 or 64 bits?
>
>Perl knows. Perhaps you could ask Perl.
>
> perl -le '$p = pack "L", 0; print length 8*$p'
This should have been
print 8 * length $p
of course.
--
@P=split//,".URRUU\c8R";@d=split//,"\nrekcah xinU / lreP rehtona tsuJ";sub p{
@p{"r$p","u$p"}=(P,P);pipe"r$p","u$p";++$p;($q*=2)+=$f=!fork;map{$P=$P[$f^ord
($p{$_})&6];$p{$_}=/ ^$P/ix?$P:close$_}keys%p}p;p;p;p;p;map{$p{$_}=~/^[P.]/&&
close$_}%p;wait until$?;map{/^r/&&<$_>}%p;$_=$d[$q];sleep rand(2)if/\S/;print
------------------------------
Date: Fri, 31 Aug 2001 18:39:35 GMT
From: "Jeff Wilson" <jwilson2000@home.com>
Subject: Re: Long on Sun?
Message-Id: <HRQj7.22551$MK5.14688095@news1.sttln1.wa.home.com>
l-sc1% perl -le '$p = pack "L", 0; print length 8*$p'
1
Can you interpret what this result means?
"Mark Jason Dominus" <mjd@plover.com> wrote in message
news:3b8fd712.2d61$e4@news.op.net...
> In article <12Qj7.22393$MK5.14623469@news1.sttln1.wa.home.com>,
> Jeff Wilson <jwilson2000@home.com> wrote:
> >Does anyone know if packing/unpacking a long on a Sun Solaris machine
uses
> >32 or 64 bits?
>
> Perl knows. Perhaps you could ask Perl.
>
> perl -le '$p = pack "L", 0; print length 8*$p'
>
>
> --
> @P=split//,".URRUU\c8R";@d=split//,"\nrekcah xinU / lreP rehtona tsuJ";sub
p{
>
@p{"r$p","u$p"}=(P,P);pipe"r$p","u$p";++$p;($q*=2)+=$f=!fork;map{$P=$P[$f^or
d
> ($p{$_})&6];$p{$_}=/
^$P/ix?$P:close$_}keys%p}p;p;p;p;p;map{$p{$_}=~/^[P.]/&&
> close$_}%p;wait until$?;map{/^r/&&<$_>}%p;$_=$d[$q];sleep
rand(2)if/\S/;print
------------------------------
Date: Fri, 31 Aug 2001 18:43:00 GMT
From: mjd@plover.com (Mark Jason Dominus)
Subject: Re: Long on Sun?
Message-Id: <3b8fdab4.2dcc$d4@news.op.net>
In article <HRQj7.22551$MK5.14688095@news1.sttln1.wa.home.com>,
Jeff Wilson <jwilson2000@home.com> wrote:
>l-sc1% perl -le '$p = pack "L", 0; print length 8*$p'
>1
>
>Can you interpret what this result means?
It appears to mean:
1. I made a mistake in the example
2. You pasted code without understanding what it was doing
--
@P=split//,".URRUU\c8R";@d=split//,"\nrekcah xinU / lreP rehtona tsuJ";sub p{
@p{"r$p","u$p"}=(P,P);pipe"r$p","u$p";++$p;($q*=2)+=$f=!fork;map{$P=$P[$f^ord
($p{$_})&6];$p{$_}=/ ^$P/ix?$P:close$_}keys%p}p;p;p;p;p;map{$p{$_}=~/^[P.]/&&
close$_}%p;wait until$?;map{/^r/&&<$_>}%p;$_=$d[$q];sleep rand(2)if/\S/;print
------------------------------
Date: 31 Aug 2001 15:07:02 -0500
From: Ren Maddox <ren@tivoli.com>
Subject: Re: Long on Sun?
Message-Id: <m38zg0t2mx.fsf@dhcp9-161.support.tivoli.com>
On Fri, 31 Aug 2001, jwilson2000@home.com wrote:
> l-sc1% perl -le '$p = pack "L", 0; print length 8*$p'
> 1
Try this instead:
perl -le print 'print 8 * length pack "L", 0'
--
Ren Maddox
ren@tivoli.com
------------------------------
Date: Fri, 31 Aug 2001 17:12:56 -0400
From: brian d foy <comdog@panix.com>
Subject: Re: Long on Sun?
Message-Id: <comdog-D6F7CC.17125631082001@news.panix.com>
In article <HRQj7.22551$MK5.14688095@news1.sttln1.wa.home.com>, "Jeff
Wilson" <jwilson2000@home.com> wrote:
> l-sc1% perl -le '$p = pack "L", 0; print length 8*$p'
> 1
>
> Can you interpret what this result means?
look at the intermediate result ($p). try it on machines that
have different sizes for a long.
good luck ;)
--
brian d foy <comdog@panix.com> - Perl services for hire
CGI Meta FAQ - http://www.perl.org/CGI_MetaFAQ.html
Troubleshooting CGI scripts - http://www.perl.org/troubleshooting_CGI.html
------------------------------
Date: Fri, 31 Aug 2001 18:12:33 GMT
From: news@mikespub.net (Michel Dalle)
Subject: Re: multithreaded LWP
Message-Id: <9mokfo$f5l$1@dackel.pdb.sbs.de>
In article <Pine.LNX.4.33.0108311009190.30407-100000@schewanella.stanford.edu>, Les Ander <citykid@nospam.edu> wrote:
>Hi,
>I have a question regarding threads. I have never used them, but
>I have a problem that needs me to use them.
>
>The problem is as follows:
>-------------------------
>I need to send about 6000 request to the
>server that takes about 3 minute to process every request. They give
>me an ID for every request, so I can retrieve the results within 24 hours.
[snip logic]
You have 2 types of requests here, not 3 :
1) submit 'job' and get back an ID
2) retrieve 'job results' based on ID
Do you know if those 'jobs' are sent to some queue at the other end
before they are processed, or if they are executed (in background)
directly after being received ?
If the job requests are being queued somewhere for execution later
on, you can presumably submit as many jobs one after the other as
you like, but if they're executed right after you submit them, you'll
probably kill the server - not such a good idea :)
A related issue is whether you can rely on the 'job results' being
available exactly 3 minutes after you submit them, or whether they
will be available somewhere 'within the next 24 hours'.
You'll need to ask some question of what happens on the other
end before changing the way you submit your jobs, I'm afraid...
Michel.
--
Welcome to Mike's Pub
http://mikespub.net/forum/
------------------------------
Date: Fri, 31 Aug 2001 11:43:49 -0700
From: Les Ander <citykid@nospam.edu>
Subject: Re: multithreaded LWP
Message-Id: <Pine.LNX.4.33.0108311138130.30726-100000@schewanella.stanford.edu>
Hi Michel, thanks for replying
> >The problem is as follows:
> >-------------------------
> >I need to send about 6000 request to the
> >server that takes about 3 minute to process every request. They give
> >me an ID for every request, so I can retrieve the results within 24 hours.
>
>
> 1) submit 'job' and get back an ID
> 2) retrieve 'job results' based on ID
>
> Do you know if those 'jobs' are sent to some queue at the other end
> before they are processed, or if they are executed (in background)
> directly after being received ?
these jobs are sent to a queue.
>
> A related issue is whether you can rely on the 'job results' being
> available exactly 3 minutes after you submit them, or whether they
> will be available somewhere 'within the next 24 hours'.
Actually, with the ID they tell me approximately how long it will take
for the job to be processed. I said 3 minutes because the kind of job
I am sending takes around 2 minutes during peak hours.
> You'll need to ask some question of what happens on the other
> end before changing the way you submit your jobs, I'm afraid...
>
> Michel.
Do you think I can use threads to spead this up?
les
------------------------------
Date: 31 Aug 2001 19:46:56 GMT
From: troc@netrus.net (Rocco Caputo)
Subject: Re: multithreaded LWP
Message-Id: <slrn9ovqdc.11u7.troc@eyrie.homenet>
On Fri, 31 Aug 2001 10:23:42 -0700, Les Ander <citykid@nospam.edu> wrote:
>
[...]
>So this is what I want to do:
>----------------------------
>currently I am sending them one request at a time, parsing out the ID,
>waiting for 3 minutes and then retrieving the results with the ID, and
>then repeating this cycle. But this will take forever if I do this
>for 6000 requests. So instead I want to send them requests every about 15
>seconds or so, store the ID and then when all the requests are done, I
>will wait till the trafic is low before I request the results in batch or
>every 15 seconds. I think this would be lot more efficient.
Here's something that might work.
-- Rocco Caputo / troc@netrus.net / poe.perl.org / poe.sourceforge.net
#!/usr/bin/perl -w
use strict;
use lib '/home/troc/perl/poe';
use POE;
### Control parameters.
# How many seconds to wait for a job to complete. Make it 180 for
# three minutes.
sub SECONDS_TO_WAIT_FOR_RESPONSE () { rand(5) }
# How many requests to wait for at any given time. Increase it to hit
# the server harder; decrease it to be friendlier.
sub MAXIMUM_REQUESTS_AT_A_TIME () { 15 }
### Given functions. These were supplied in the original post. They
### have been given dummy implementations so the program can display
### interesting output.
sub make_initial_request {
my $job_parameters = shift;
return "(initial response for $job_parameters)";
}
sub parse_id {
my $initial_response = shift;
return $1 if $initial_response =~ /for (\d+)/;
return "(unknown)";
}
sub get_results_id {
my $request_id = shift;
return "(results for $request_id)";
}
# This is an extra function to fetch the information for the next
# request. For testing, it just returns the next number in a
# sequence.
my $request_index = 0;
sub TOTAL_REQUESTS_TO_RUN () { 100 }
sub fetch_next_request {
if ($request_index < TOTAL_REQUESTS_TO_RUN) {
return ++$request_index;
}
return undef;
}
### Queen session. This manages the drones, spawning new ones to
### replace old ones for as long as there are requests to process.
# Spawn the queen session.
sub spawn_a_queen {
POE::Session->create
( inline_states =>
{ _start => \&queen_start,
_stop => \&queen_stop,
_child => \&queen_child,
spawn_new_drones => \&queen_spawn_new_drones,
}
);
}
# Event handler: Start the queen session once it's been spawned.
sub queen_start {
my ($kernel, $heap) = @_[KERNEL, HEAP];
$heap->{drones_alive} = 0;
$kernel->yield( 'spawn_new_drones' );
print "Queen started.\n";
}
# Event handler: Spawn new drones as necessary.
sub queen_spawn_new_drones {
my ($kernel, $heap) = @_[KERNEL, HEAP];
while ($heap->{drones_alive} < MAXIMUM_REQUESTS_AT_A_TIME) {
my $next_request_parameters = fetch_next_request();
last unless defined $next_request_parameters;
spawn_a_drone($next_request_parameters);
$heap->{drones_alive}++;
print "Queen started a drone (now $heap->{drones_alive}).\n";
}
}
# Event handler: A child session (drone) has come or gone.
sub queen_child {
my ($kernel, $heap, $operation) = @_[KERNEL, HEAP, ARG0];
if ($operation eq 'lose') {
$heap->{drones_alive}--;
print "Queen lost a drone (now $heap->{drones_alive}).\n";
$kernel->yield( 'spawn_new_drones' );
}
}
# Event handler: Last minute cleanup after the queen has stopped.
sub queen_stop {
print "Queen stopped.\n";
}
### Drone/worker session. Instances of this do the work of each
### request.
# Spawn a drone session. Pass the job parameters to it.
sub spawn_a_drone {
my $job_parameters = shift;
POE::Session->create
( inline_states =>
{ _start => \&drone_start,
_stop => \&drone_stop,
delay_done => \&drone_delay_done,
},
args => [ $job_parameters ],
);
}
# Event handler: Start the drone's job once it's been spawned.
sub drone_start {
my ($kernel, $heap, $job_parameters) = @_[KERNEL, HEAP, ARG0];
$heap->{job_parameters} = $job_parameters;
my $initial_response = make_initial_request($job_parameters);
my $request_id = parse_id($initial_response);
$heap->{request_id} = $request_id;
$kernel->delay( delay_done => SECONDS_TO_WAIT_FOR_RESPONSE );
my $session_id = $_[SESSION]->ID;
print "Drone $session_id started.\n";
}
# Event handler: Last minute cleanup after the drone has stopped.
sub drone_stop {
my $session_id = $_[SESSION]->ID;
print "Drone $session_id stopped.\n";
}
# Event handler: The drone has waited long enough for its initial
# request to be processed. Run the second request to fetch the job's
# results.
sub drone_delay_done {
my $heap = $_[HEAP];
my $session_id = $_[SESSION]->ID;
print "Drone $session_id waited long enough.\n";
my $results = get_results_id( $heap->{request_id} );
print( "Drone $session_id job ",
$heap->{job_parameters},
" finished: $results\n"
);
}
### Main loop. Spawn a single queen session, and run POE's event loop
### until everything is done.
spawn_a_queen();
$poe_kernel->run();
exit(0);
------------------------------
Date: Fri, 31 Aug 2001 21:13:00 GMT
From: davidkrainess@yahoo.com (David Krainess)
Subject: Re: Net::FTP question
Message-Id: <910E958DCdavidkrainessyahooco@38.9.69.2>
Please Anybody???
------------------------------
Date: Fri, 31 Aug 2001 14:08:08 -0500
From: Mr Sunblade <djberge@uswest.com>
Subject: Re: Perl 5.6.1 and PerlCC (B module) bombs using tie.
Message-Id: <3B8FE098.6DC032CE@uswest.com>
Ilya Martynov wrote:
<snip>
>
> AFAIK PerlCC is not stable enough yet. I never seen it work with any
> Perl application I've wrote which is more complex than 'Hello, world'.
>
My sentiments exactly. On the other hand, I had pretty good luck with
ActiveState's PerlApp (using Tk, no less).
Regards,
Mr. Sunblade
--
"Evil will always triumph because Good is *dumb*."
-- Dark Helmet, 'Spaceballs: The Movie'
------------------------------
Date: 31 Aug 2001 13:52:39 -0700
From: hal_lesesne@yahoo.com (Hal Lesesne)
Subject: Perl Discussion
Message-Id: <e587126.0108311252.5a985c3@posting.google.com>
I hope you don't consider this an ad, but I am trying to start a new
discussion forum for all things web development. I hope to ultimately
turn the site into something of real value, but for now it is nothing
more than a shell.
Please take a look - there are no advertisements and I am not
currently making or planning to make any money off the thing. I just
thought it might become a good resource for developers of all types.
Any suggestions or comments would be greatly appreciated.
You can find the site at http://www.hostfind.net
Thanks,
Hal
------------------------------
Date: Fri, 31 Aug 2001 19:12:42 GMT
From: Mel Matsuoka <melorama@nospam.gov>
Subject: Re: Perl in Non-IT Work Environments
Message-Id: <46ovotcb82uaqd62m3rhc0ub2fvnkfhij0@4ax.com>
On 28 Aug 2001 00:09:03 GMT, "James E Keenan" <jkeen@concentric.net> wrote:
>I would like to hear from people who are employed in jobs where their
>official responsibilities are *not* primarily IT-related and who use Perl on
>the job.
>
>I'm not paid as a programmer or technologist. My employers know nothing
>about my use of Perl. But being able to extract information from databases
>and reformat it with Perl has been crucial to my success. Are their other
>people out there like me?
I work as an editor and all-around-geek at a Video/Media production company, and
use Perl "behind the scenes" to interface with internal MySQL and Microsoft
Access databases which I have developed for asset managment and tape logging
purposes. I have also used Perl on several occasions to massage and getting data
out of EDL's (Edit Decision Lists) which have been generated by other video
editing systems for use with our system.
My use of Perl at work actually pre-dates my current un-official role as
Director of New Media Productions, and (not surprisingly) was one of the main
reasons why I basically "promoted" myself to my new, exalted code-geek status :)
It has enabled my company to easily diversify our services to include webdesign
and web application development for our exisiting clients, as well as generate
new, non video-production oriented business.
Thankfully, since my company is a small, non IT related business, I didn't have
to deal with any objections to open-source solutions by PHB's.
Aloha,
mel
------------------------------
Date: 6 Apr 2001 21:33:47 GMT (Last modified)
From: Perl-Users-Request@ruby.oce.orst.edu (Perl-Users-Digest Admin)
Subject: Digest Administrivia (Last modified: 6 Apr 01)
Message-Id: <null>
Administrivia:
The Perl-Users Digest is a retransmission of the USENET newsgroup
comp.lang.perl.misc. For subscription or unsubscription requests, send
the single line:
subscribe perl-users
or:
unsubscribe perl-users
to almanac@ruby.oce.orst.edu.
To submit articles to comp.lang.perl.announce, send your article to
clpa@perl.com.
To request back copies (available for a week or so), send your request
to almanac@ruby.oce.orst.edu with the command "send perl-users x.y",
where x is the volume number and y is the issue number.
For other requests pertaining to the digest, send mail to
perl-users-request@ruby.oce.orst.edu. Do not waste your time or mine
sending perl questions to the -request address, I don't have time to
answer them even if I did know the answer.
------------------------------
End of Perl-Users Digest V10 Issue 1671
***************************************
