[30149] in Perl-Users-Digest
Perl-Users Digest, Issue: 1392 Volume: 11
daemon@ATHENA.MIT.EDU (Perl-Users Digest)
Tue Mar 25 18:14:26 2008
Date: Tue, 25 Mar 2008 15:14:18 -0700 (PDT)
From: Perl-Users Digest <Perl-Users-Request@ruby.OCE.ORST.EDU>
To: Perl-Users@ruby.OCE.ORST.EDU (Perl-Users Digest)
Perl-Users Digest Tue, 25 Mar 2008 Volume: 11 Number: 1392
Today's topics:
Re: Readline using foreach and while <benkasminbullock@gmail.com>
Re: Readline using foreach and while <ben@morrow.me.uk>
Re: Readline using foreach and while <devnull4711@web.de>
Re: Readline using foreach and while <simon.chao@fmr.com>
Re: Readline using foreach and while <simon.chao@fmr.com>
Re: Readline using foreach and while <ben@morrow.me.uk>
Re: Readline using foreach and while <someone@example.com>
Re: Readline using foreach and while <szrRE@szromanMO.comVE>
Re: Readline using foreach and while <someone@example.com>
Re: Readline using foreach and while <ced@blv-sam-01.ca.boeing.com>
Re: strategys other than subroutine and OO? sheinrich@my-deja.com
Re: The huge amount response data problem <jimsgibson@gmail.com>
Re: The huge amount response data problem xhoster@gmail.com
Re: The huge amount response data problem xhoster@gmail.com
Digest Administrivia (Last modified: 6 Apr 01) (Perl-Users-Digest Admin)
----------------------------------------------------------------------
Date: Tue, 25 Mar 2008 15:30:35 +0000 (UTC)
From: Ben Bullock <benkasminbullock@gmail.com>
Subject: Re: Readline using foreach and while
Message-Id: <fsb5qr$7a6$1@ml.accsnet.ne.jp>
On Tue, 25 Mar 2008 11:59:23 +0000, John W. Krahn wrote:
> Ben Bullock wrote:
>> The foreach version seems to first read the whole of the file into an
>> array, and then go through it line by line:
>
> perldoc -q "What is the difference between a list and an array"
Found in /usr/local/lib/perl5/5.10.0/pod/perlfaq4.pod
What is the difference between a list and an array?
An array has a changeable length. A list does not.
If I had written "the foreach version reads the whole of the file into a
list", I would have contradicted this FAQ entry, which says I can't read
things into a list, because reading things into a list would change the
list's length, and "a list does not" have a changeable length.
------------------------------
Date: Tue, 25 Mar 2008 16:37:15 +0000
From: Ben Morrow <ben@morrow.me.uk>
Subject: Re: Readline using foreach and while
Message-Id: <ramnb5-j0c1.ln1@osiris.mauzo.dyndns.org>
Quoth Ben Bullock <benkasminbullock@gmail.com>:
> On Tue, 25 Mar 2008 11:59:23 +0000, John W. Krahn wrote:
> > Ben Bullock wrote:
>
> >> The foreach version seems to first read the whole of the file into an
> >> array, and then go through it line by line:
> >
> > perldoc -q "What is the difference between a list and an array"
>
> Found in /usr/local/lib/perl5/5.10.0/pod/perlfaq4.pod
> What is the difference between a list and an array?
>
> An array has a changeable length. A list does not.
>
> If I had written "the foreach version reads the whole of the file into a
> list", I would have contradicted this FAQ entry, which says I can't read
> things into a list, because reading things into a list would change the
> list's length, and "a list does not" have a changeable length.
No, you're misunderstanding. A list is immutable *after it has been
created*. Obviously you can create lists with any contents, otherwise
you would be limited to using only lists compiled into perl. foreach
accepts a list as argument and iterates over it; <> in list context
(which is the real problem here) reads the entire file, splits it on
newline (or rather $/), and returns a (newly created) list with the
results. You can't modify the list after that: for an example where you
can, see Tie::File, which reads a file into an *array* instead.
Ben
------------------------------
Date: Tue, 25 Mar 2008 18:14:23 +0100
From: Frank Seitz <devnull4711@web.de>
Subject: Re: Readline using foreach and while
Message-Id: <64sq7gF2d6r5nU1@mid.individual.net>
Ben Morrow wrote:
> Quoth Ben Bullock <benkasminbullock@gmail.com>:
>>On Tue, 25 Mar 2008 11:59:23 +0000, John W. Krahn wrote:
>>>Ben Bullock wrote:
>>>>
>>>>The foreach version seems to first read the whole of the file into an
>>>>array, and then go through it line by line:
>>>
>>>perldoc -q "What is the difference between a list and an array"
Hm. Why is this distinction relevant here?
> A list is immutable *after it has been
> created*. Obviously you can create lists with any contents, otherwise
> you would be limited to using only lists compiled into perl. foreach
> accepts a list as argument and iterates over it;
use strict;
use warnings;
my @a = qw/a b c/;
for my $v (@a) {
push @a,'d' if $v eq 'c';
print "$v\n";
}
__END__
a
b
c
d
Frank
--
Dipl.-Inform. Frank Seitz; http://www.fseitz.de/
Anwendungen für Ihr Internet und Intranet
Tel: 04103/180301; Fax: -02; Industriestr. 31, 22880 Wedel
------------------------------
Date: Tue, 25 Mar 2008 10:50:33 -0700 (PDT)
From: nolo contendere <simon.chao@fmr.com>
Subject: Re: Readline using foreach and while
Message-Id: <56a271c3-dba1-4a13-857b-e2864fe31adf@m44g2000hsc.googlegroups.com>
On Mar 25, 1:14=A0pm, Frank Seitz <devnull4...@web.de> wrote:
> Ben Morrow wrote:
> > Quoth Ben Bullock <benkasminbull...@gmail.com>:
> >>On Tue, 25 Mar 2008 11:59:23 +0000, John W. Krahn wrote:
> >>>Ben Bullock wrote:
>
> >>>>The foreach version seems to first read the whole of the file into an
> >>>>array, and then go through it line by line:
>
> >>>perldoc -q "What is the difference between a list and an array"
>
> Hm. Why is this distinction relevant here?
>
> > A list is immutable *after it has been
> > created*. Obviously you can create lists with any contents, otherwise
> > you would be limited to using only lists compiled into perl. foreach
> > accepts a list as argument and iterates over it;
>
> use strict;
> use warnings;
>
> my @a =3D qw/a b c/;
> for my $v (@a) {
> =A0 =A0 push @a,'d' if $v eq 'c';
> =A0 =A0 print "$v\n";}
>
> __END__
> a
> b
> c
> d
>
http://groups.google.com/group/comp.lang.perl.misc/msg/9f2ebca559578bcf
------------------------------
Date: Tue, 25 Mar 2008 10:55:40 -0700 (PDT)
From: nolo contendere <simon.chao@fmr.com>
Subject: Re: Readline using foreach and while
Message-Id: <b8b4b3d3-4e01-4510-80af-843a9148818f@m36g2000hse.googlegroups.com>
On Mar 25, 1:50=A0pm, nolo contendere <simon.c...@fmr.com> wrote:
> On Mar 25, 1:14=A0pm, Frank Seitz <devnull4...@web.de> wrote:
>
>
>
> > Ben Morrow wrote:
> > > Quoth Ben Bullock <benkasminbull...@gmail.com>:
> > >>On Tue, 25 Mar 2008 11:59:23 +0000, John W. Krahn wrote:
> > >>>Ben Bullock wrote:
>
> > >>>>The foreach version seems to first read the whole of the file into a=
n
> > >>>>array, and then go through it line by line:
>
> > >>>perldoc -q "What is the difference between a list and an array"
>
> > Hm. Why is this distinction relevant here?
>
> > > A list is immutable *after it has been
> > > created*. Obviously you can create lists with any contents, otherwise
> > > you would be limited to using only lists compiled into perl. foreach
> > > accepts a list as argument and iterates over it;
>
> > use strict;
> > use warnings;
>
> > my @a =3D qw/a b c/;
> > for my $v (@a) {
> > =A0 =A0 push @a,'d' if $v eq 'c';
> > =A0 =A0 print "$v\n";}
>
> > __END__
> > a
> > b
> > c
> > d
>
> http://groups.google.com/group/comp.lang.perl.misc/msg/9f2ebca559578bcf
also, see 'Perl: modifying an array in a loop'
http://blog.plover.com/prog/perl/undefined.html#3
------------------------------
Date: Tue, 25 Mar 2008 17:49:34 +0000
From: Ben Morrow <ben@morrow.me.uk>
Subject: Re: Readline using foreach and while
Message-Id: <eiqnb5-s8j1.ln1@osiris.mauzo.dyndns.org>
Quoth Frank Seitz <devnull4711@web.de>:
> Ben Morrow wrote:
> > Quoth Ben Bullock <benkasminbullock@gmail.com>:
> >>On Tue, 25 Mar 2008 11:59:23 +0000, John W. Krahn wrote:
> >>>Ben Bullock wrote:
> >>>>
> >>>>The foreach version seems to first read the whole of the file into an
> >>>>array, and then go through it line by line:
> >>>
> >>>perldoc -q "What is the difference between a list and an array"
>
> Hm. Why is this distinction relevant here?
I'm not at all sure it is, except in the 'variable vs. value' sense.
> > A list is immutable *after it has been
> > created*. Obviously you can create lists with any contents, otherwise
> > you would be limited to using only lists compiled into perl. foreach
> > accepts a list as argument and iterates over it;
>
> use strict;
> use warnings;
>
> my @a = qw/a b c/;
> for my $v (@a) {
> push @a,'d' if $v eq 'c';
> print "$v\n";
> }
Good point. for is a little weird in this respect...
Ben
------------------------------
Date: Tue, 25 Mar 2008 20:56:33 GMT
From: "John W. Krahn" <someone@example.com>
Subject: Re: Readline using foreach and while
Message-Id: <5GdGj.47$_v3.44@edtnps90>
Frank Seitz wrote:
> Ben Morrow wrote:
>> Quoth Ben Bullock <benkasminbullock@gmail.com>:
>>> On Tue, 25 Mar 2008 11:59:23 +0000, John W. Krahn wrote:
>>>> Ben Bullock wrote:
>>>>> The foreach version seems to first read the whole of the file into an
>>>>> array, and then go through it line by line:
>>>> perldoc -q "What is the difference between a list and an array"
>
> Hm. Why is this distinction relevant here?
>
>> A list is immutable *after it has been
>> created*. Obviously you can create lists with any contents, otherwise
>> you would be limited to using only lists compiled into perl. foreach
>> accepts a list as argument and iterates over it;
>
> use strict;
> use warnings;
>
> my @a = qw/a b c/;
> for my $v (@a) {
We were talking about using a *list* in a foreach loop so that should be:
for my $v ( qw/a b c/ ) {
> push @a,'d' if $v eq 'c';
You are modifying an *array*, not a list.
> print "$v\n";
> }
> __END__
> a
> b
> c
> d
John
--
Perl isn't a toolbox, but a small machine shop where you
can special-order certain sorts of tools at low cost and
in short order. -- Larry Wall
------------------------------
Date: Tue, 25 Mar 2008 13:57:41 -0700
From: "szr" <szrRE@szromanMO.comVE>
Subject: Re: Readline using foreach and while
Message-Id: <fsbp060ot3@news2.newsguy.com>
John W. Krahn wrote:
> Ben Bullock wrote:
>> On Mar 25, 4:25 pm, Saurabh Jain <hundredr...@gmail.com> wrote:
>>> Hi,
>>> Is there any difference in reading a file using a while or a
>>> foreach in perl?
>>
>> The foreach version seems to first read the whole of the file into an
>> array, and then go through it line by line:
>
> perldoc -q "What is the difference between a list and an array"
Array is the variable type, List is the type of valve an Array
takes/holds.
> Perl isn't a toolbox, but a small machine shop where you
> can special-order certain sorts of tools at low cost and
> in short order. -- Larry Wall
Amen. I always loved this quote.
--
szr
------------------------------
Date: Tue, 25 Mar 2008 21:11:06 GMT
From: "John W. Krahn" <someone@example.com>
Subject: Re: Readline using foreach and while
Message-Id: <KTdGj.49$_v3.23@edtnps90>
Ben Bullock wrote:
> On Tue, 25 Mar 2008 11:59:23 +0000, John W. Krahn wrote:
>
>> Ben Bullock wrote:
>
>>> The foreach version seems to first read the whole of the file into an
>>> array, and then go through it line by line:
>> perldoc -q "What is the difference between a list and an array"
>
> Found in /usr/local/lib/perl5/5.10.0/pod/perlfaq4.pod
> What is the difference between a list and an array?
>
> An array has a changeable length. A list does not.
>
> If I had written "the foreach version reads the whole of the file into a
> list", I would have contradicted this FAQ entry, which says I can't read
> things into a list, because reading things into a list would change the
> list's length, and "a list does not" have a changeable length.
The FAQ entry does not contain the phrase "read things into a list".
print reverse grep /a/, readdir DIR;
In the above example, where does grep() get its list from? Is the list
that grep() returns the same length as the list it gets? Where does
reverse() get its list from, and is it the same list that grep() gets?
Where does print() get its list from, and is it the same list that
reverse() gets?
John
--
Perl isn't a toolbox, but a small machine shop where you
can special-order certain sorts of tools at low cost and
in short order. -- Larry Wall
------------------------------
Date: Tue, 25 Mar 2008 14:52:51 -0700 (PDT)
From: "comp.llang.perl.moderated" <ced@blv-sam-01.ca.boeing.com>
Subject: Re: Readline using foreach and while
Message-Id: <e122bc16-bd96-487f-a037-04faa41923b2@i12g2000prf.googlegroups.com>
On Mar 25, 1:57 pm, "szr" <sz...@szromanMO.comVE> wrote:
> John W. Krahn wrote:
> > Ben Bullock wrote:
> >> On Mar 25, 4:25 pm, Saurabh Jain <hundredr...@gmail.com> wrote:
> >>> Hi,
> >>> Is there any difference in reading a file using a while or a
> >>> foreach in perl?
>
> >> The foreach version seems to first read the whole of the file into an
> >> array, and then go through it line by line:
>
> > perldoc -q "What is the difference between a list and an array"
>
> Array is the variable type, List is the type of valve an Array
> takes/holds.
>
Actually, an array just holds scalars... or references which is a
special type of scalar.
An array can be populated from a list though.
--
Charles DeRykus
------------------------------
Date: Tue, 25 Mar 2008 15:07:22 -0700 (PDT)
From: sheinrich@my-deja.com
Subject: Re: strategys other than subroutine and OO?
Message-Id: <0f56f864-ee68-4c27-80b8-2223c9826816@a23g2000hsc.googlegroups.com>
On Mar 20, 12:34 pm, "Ela" <e...@yantai.org> wrote:
> I have 300 lines of codes that are used roughly twice. Some of the lines
> have delicate difference in execution (e.g. ~6 variables have to be replaced
> and conditional statements are also different) and therefore many arguments
> need to be passed into it in order to differentiate the execution parts. OO
> development is analogously also difficult. I wonder if there is any good way
> to reuse the codes, e.g. some sort of "goto" rather than duplicating these
> lines. Whenever there is any change, I find that is really error-prone.
I'm not sure why none of the luminaries were coming forward with
closures, but I'd highly recommend reading up on them.
Of course they _are_ subroutines but IMHO they're offering the most
versatile re-use of programm code.
GOTO
perldoc -q closure
Cheers,
Steffen
------------------------------
Date: Tue, 25 Mar 2008 10:09:53 -0700
From: Jim Gibson <jimsgibson@gmail.com>
Subject: Re: The huge amount response data problem
Message-Id: <250320081009536174%jimsgibson@gmail.com>
In article <47e8caa2$0$32055$da0feed9@news.zen.co.uk>, RedGrittyBrick
<RedGrittyBrick@SpamWeary.foo> wrote:
> falconzyx@gmail.com wrote:
[problem getting data from 100_000 URLs snipped]
> That's because, if your file contains 100000 lines, your program tries
> to create 100000 simultaneous threads doesn't it?
>
> I would create a pool with a fixed number of threads (say 10). I'd read
> the file adding tasks to a queue of the same size, after filling the
> queue I'd pause reading the file until the queue has a spare space.
> Maybe this could be achieved by sleeping a while (say 100ms) and
> re-checking if the queue is stuill full. When a thread is created or has
> finished a task it should remove a task from the queue and process it.
> If the queue is empty the thread should sleep for a while (say 200ms)
> and try again, you'd need some mechanism to signal threads that all
> tasks have been queued (maybe a flag, a special marker task, a signal or
> a certain number of consecutive failed attempts to find work.)
With a fixed number of predefined URLs, I would use a simpler approach:
Fork off some number of processes (say 10) and assign to each of them
the job of reading the URL file and fetching the results from their set
of URLs sequentially.
The OP has not told us how the results from each URL will be used, so
that may affect whether to use threads or forked processes. A database
would be appropriate for merging or saving the results from all of
these URLs.
There is also the LWP::Parallel module, which allows one process to
simultaneously fetch responses from many URLs (I have not used it).
--
Jim Gibson
Posted Via Usenet.com Premium Usenet Newsgroup Services
----------------------------------------------------------
** SPEED ** RETENTION ** COMPLETION ** ANONYMITY **
----------------------------------------------------------
http://www.usenet.com
------------------------------
Date: 25 Mar 2008 17:42:49 GMT
From: xhoster@gmail.com
Subject: Re: The huge amount response data problem
Message-Id: <20080325134251.121$dx@newsreader.com>
"falconzyx@gmail.com" <falconzyx@gmail.com> wrote:
> I have a issue:
> 1. I want to open a file and use the data from the file to construct
> the url.
> 2. After I constructed the url and sent it, I got the response html
> data and some parts are what I want store inot the files.
>
> It seems like a very easy thing, however, the issue is that the data
> from the file that I have to open are too huge, which I have to
> consturct almost 200000 url address to send and parse response data.
> And the speed is very very slow.
What part is slow, waiting for the response or parsing it?
Does those URLs point to *your* servers? If so, then you should be able
to bypass http and go directly to the source. If not, then do you have
permission from the owner of the servers to launch what could very well
be a denial of service attack against them?
Xho
--
-------------------- http://NewsReader.Com/ --------------------
The costs of publication of this article were defrayed in part by the
payment of page charges. This article must therefore be hereby marked
advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate
this fact.
------------------------------
Date: 25 Mar 2008 17:50:04 GMT
From: xhoster@gmail.com
Subject: Re: The huge amount response data problem
Message-Id: <20080325135006.452$el@newsreader.com>
RedGrittyBrick <RedGrittyBrick@SpamWeary.foo> wrote:
>
> I find it hard to understand what you are saying but I think the answer
> is: Yes, Perl is well suited to programming with multiple threads (or
> processes).
I agree with the "(or processes)" part, provided you are running on a Unix
like platform. But in my experience/opinion Perl threads mostly suck.
--
-------------------- http://NewsReader.Com/ --------------------
The costs of publication of this article were defrayed in part by the
payment of page charges. This article must therefore be hereby marked
advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate
this fact.
------------------------------
Date: 6 Apr 2001 21:33:47 GMT (Last modified)
From: Perl-Users-Request@ruby.oce.orst.edu (Perl-Users-Digest Admin)
Subject: Digest Administrivia (Last modified: 6 Apr 01)
Message-Id: <null>
Administrivia:
#The Perl-Users Digest is a retransmission of the USENET newsgroup
#comp.lang.perl.misc. For subscription or unsubscription requests, send
#the single line:
#
# subscribe perl-users
#or:
# unsubscribe perl-users
#
#to almanac@ruby.oce.orst.edu.
NOTE: due to the current flood of worm email banging on ruby, the smtp
server on ruby has been shut off until further notice.
To submit articles to comp.lang.perl.announce, send your article to
clpa@perl.com.
#To request back copies (available for a week or so), send your request
#to almanac@ruby.oce.orst.edu with the command "send perl-users x.y",
#where x is the volume number and y is the issue number.
#For other requests pertaining to the digest, send mail to
#perl-users-request@ruby.oce.orst.edu. Do not waste your time or mine
#sending perl questions to the -request address, I don't have time to
#answer them even if I did know the answer.
------------------------------
End of Perl-Users Digest V11 Issue 1392
***************************************
