[28375] in Perl-Users-Digest
Perl-Users Digest, Issue: 9739 Volume: 10
daemon@ATHENA.MIT.EDU (Perl-Users Digest)
Tue Sep 19 03:10:14 2006
Date: Tue, 19 Sep 2006 00:10:08 -0700 (PDT)
From: Perl-Users Digest <Perl-Users-Request@ruby.OCE.ORST.EDU>
To: Perl-Users@ruby.OCE.ORST.EDU (Perl-Users Digest)
Perl-Users Digest Tue, 19 Sep 2006 Volume: 10 Number: 9739
Today's topics:
Question on download by LWP <sapience@gmail.com>
Re: Question on download by LWP <john@castleamber.com>
Re: Question on download by LWP <David.Squire@no.spam.from.here.au>
Re: Question on download by LWP <john@castleamber.com>
Re: Question on download by LWP <David.Squire@no.spam.from.here.au>
Re: Question on download by LWP <tadmc@augustmail.com>
Re: Question on download by LWP <john@castleamber.com>
Re: Sum the middle column for a given unique volser. <veatchla@yahoo.com>
Re: Use of uninitialized value in print <mumia.w.18.spam+nospam.usenet@earthlink.net>
Digest Administrivia (Last modified: 6 Apr 01) (Perl-Users-Digest Admin)
----------------------------------------------------------------------
Date: 18 Sep 2006 17:31:04 -0700
From: "Wonder" <sapience@gmail.com>
Subject: Question on download by LWP
Message-Id: <1158625864.546650.269460@b28g2000cwb.googlegroups.com>
Hello,
I'm trying to use a perl script to download some html files on a
website, but it's very strange that the response for the fetch-file is
always "404 not found", even though I have no problem to open the same
url in IE. And the same script can work for the urls of other websites.
It seems that website only allow the connection from web browser, but
not download tools ( it also denied the download request of tools like
Flashget). Is there any field I need to set up in my perl code to cheat
the remote website to make it think this is a request from the web
browser? Thanks.
The following is my code:
use LWP::UserAgent;
use HTTP::Response;
$ua = LWP::UserAgent->new;
$CurrentURL = "http://DomainName.com/RemoteFileName.html";
$filename = ".LocalFileName.html";
$response = $ua->mirror($CurrentURL, $filename);
die "Can't get $CurrentURL -- ", $response->status_line unless
$response->is_success;
------------------------------
Date: 19 Sep 2006 02:25:10 GMT
From: John Bokma <john@castleamber.com>
Subject: Re: Question on download by LWP
Message-Id: <Xns9842D9E463BD8castleamber@130.133.1.4>
"Wonder" <sapience@gmail.com> wrote:
> Hello,
>
> I'm trying to use a perl script to download some html files on a
> website, but it's very strange that the response for the fetch-file is
> always "404 not found", even though I have no problem to open the same
> url in IE. And the same script can work for the urls of other
> websites.
>
> It seems that website only allow the connection from web browser, but
> not download tools ( it also denied the download request of tools like
> Flashget). Is there any field I need to set up in my perl code to
> cheat the remote website to make it think this is a request from the
> web browser? Thanks.
>
> The following is my code:
>
> use LWP::UserAgent;
> use HTTP::Response;
use strict;
use warnings;
> $ua = LWP::UserAgent->new;
You might want to try:
my $ua = LWP::UserAgent->new(
agent => 'Mozilla/5.0'
);
If that gives the same problem, try to use a longer agent, e.g.
my $ua = LWP::UserAgent->new(
agent => 'Mozilla/5.0'
. ' (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.6)'
. ' Gecko/20060728 Firefox/1.5.0.6'
);
(yes, I am going to update after this post :-) ).
Quite some sites block well known "grabbers".
> $CurrentURL = "http://DomainName.com/RemoteFileName.html";
don't use camelcase with Perl (yes, that sounds like a contradiction).
Also, RemoteFileName.html is not really a filename Also, use example.com
for examples, don't invent domain names.
> $filename = ".LocalFileName.html";
> $response = $ua->mirror($CurrentURL, $filename);
> die "Can't get $CurrentURL -- ", $response->status_line unless
> $response->is_success;
I would change that into:
$response->is_success or die .....
Which you can read as: the response has to be sucessful or we give up.
--
John Experienced Perl programmer: http://castleamber.com/
Perl help, tutorials, and examples: http://johnbokma.com/perl/
------------------------------
Date: Tue, 19 Sep 2006 03:36:10 +0100
From: David Squire <David.Squire@no.spam.from.here.au>
Subject: Re: Question on download by LWP
Message-Id: <eenl25$hgg$1@gemini.csx.cam.ac.uk>
John Bokma wrote:
> don't use camelcase with Perl
Now, familiar as I am with the doctrinaire nature of this group, this
seems to me to go too far. Surely folk can have their own naming
conventions and you (we) can be quiet about that.
DS
------------------------------
Date: 19 Sep 2006 02:47:06 GMT
From: John Bokma <john@castleamber.com>
Subject: Re: Question on download by LWP
Message-Id: <Xns9842DD9BB69D4castleamber@130.133.1.4>
David Squire <David.Squire@no.spam.from.here.au> wrote:
> John Bokma wrote:
>
>> don't use camelcase with Perl
>
> Now, familiar as I am with the doctrinaire nature of this group, this
> seems to me to go too far. Surely folk can have their own naming
> conventions and you (we) can be quiet about that.
Yup, as they can decide not to use strict & warnings. Yet this group
insists on some form of readability, and *I* include not using camel case
with that. If I have to include a disclaimer with each and every sentence
my replies take too much time. It's free advice, after all, and I assume
that the reader is a grown up who can ignore advice at will.
--
John Experienced Perl programmer: http://castleamber.com/
Perl help, tutorials, and examples: http://johnbokma.com/perl/
------------------------------
Date: Tue, 19 Sep 2006 03:58:09 +0100
From: David Squire <David.Squire@no.spam.from.here.au>
Subject: Re: Question on download by LWP
Message-Id: <eenmbb$k5c$1@gemini.csx.cam.ac.uk>
John Bokma wrote:
> David Squire <David.Squire@no.spam.from.here.au> wrote:
>
>> John Bokma wrote:
>>
>>> don't use camelcase with Perl
>> Now, familiar as I am with the doctrinaire nature of this group, this
>> seems to me to go too far. Surely folk can have their own naming
>> conventions and you (we) can be quiet about that.
>
> Yup, as they can decide not to use strict & warnings. Yet this group
> insists on some form of readability, and *I* include not using camel case
> with that.
I can't agree with this. Using strict and warnings is language-specific
advice that helps folk solve and avoid problems. The readability or not
of "camelcase" is person-specific and has nothing to do with Perl.
So, I have no problems with your saying "I don't like camelcase", but
"don't use camelcase with Perl" to me over-steps the mark.
I, and no doubt others, participate in multiple Perl projects where we
don't always have control over naming conventions (how do you feel about
Hungarian?), and editing that down should surely not be a requirement
for posting here. Getting a minimal script is a big enough ask for most
folk.
DS
------------------------------
Date: Mon, 18 Sep 2006 23:00:05 -0500
From: Tad McClellan <tadmc@augustmail.com>
Subject: Re: Question on download by LWP
Message-Id: <slrneguqq5.5ok.tadmc@magna.augustmail.com>
David Squire <David.Squire@no.spam.from.here.au> wrote:
> John Bokma wrote:
>> David Squire <David.Squire@no.spam.from.here.au> wrote:
>>
>>> John Bokma wrote:
>>>
>>>> don't use camelcase with Perl
>>> Now, familiar as I am with the doctrinaire nature of this group, this
>>> seems to me to go too far. Surely folk can have their own naming
>>> conventions and you (we) can be quiet about that.
Except when their convention is to use PERL, perhaps? :-)
> So, I have no problems with your saying "I don't like camelcase", but
> "don't use camelcase with Perl" to me over-steps the mark.
He should have probably said:
people will think less of you if you use camelcase with Perl
That is not to say that it is right to think that, but I am quite sure
that my re-phrase is true nonetheless.
Similar to:
people will think less of you if you spell it PERL
They both just tell folks that you don't know the secret handshake.
--
Tad McClellan SGML consulting
tadmc@augustmail.com Perl programming
Fort Worth, Texas
------------------------------
Date: 19 Sep 2006 06:46:45 GMT
From: John Bokma <john@castleamber.com>
Subject: Re: Question on download by LWP
Message-Id: <Xns98431219B57B4castleamber@130.133.1.4>
David Squire <David.Squire@no.spam.from.here.au> wrote:
> John Bokma wrote:
[..]
> I can't agree with this. Using strict and warnings is
> language-specific advice that helps folk solve and avoid problems. The
> readability or not of "camelcase" is person-specific and has nothing
> to do with Perl.
"... use underscores to separate words." Programming Perl 3rd ed, p605
(Programming with Style).
"... use underscores to separate words in longer identifiers."
perldoc perlstyle
IMO this has everything to do with Perl as for the same reason using camel
case in Java has everything to do with Java.
> So, I have no problems with your saying "I don't like camelcase", but
> "don't use camelcase with Perl" to me over-steps the mark.
I just passed on the advice given in Programming Perl ;-)
> I, and no doubt others, participate in multiple Perl projects where we
> don't always have control over naming conventions (how do you feel
> about Hungarian?), and editing that down should surely not be a
> requirement for posting here.
For the same reason not using use strict / warnings shouldn't be an issue
if that isn't causing any problem. Yet "we" insist on it being present.
Personally I think the posted problem should conform to at least to
perldoc perlstyle.
I agree with you to stick with the naming conventions of a given project.
And for that very reason, I think it's good to follow in Perl related
newsgroups the naming convention as outlined in perldoc perlstyle /
Programming Perl.
> Getting a minimal script is a big enough
> ask for most folk.
They need help, and ask "us" do answer their question(s). I think its fair
to ask of them to present their question in a format that's easy to read
for most people. To me that's following perlstyle as close as possible.
Finally, I am not saying to have no point at all: I agree that too much
nit picking can be overwhelming for the OP :-) But I also think I have a
point ;-)
--
John Experienced Perl programmer: http://castleamber.com/
Perl help, tutorials, and examples: http://johnbokma.com/perl/
------------------------------
Date: Mon, 18 Sep 2006 19:57:21 -0500
From: l v <veatchla@yahoo.com>
Subject: Re: Sum the middle column for a given unique volser.
Message-Id: <1158626694_31251@sp6iad.superfeed.net>
Noatec wrote:
[snip]
> while (my $line = <$MLIST>) {
> my ($volser, $reads, $date) = split(/,/,$line);
> if ($tmp_vol eq ''){
> $tmp_vol=$volser;
> }
> $hash{$volser}{reads}+=$reads;
> $hash{$volser}{date}=$date;
> if ($tmp_vol != $volser) {
> if ($hash{$tmp_vol}{reads} <= $thresh) {
> print "$tmp_vol $hash{$tmp_vol}{reads} $date";
> }
> $tmp_vol=$volser;
> }
> }
It appears you are using $temp_vol to determine when you changed
volser's in order to print. Is this correct? If so, then I'd either 1)
remove the hash logic and use scalars as the hash is overkill and wastes
memory or 2) remove the $tmp_vol code, build the hash like you are, then
after you have read the entire input file; iterate over the hash
printing depending on your age and read count thresholds.
The second option (the hash) also allows sorting of your output if you
later determine you want to do so.
[snip]
> I added the threshold since I only care about tapes with less than x
> reads.
> The next step is to add some date logic to calculate whether three
> years has past since the last write date.
> I'm very interested in your critique of this.
> Is there a way to do it better, faster or shorter?
>
--
Len
Posted Via Usenet.com Premium Usenet Newsgroup Services
----------------------------------------------------------
** SPEED ** RETENTION ** COMPLETION ** ANONYMITY **
----------------------------------------------------------
http://www.usenet.com
------------------------------
Date: Mon, 18 Sep 2006 23:22:21 GMT
From: "Mumia W." <mumia.w.18.spam+nospam.usenet@earthlink.net>
Subject: Re: Use of uninitialized value in print
Message-Id: <NSFPg.13396$bM.5767@newsread4.news.pas.earthlink.net>
On 09/18/2006 01:42 PM, Pam wrote:
> Hello
> I ran into a problem that I have been working on, the warning use of
> unitialized value at print
> [...]
> # Row and column are zero indexed
> my $row = 0;
>
>
> while (<CSVFILE>) {
> if ($csv->parse($_)) {
> my @Fld = $csv->fields;
>
>
> my $col = 0;
> foreach my $token (@Fld) {
> $worksheet->write($row, $col, $token);
> $col++;
> }
> $row++;
> if ($row > 1){
> $count = $count + 1;
>
>
> $total = $count;
>
>
> }
> [...]
If $row is never greater than one, $total will never be assigned, so I
suspect that $row never got to be greater than one. Either that or
$csv->parse($_) failed (returned a false value).
Down below, $total is undefined when you print it because it was never
assigned up above.
------------------------------
Date: 6 Apr 2001 21:33:47 GMT (Last modified)
From: Perl-Users-Request@ruby.oce.orst.edu (Perl-Users-Digest Admin)
Subject: Digest Administrivia (Last modified: 6 Apr 01)
Message-Id: <null>
Administrivia:
#The Perl-Users Digest is a retransmission of the USENET newsgroup
#comp.lang.perl.misc. For subscription or unsubscription requests, send
#the single line:
#
# subscribe perl-users
#or:
# unsubscribe perl-users
#
#to almanac@ruby.oce.orst.edu.
NOTE: due to the current flood of worm email banging on ruby, the smtp
server on ruby has been shut off until further notice.
To submit articles to comp.lang.perl.announce, send your article to
clpa@perl.com.
#To request back copies (available for a week or so), send your request
#to almanac@ruby.oce.orst.edu with the command "send perl-users x.y",
#where x is the volume number and y is the issue number.
#For other requests pertaining to the digest, send mail to
#perl-users-request@ruby.oce.orst.edu. Do not waste your time or mine
#sending perl questions to the -request address, I don't have time to
#answer them even if I did know the answer.
------------------------------
End of Perl-Users Digest V10 Issue 9739
***************************************
