[28860] in Perl-Users-Digest
Perl-Users Digest, Issue: 104 Volume: 11
daemon@ATHENA.MIT.EDU (Perl-Users Digest)
Fri Feb 9 22:06:05 2007
Date: Fri, 9 Feb 2007 19:05:28 -0800 (PST)
From: Perl-Users Digest <Perl-Users-Request@ruby.OCE.ORST.EDU>
To: Perl-Users@ruby.OCE.ORST.EDU (Perl-Users Digest)
Perl-Users Digest Fri, 9 Feb 2007 Volume: 11 Number: 104
Today's topics:
download a text file from a URL <vivekmahanta@gmail.com>
Re: download a text file from a URL <john@castleamber.com>
Re: download a text file from a URL <tfeserver@gmail.com>
Re: download a text file from a URL <greg.ferguson@icrossing.com>
Re: download a text file from a URL <vivekmahanta@gmail.com>
Re: download a text file from a URL <john@castleamber.com>
Re: download a text file from a URL <greg.ferguson@icrossing.com>
Re: download a text file from a URL <vivekmahanta@gmail.com>
Re: download a text file from a URL <glex_no-spam@qwest-spam-no.invalid>
Re: download a text file from a URL <jgibson@mail.arc.nasa.gov>
dual $SIG{ALRM} in a same program <scottalorda@libello.com>
Extending Parse::Yapp with new functionalities <casiano.rodriguez.leon@gmail.com>
find lines in a file jesse_hardy@premierinc.com
Re: find lines in a file <mritty@gmail.com>
Re: French Accents appear incorrectly... nigel@bouteyres.com
Re: FTP Link extraction (Was: Re: Request for a PERL sc <spamtrap@dot-app.org>
get data from xml <dontmewithme@got.it>
Re: get data from xml <jurgenex@hotmail.com>
Re: get data from xml <rvtol+news@isolution.nl>
Re: get data from xml <bik.mido@tiscalinet.it>
Digest Administrivia (Last modified: 6 Apr 01) (Perl-Users-Digest Admin)
----------------------------------------------------------------------
Date: 5 Feb 2007 20:24:21 -0800
From: "vivekmahanta@gmail.com" <vivekmahanta@gmail.com>
Subject: download a text file from a URL
Message-Id: <1170735861.699742.112240@v45g2000cwv.googlegroups.com>
Hey guys,
I want to download a text file from a URL and store its contents in my
webhosting server. Eg, if the URL is something like http://www.yahoo.com/file.txt
then I want to download file.txt and then store its contents. How can
I do that in Perl ?? Any help will be really appreciated. Please reply
soon.
Thank you
Vivek
------------------------------
Date: 6 Feb 2007 05:20:29 GMT
From: John Bokma <john@castleamber.com>
Subject: Re: download a text file from a URL
Message-Id: <Xns98CEED7122D4Bcastleamber@130.133.1.4>
"vivekmahanta@gmail.com" <vivekmahanta@gmail.com> wrote:
> Hey guys,
>
> I want to download a text file from a URL and store its contents in my
> webhosting server. Eg, if the URL is something like
> http://www.yahoo.com/file.txt then I want to download file.txt and
> then store its contents. How can I do that in Perl ?? Any help will be
> really appreciated.
check out LWP::UserAgent and friends
--
John Experienced Perl programmer: http://castleamber.com/
Perl help, tutorials, and examples: http://johnbokma.com/perl/
------------------------------
Date: 5 Feb 2007 22:36:54 -0800
From: "tfe" <tfeserver@gmail.com>
Subject: Re: download a text file from a URL
Message-Id: <1170743814.056581.305390@p10g2000cwp.googlegroups.com>
You can use LWP::Simple with method getstore:
getstore($url, $file);
--
tfe
http://tfeserver.be
On Feb 6, 6:20 am, John Bokma <j...@castleamber.com> wrote:
> "vivekmaha...@gmail.com" <vivekmaha...@gmail.com> wrote:
> > Hey guys,
>
> > I want to download a text file from a URL and store its contents in my
> > webhosting server. Eg, if the URL is something like
> >http://www.yahoo.com/file.txtthen I want to download file.txt and
> > then store its contents. How can I do that in Perl ?? Any help will be
> > really appreciated.
>
> check out LWP::UserAgent and friends
>
> --
> John Experienced Perl programmer:http://castleamber.com/
>
> Perl help, tutorials, and examples:http://johnbokma.com/perl/
------------------------------
Date: 6 Feb 2007 11:10:52 -0800
From: "gf" <greg.ferguson@icrossing.com>
Subject: Re: download a text file from a URL
Message-Id: <1170789051.978857.285320@v45g2000cwv.googlegroups.com>
Or...
use LWP::UserAgent;
my $ua = LWP::UserAgent->new;
my $resp = $ua->get('http://url', './file_to_save_to');
There are so many ways to get there that it must be Perl.
------------------------------
Date: 6 Feb 2007 12:44:00 -0800
From: "vivekmahanta@gmail.com" <vivekmahanta@gmail.com>
Subject: Re: download a text file from a URL
Message-Id: <1170794640.090235.72970@j27g2000cwj.googlegroups.com>
On Feb 6, 2:10 pm, "gf" <greg.fergu...@icrossing.com> wrote:
> Or...
>
> use LWP::UserAgent;
>
> my $ua = LWP::UserAgent->new;
>
> my $resp = $ua->get('http://url', './file_to_save_to');
>
> There are so many ways to get there that it must be Perl.
thank you so much for all the tips. Now, I want to create Cron Jobs
that will keep downloading such files from the URLs every 5 mins or
so. How do I create the scripts for doing that which can be later fed
to a Cron Job setup menu ?? I think after I have written the scripts
in Perl I can definitely convert them into executables which can be
set up as Cron Jobs. Can you tell me how that can be done ?? Any help
is really appreciated.
thank you,
Vivek
------------------------------
Date: 6 Feb 2007 22:08:14 GMT
From: John Bokma <john@castleamber.com>
Subject: Re: download a text file from a URL
Message-Id: <Xns98CFA427FD356castleamber@130.133.1.4>
"vivekmahanta@gmail.com" <vivekmahanta@gmail.com> wrote:
> thank you so much for all the tips. Now, I want to create Cron Jobs
> that will keep downloading such files from the URLs every 5 mins or
> so. How do I create the scripts for doing that which can be later fed
> to a Cron Job setup menu ?? I think after I have written the scripts
> in Perl I can definitely convert them into executables which can be
> set up as Cron Jobs. Can you tell me how that can be done ?? Any help
> is really appreciated.
You might want to check out wget or curl.
--
John Experienced Perl programmer: http://castleamber.com/
Perl help, tutorials, and examples: http://johnbokma.com/perl/
------------------------------
Date: 7 Feb 2007 08:31:44 -0800
From: "gf" <greg.ferguson@icrossing.com>
Subject: Re: download a text file from a URL
Message-Id: <1170865904.498362.44100@p10g2000cwp.googlegroups.com>
On Feb 6, 1:44 pm, "vivekmaha...@gmail.com" <vivekmaha...@gmail.com>
wrote:
[...]
> How do I create the scripts for doing that which can be later fed
> to a Cron Job setup menu ?? I think after I have written the scripts
> in Perl I can definitely convert them into executables which can be
> set up as Cron Jobs. Can you tell me how that can be done ?? Any help
> is really appreciated.
Convert to executables? Not with Perl. It's a scripting language and
has no compiler output like C/C++ or assembler. You can set a script
to be executable, but it remains a script. (Yes there are converted
core-dumps but that's just a veneer.)
Once you have a Perl script for downloading one file you can easily
have cron launch it at intervals. Read the documentation for cron and
you'll be done since cron does pretty much anything that you do from
the command line.
Whether to download every time is an additional question you need to
consider. From Perl you can check and make a decision to skip the
get() if the file hasn't changed - see the head() command in LWP - but
you might want to consider using wget or one of the other apps that
have been invented to solve this sort of problem. If you're mirroring
a file you could easily add the --mirror parameter and be done with
it.
Sometimes it's a lot easier to use wget or curl rather than code from
scratch, but we get wrapped up in doing it ourselves.
------------------------------
Date: 7 Feb 2007 17:42:45 -0800
From: "vivekmahanta@gmail.com" <vivekmahanta@gmail.com>
Subject: Re: download a text file from a URL
Message-Id: <1170898965.503121.199640@l53g2000cwa.googlegroups.com>
On Feb 7, 11:31 am, "gf" <greg.fergu...@icrossing.com> wrote:
> On Feb 6, 1:44 pm, "vivekmaha...@gmail.com" <vivekmaha...@gmail.com>
> wrote:
> [...]
>
> > How do I create the scripts for doing that which can be later fed
> > to a Cron Job setup menu ?? I think after I have written the scripts
> > in Perl I can definitely convert them into executables which can be
> > set up as Cron Jobs. Can you tell me how that can be done ?? Any help
> > is really appreciated.
>
> Convert to executables? Not with Perl. It's a scripting language and
> has no compiler output like C/C++ or assembler. You can set a script
> to be executable, but it remains a script. (Yes there are converted
> core-dumps but that's just a veneer.)
>
> Once you have a Perl script for downloading one file you can easily
> have cron launch it at intervals. Read the documentation for cron and
> you'll be done since cron does pretty much anything that you do from
> the command line.
>
> Whether to download every time is an additional question you need to
> consider. From Perl you can check and make a decision to skip the
> get() if the file hasn't changed - see the head() command in LWP - but
> you might want to consider using wget or one of the other apps that
> have been invented to solve this sort of problem. If you're mirroring
> a file you could easily add the --mirror parameter and be done with
> it.
>
> Sometimes it's a lot easier to use wget or curl rather than code from
> scratch, but we get wrapped up in doing it ourselves.
Actually, I have to download a lot of files every 5 mins or so. I have
written such programs in java and C++ already, but I am not sure how I
can use Cron Jobs to execute them every 5 mins. I can write a similar
program in Perl or any other language too, but I want it to be fast
and reliable. Can you suggest what language I should use ??
Vivek
------------------------------
Date: Thu, 08 Feb 2007 10:43:05 -0600
From: "J. Gleixner" <glex_no-spam@qwest-spam-no.invalid>
Subject: Re: download a text file from a URL
Message-Id: <45cb52d6$0$507$815e3792@news.qwest.net>
vivekmahanta@gmail.com wrote:
> Actually, I have to download a lot of files every 5 mins or so. I have
> written such programs in java and C++ already, but I am not sure how I
> can use Cron Jobs to execute them every 5 mins. I can write a similar
> program in Perl or any other language too, but I want it to be fast
> and reliable. Can you suggest what language I should use ??
The language doesn't matter! Use the language or tools that you
know and move on.
Talk to your administrator about using cron, or look at
the documentation: man crontab Discussing how to use cron
isn't the purpose of this newsgroup, try one dealing with
your particular OS.
Also, if it's your machine and you have to "download a lot
of files", use something like rsync (no.. I won't discuss
how to use it here.. search the Internet..).
------------------------------
Date: Thu, 08 Feb 2007 17:34:28 -0800
From: Jim Gibson <jgibson@mail.arc.nasa.gov>
Subject: Re: download a text file from a URL
Message-Id: <080220071734287654%jgibson@mail.arc.nasa.gov>
In article <1170898965.503121.199640@l53g2000cwa.googlegroups.com>,
<"vivekmahanta@gmail.com"> wrote:
> On Feb 7, 11:31 am, "gf" <greg.fergu...@icrossing.com> wrote:
> > On Feb 6, 1:44 pm, "vivekmaha...@gmail.com" <vivekmaha...@gmail.com>
> > wrote:
> > [...]
> >
> > > How do I create the scripts for doing that which can be later fed
> > > to a Cron Job setup menu ?? I think after I have written the scripts
> > > in Perl I can definitely convert them into executables which can be
> > > set up as Cron Jobs. Can you tell me how that can be done ?? Any help
> > > is really appreciated.
> >
> Actually, I have to download a lot of files every 5 mins or so. I have
> written such programs in java and C++ already, but I am not sure how I
> can use Cron Jobs to execute them every 5 mins.
0,5,10,15,20,25,30,35,40,45,50,55 * * * * download.pl
or possibly
*/5 * * * * download.pl
See 'man 5 crontab' on Unix.
> ... I can write a similar
> program in Perl or any other language too, but I want it to be fast
> and reliable. Can you suggest what language I should use ??
Whatever language you know best, unless you want to process text files
after downloading them. Then Perl is an obvious choice.
Posted Via Usenet.com Premium Usenet Newsgroup Services
----------------------------------------------------------
** SPEED ** RETENTION ** COMPLETION ** ANONYMITY **
----------------------------------------------------------
http://www.usenet.com
------------------------------
Date: 7 Feb 2007 06:17:29 -0800
From: "=?iso-8859-1?q?S=E9bastien_Cottalorda?=" <scottalorda@libello.com>
Subject: dual $SIG{ALRM} in a same program
Message-Id: <1170857849.625979.18650@p10g2000cwp.googlegroups.com>
Hi all,
I use eval {} , $SIG{ALRM} and alarm to timeout some part of a
program.
This program uses the IPC::Shareable program too.
Included in a real-time project, this program need to share some share
memory segments with another process.
My problem is the following:
As this program is a real-time one, it cannot wait a too long time to
obtain the lock on a segment (with _tie or shlock).
That's why I nned to timeout too this lock.
Will I get problem using in a program eval, $SIG{ALRM} *and* in a
module the same mechanism ?
Thaks in advance for your help.
Sebastien
------------------------------
Date: Tue, 6 Feb 2007 12:29:58 GMT
From: "Casiano" <casiano.rodriguez.leon@gmail.com>
Subject: Extending Parse::Yapp with new functionalities
Message-Id: <JD1M7y.21yH@zorch.sf-bay.org>
Version 1.06600 of Parse::Eyapp have been released.
Parse::Eyapp (Extended yapp) is a collection of modules
that extends Francois Desarmenien Parse::Yapp 1.05.
Eyapp extends yacc/yapp syntax with
functionalities like
* Named attributes,
* EBNF-like expressions (via *, + and ? operators),
* Modifiable default action,
* Automatic syntax tree building,
* Semi-automatic abstract syntax tree building,
* Translation schemes (like in the Dragon's book on Compilers),
* Tree transformations via a language that mimics ordinary regular
expressions
but works on trees instead of strings.
* Support for Nested Scope Analysis,
* Facilities to build directed acyclic graphs instead of Trees.
If you like yacc/yapp and Perl you'll find Parse::Eyapp is for you.
Casiano Rodriguez-Leon <casiano@cpan.org>
------------------------------
Date: 6 Feb 2007 11:56:56 -0800
From: jesse_hardy@premierinc.com
Subject: find lines in a file
Message-Id: <1170791816.077514.90130@v33g2000cwv.googlegroups.com>
I have log files that I want to parse through every day and search for
failures and errors. I understand how to open the file and search for
failures and errors but I may have several lines that have the same
error. What I want to do is print the first occurence to a file and
count each occurence after the first and print that to the same file.
So if I had 50 lines with the same error I wanted to print the first
error to a file and say there were 49 more occurences of that error.
Thanks
Jesse
------------------------------
Date: 6 Feb 2007 12:22:47 -0800
From: "Paul Lalli" <mritty@gmail.com>
Subject: Re: find lines in a file
Message-Id: <1170793367.252777.158210@v45g2000cwv.googlegroups.com>
On Feb 6, 2:56 pm, jesse_ha...@premierinc.com wrote:
> I have log files that I want to parse through every day and search for
> failures and errors. I understand how to open the file and search for
> failures and errors but I may have several lines that have the same
> error. What I want to do is print the first occurence to a file and
> count each occurence after the first and print that to the same file.
> So if I had 50 lines with the same error I wanted to print the first
> error to a file and say there were 49 more occurences of that error.
So what have you tried so far, and where did it go wrong?
I would use a hash to keep track of the errors and how many more there
are
[untested]
my %count_of;
while (<DATA>) {
if (/Error: (.*)/) {
$count_of{$1}++;
}
}
for my $error (keys %count_of) {
print "There were $count_of{$error} instances of $error\n";
}
If that's not what you're looking for, please post a short-but-
complete script that demonstrates your problem, along with sample
input and desired output.
(Have you read the Posting Guidelines that are posted here twice a
week?)
Paul Lalli
------------------------------
Date: 6 Feb 2007 07:09:38 -0800
From: nigel@bouteyres.com
Subject: Re: French Accents appear incorrectly...
Message-Id: <1170774578.537073.59470@h3g2000cwc.googlegroups.com>
Hello again,
Thank you all for your input.
One solution to my problem was certainly to replace the embedded
characters with their html entities, but I thought that gf's comments
were perhaps a bit extreme - I'm working with French web site
developers and they simply type merrily away on their french keyboards
and therefore have embedded characters everywhere without too many
problems apparently.
In the end I opted for Marc Espie's solution which works just fine.
Thanks again,
Nigel
On Jan 30, 2:11 am, "Mumia W. (NOSPAM)" <paduille.4060.mumia.w
+nos...@earthlink.net> wrote:
> On 01/29/2007 11:36 AM, n...@bouteyres.com wrote:
>
> > Hi,
>
> > I'm creating a French web site. I've used Dreamweaver to develop most
> > of the site and have simply typed the french with accents. This all
> > displays fine. I've used DreamWeaver to create a library object for
> > the menu of the site (to avoid having multiple copies of exactly the
> > same html). I've now uploaded the library object to my server in order
> > that a perl program can read the contents of that file and output a
> > page of html complete with the menu for the site, but here I've run
> > into a problem: the accented letters are being displayed incorrectly.
> > For example an '=E9' (e with an acute accent) is being displayed as a
> > capital A with two dots over it folowed by the copyright symbol.
>
> It sounds like the library file was encoded in UTF8, and your perl
> program is reading it in iso-8859-1 (or something like that).
>
> > This
> > only happens for the text that is being read from the library object
> > file, an e with an acute accent typed into the perl program and
> > printed directly displays correctly. So I think the problem must be
> > related to how I read and then write the library object file. Here is
> > my code:
>
> > #######################################################################
> > ######################
>
> > # Open and read the contents of the headerMenu template...
> > open(INPUT, "../Library/headerMenuFr.lbi") ||
>
> Try this:
>
> open(INPUT, '<:utf8', "../Library/headerMenuFr.lbi") ||
>
>
>
>
>
> > &printErrorMessage("Error 001 : Can't open the 'headerMenuFr' file in:
> > $program");
>
> > while(<INPUT>) {
> > push(@records1,$_);
> > }
>
> > close(INPUT)|| &printErrorMessage("Error 012 : Can't close the
> > 'headerMenuFr' file in: $program");
>
> > # Now read through the header and print it...
>
> > $recnum =3D @records1;
> > for ($i=3D0; $i<$recnum; $i++) {
>
> > print $records1[$i];
> > }
>
> > #######################################################################
> > #####################
>
> > If anyone has any bright ideas as to what I'm doing wrong and how to
> > correct it, I'd like to hear them!
>
> > Thanks in advance,
>
> > Nigel
>
> Also read "perldoc -f binmode"
>
> HTH
>
> --
> Windows Vista and your freedom in conflict:http://techdirt.com/articles/2=
0061019/102225.shtml- Hide quoted text -
>
> - Show quoted text -
------------------------------
Date: Sun, 04 Feb 2007 14:55:09 -0500
From: Sherm Pendley <spamtrap@dot-app.org>
Subject: Re: FTP Link extraction (Was: Re: Request for a PERL script to download files)
Message-Id: <m2abztaeaq.fsf_-_@Sherm-Pendleys-Computer.local>
Mike Flannigan <mikeflan@earthlink.net> writes:
> What is the best way to do simple link extraction
> from an FTP site using native Perl. I have native perl
> HTML extraction scripts, and I can find the
> packaged Extract Link programs on the web, but
> I'd like a simple script that will do the job for me.
> I'm surprised I can't find this on my own.
>
> Is Net::FTP the way to go?
Have you tried passing ftp:// URLs to WWW::Mechanize?
sherm--
--
Web Hosting by West Virginians, for West Virginians: http://wv-www.net
Cocoa programming in Perl: http://camelbones.sourceforge.net
------------------------------
Date: Sun, 04 Feb 2007 00:56:14 +0100
From: Larry <dontmewithme@got.it>
Subject: get data from xml
Message-Id: <dontmewithme-2BD16B.00561404022007@reader2.news.tin.it>
Hi,
Allthough I could use some great XML module from cpan, I would
like to get data from the following xml file by using regexp:
<?xml version="1.0" encoding="UTF-8"?>
<plist version="1.0">
<dict>
<key></key>
<dict>
<key>1</key>
<dict>
<key>Track ID</key><integer>1</integer>
<key>Name</key><string>Brano 1</string>
</dict>
<key>2</key>
<dict>
<key>Track ID</key><integer>2</integer>
<key>Name</key><string>Brano 2</string>
</dict>
...etc...
</dict>
</dict>
I would like to get this and so forth:
<dict>
<key>Track ID</key><integer>1</integer>
<key>Name</key><string>Brano 1</string>
</dict>
sadly, i don't know how to go about it...
any help'll be appreciated
thanks
------------------------------
Date: Sun, 04 Feb 2007 00:21:32 GMT
From: "Jürgen Exner" <jurgenex@hotmail.com>
Subject: Re: get data from xml
Message-Id: <gG9xh.776$da1.452@trndny03>
Larry wrote:
> Allthough I could use some great XML module from cpan, I would
> like to get data from the following xml file by using regexp:
XML is not a regular language, therefore using a tool that is by design
limited to regular languages is flawed at best.(1)
> sadly, i don't know how to go about it...
Me neither. If you find a solution you may want to contact the ACM. It would
be a candidate for a Turing Award.
jue
(1): Yes, I know there are extensions in Perl's REs that make them more
powerful than plain regular. But still: trying got parse XML using REs is
ridiculous.
------------------------------
Date: Sun, 4 Feb 2007 11:49:18 +0100
From: "Dr.Ruud" <rvtol+news@isolution.nl>
Subject: Re: get data from xml
Message-Id: <eq4hgu.1m4.1@news.isolution.nl>
Larry schreef:
> Allthough I could use some great XML module from cpan, I would
> like to get data from the following xml file by using regexp:
There is nothing to like there, so please move on.
--
Affijn, Ruud
"Gewoon is een tijger."
------------------------------
Date: Sun, 04 Feb 2007 12:11:07 +0100
From: Michele Dondi <bik.mido@tiscalinet.it>
Subject: Re: get data from xml
Message-Id: <ikfbs298h4lg1782m42vf4eppobn6t5avm@4ax.com>
On Sun, 04 Feb 2007 00:56:14 +0100, Larry <dontmewithme@got.it> wrote:
> Allthough I could use some great XML module from cpan, I would
>like to get data from the following xml file by using regexp:
"Regular expressions" have not been regular any more for quite a long
time. However Perl's regexen are not *really* ready for this yet. I
suppose you will have to wait for Perl 6...
Michele
--
{$_=pack'B8'x25,unpack'A8'x32,$a^=sub{pop^pop}->(map substr
(($a||=join'',map--$|x$_,(unpack'w',unpack'u','G^<R<Y]*YB='
.'KYU;*EVH[.FHF2W+#"\Z*5TI/ER<Z`S(G.DZZ9OX0Z')=~/./g)x2,$_,
256),7,249);s/[^\w,]/ /g;$ \=/^J/?$/:"\r";print,redo}#JAPH,
------------------------------
Date: 6 Apr 2001 21:33:47 GMT (Last modified)
From: Perl-Users-Request@ruby.oce.orst.edu (Perl-Users-Digest Admin)
Subject: Digest Administrivia (Last modified: 6 Apr 01)
Message-Id: <null>
Administrivia:
#The Perl-Users Digest is a retransmission of the USENET newsgroup
#comp.lang.perl.misc. For subscription or unsubscription requests, send
#the single line:
#
# subscribe perl-users
#or:
# unsubscribe perl-users
#
#to almanac@ruby.oce.orst.edu.
NOTE: due to the current flood of worm email banging on ruby, the smtp
server on ruby has been shut off until further notice.
To submit articles to comp.lang.perl.announce, send your article to
clpa@perl.com.
#To request back copies (available for a week or so), send your request
#to almanac@ruby.oce.orst.edu with the command "send perl-users x.y",
#where x is the volume number and y is the issue number.
#For other requests pertaining to the digest, send mail to
#perl-users-request@ruby.oce.orst.edu. Do not waste your time or mine
#sending perl questions to the -request address, I don't have time to
#answer them even if I did know the answer.
------------------------------
End of Perl-Users Digest V11 Issue 104
**************************************
