[25784] in Perl-Users-Digest
Perl-Users Digest, Issue: 8023 Volume: 10
daemon@ATHENA.MIT.EDU (Perl-Users Digest)
Thu Apr 28 11:05:31 2005
Date: Thu, 28 Apr 2005 08:05:08 -0700 (PDT)
From: Perl-Users Digest <Perl-Users-Request@ruby.OCE.ORST.EDU>
To: Perl-Users@ruby.OCE.ORST.EDU (Perl-Users Digest)
Perl-Users Digest Thu, 28 Apr 2005 Volume: 10 Number: 8023
Today's topics:
Entirely new idea for a webpage <hackeras@gmail.com>
Re: Entirely new idea for a webpage <dave@sackheads.org>
Re: Entirely new idea for a webpage <sbryce@scottbryce.com>
Re: Entirely new idea for a webpage <hackeras@gmail.com>
Re: Extracting the link text <matternc@comcast.net>
Re: Extracting the link text chris-usenet@roaima.co.uk
Re: Extracting the link text <noreply@gunnar.cc>
Re: Extracting the link text <noreply@gunnar.cc>
how to count matches? <geoff.cox@notquitecorrectfreeuk.com>
Re: how to count matches? <john@castleamber.com>
Re: how to count matches? <do-not-use@invalid.net>
Re: how to count matches? <geoff.cox@notquitecorrectfreeuk.com>
Re: how to count matches? <geoff.cox@notquitecorrectfreeuk.com>
Re: one-time and one-line programs <bernard.el-haginDODGE_THIS@lido-tech.net>
Re: Re ANN: A new scripting language Tao 0.9.0 beta rel <donal.k.fellows@manchester.ac.uk>
regular expression <luciano@tolomei.name>
Re: regular expression <john@castleamber.com>
Re: regular expression <pilkowsk@informatik.uni-marburg.de>
Re: regular expression <tadmc@augustmail.com>
Re: regular expression <m@remove.this.part.rtij.nl>
Re: regular expression <luciano@tolomei.name>
using LWP to get a very large file (justme)
Re: using LWP to get a very large file <john@castleamber.com>
Re: using LWP to get a very large file <mark.clementsREMOVETHIS@wanadoo.fr>
Digest Administrivia (Last modified: 6 Apr 01) (Perl-Users-Digest Admin)
----------------------------------------------------------------------
Date: Thu, 28 Apr 2005 11:06:43 +0300
From: Nikos <hackeras@gmail.com>
Subject: Entirely new idea for a webpage
Message-Id: <d4q5ij$lk7$1@nic.grnet.gr>
Hello and Good Morning,
I'll try to be as exact as possibl to what iam trying to do:
Today i decided to change entirely my index.pl page and template and to
create a new one.
I will no longer presentate religious scripts from my webpage but i
think its a good idea to create a webpage
that will show up a list of movies that i do have from the current drop
down menu that i already have or better just
tweak the already games.pl script to a movies.pl script that would
presentate the movies as it now have presentated the
games.
Users have to signup first and their username and password will be
stored in a mysql users database and then everytime
they visit my webpage they would have to log in first and after that
they are ready to view and select movies from
a dropdown menu and add them to a basket or something.
Then they will hit "done" and their selection would be stored in a mysql
order database that will have some connection
to the users database table.
Then if they want to see what they ordered they could be able to press a
"show what i ordered" button and then that will
be presenatte to them. I currently have a show.pl so that too is almost
ready except that its a cgi one and needs
converting to a template.
Will you please help me make the appropriate tweaks to the already
created scripts to implement that.
Iam trying too but unfortunately iam no expert.
Thank you.
ps. iam trying now to chnage the column names from games and otehr
database tables to more appropriate ones.
Do you agree with those names of the database tables and the columns in
it to what i want to do? Dis i pick them correctly?
I tried the names to be as much self-explanatory as i can:
$dbh->do( "CREATE DATABASE user (username text, password text, name
text, surname text, address text, phone text, mail text)" );
$dbh->do( "CREATE DATABASE userselect (username text auto_increment
primary key, moviename text" ) or die $!;
$dbh->do( "CREATE DATABASE userdesire (title text, remark text, mail
text, date text, host text )" ) or die $!;
$dbh->do( "CREATE DATABASE movies (moviename text, moviedesc text,
moviecount int )" ) or die $!;
------------------------------
Date: Thu, 28 Apr 2005 08:55:04 +0000 (UTC)
From: Dave Peacock <dave@sackheads.org>
Subject: Re: Entirely new idea for a webpage
Message-Id: <slrnd7197k.pl.dave@david-peacocks-computer.local>
In article <d4q5ij$lk7$1@nic.grnet.gr>, Nikos wrote:
> Hello and Good Morning,
>
> I'll try to be as exact as possibl to what iam trying to do:
<snip>
Dude.
Seriously.
http://learn.perl.org/
*plonk*
--
Dave Peacock - dave@sackheads.org
------------------------------
Date: Thu, 28 Apr 2005 08:24:57 -0600
From: Scott Bryce <sbryce@scottbryce.com>
Subject: Re: Entirely new idea for a webpage
Message-Id: <fJidneeDWt-kb-3fRVn-tQ@comcast.com>
Nikos wrote:
> Will you please help me make the appropriate tweaks to the already
> created scripts to implement that.
No. That isn't what this newsgroup is about.
> Do you agree with those names of the database tables and the columns in
> it to what i want to do? Dis i pick them correctly?
This newsgroup is not about choosing field names for your databases.
------------------------------
Date: Thu, 28 Apr 2005 17:51:51 +0300
From: Nikos <hackeras@gmail.com>
Subject: Re: Entirely new idea for a webpage
Message-Id: <d4qta7$ehd$1@nic.grnet.gr>
Scott Bryce wrote:
> Nikos wrote:
>
>> Will you please help me make the appropriate tweaks to the already
>> created scripts to implement that.
>
>
> No. That isn't what this newsgroup is about.
>
>
>> Do you agree with those names of the database tables and the columns
>> in it to what i want to do? Dis i pick them correctly?
>
>
> This newsgroup is not about choosing field names for your databases.
I will need some guidelines, i have already started it but i dont knwo
how to proceed.
Well i think columns names do matter because they have to be
self-explanatory and help user think why they are existing.
I don't know how to start to what i have asked, can you please help me out!
For starters i must have a button that a user will click in order to
sign up and then login with user name and password.
I just dont know hot to write it neither i can afforf paying someone
else to write it. I can though try step by step something.
ps. Hiring a Perl programmer write something costs 147$ per hour? Oh
boy, this job is a goldmine if someone knows how to program.
------------------------------
Date: Thu, 28 Apr 2005 04:19:46 -0400
From: Chris Mattern <matternc@comcast.net>
Subject: Re: Extracting the link text
Message-Id: <F4-dnSbDXtM_Ae3fRVn-2g@comcast.com>
Fritz Bayer wrote:
> Hi,
>
> I would like to extract all the links from a html page, which I store
> in a string variable.
>
> For each link, I would also like to print out the link text, however,
> omitting ALL possible tags in which the text could be embedded.
Use one of the modules for parsing HTML.
>
> I'm looking for a regular expression, which does just that.
No, you aren't, because there ain't no such thing.
> Can
> somebody help me out?
>
> Fritz
--
Christopher Mattern
"Which one you figure tracked us?"
"The ugly one, sir."
"...Could you be more specific?"
------------------------------
Date: Thu, 28 Apr 2005 09:16:26 +0100
From: chris-usenet@roaima.co.uk
Subject: Re: Extracting the link text
Message-Id: <pni6k2-tq6.ln1@news.roaima.co.uk>
Fritz Bayer <fritz-bayer@web.de> wrote:
> I would like to extract all the links from a html page [...]
> I'm looking for a regular expression, which does just that. Can
> somebody help me out?
perldoc -q "remove html"
Chris
------------------------------
Date: Thu, 28 Apr 2005 13:07:28 +0200
From: Gunnar Hjalmarsson <noreply@gunnar.cc>
Subject: Re: Extracting the link text
Message-Id: <3dbuihF6r8g91U1@individual.net>
Chris Mattern wrote:
> Fritz Bayer wrote:
>> I would like to extract all the links from a html page, which I store
>> in a string variable.
>>
>> For each link, I would also like to print out the link text, however,
>> omitting ALL possible tags in which the text could be embedded.
>
> Use one of the modules for parsing HTML.
HTML::LinkExtor sounds promising. :)
--
Gunnar Hjalmarsson
Email: http://www.gunnar.cc/cgi-bin/contact.pl
------------------------------
Date: Thu, 28 Apr 2005 13:11:11 +0200
From: Gunnar Hjalmarsson <noreply@gunnar.cc>
Subject: Re: Extracting the link text
Message-Id: <3dbuphF6rgsa9U1@individual.net>
chris-usenet@roaima.co.uk wrote:
> Fritz Bayer <fritz-bayer@web.de> wrote:
>>I would like to extract all the links from a html page [...]
>>I'm looking for a regular expression, which does just that. Can
>>somebody help me out?
>
> perldoc -q "remove html"
Better yet:
perldoc -q "extract URLs"
--
Gunnar Hjalmarsson
Email: http://www.gunnar.cc/cgi-bin/contact.pl
------------------------------
Date: Thu, 28 Apr 2005 10:29:04 GMT
From: Geoff Cox <geoff.cox@notquitecorrectfreeuk.com>
Subject: how to count matches?
Message-Id: <mae171hqo9v2kavq9pr9smr39ggoeqblf2@4ax.com>
Hello
I want to use regex to find/replace say <jim> with $num-fred<jim> so
that $num increase by 1 for each occurrence.
It might that there are 2 of <jim> on a line so my code below is not
working - how should I change it? Clues please!
$num = 1;
if ($line =~ /<jim>/) {
my $orig = "<jim>";
my $new = "$num-jim";
$line =~ s/$orig/$new/g;
$num++;
}
Thanks
Geoff
------------------------------
Date: 28 Apr 2005 10:48:10 GMT
From: John Bokma <john@castleamber.com>
Subject: Re: how to count matches?
Message-Id: <Xns96463AC69DFE8castleamber@130.133.1.4>
Geoff Cox wrote:
> Hello
>
> I want to use regex to find/replace say <jim> with $num-fred<jim> so
> that $num increase by 1 for each occurrence.
>
> It might that there are 2 of <jim> on a line so my code below is not
> working - how should I change it? Clues please!
>
> $num = 1;
>
> if ($line =~ /<jim>/) {
> my $orig = "<jim>";
> my $new = "$num-jim";
> $line =~ s/$orig/$new/g;
> $num++;
Not tested: s{$orig}{ $num++ . "-jim" }ge;
note the e.
Also note that the first part of your posting and your actual code are in
disagreement.
--
John Small Perl scripts: http://johnbokma.com/perl/
Perl programmer available: http://castleamber.com/
Happy Customers: http://castleamber.com/testimonials.html
------------------------------
Date: 28 Apr 2005 12:51:38 +0200
From: Arndt Jonasson <do-not-use@invalid.net>
Subject: Re: how to count matches?
Message-Id: <yzd64y718ed.fsf@invalid.net>
Geoff Cox <geoff.cox@notquitecorrectfreeuk.com> writes:
> I want to use regex to find/replace say <jim> with $num-fred<jim> so
> that $num increase by 1 for each occurrence.
>
> It might that there are 2 of <jim> on a line so my code below is not
> working - how should I change it? Clues please!
>
> $num = 1;
>
> if ($line =~ /<jim>/) {
> my $orig = "<jim>";
> my $new = "$num-jim";
> $line =~ s/$orig/$new/g;
> $num++;
Use the 'e' modifier:
if ($line =~ /<jim>/) {
my $orig = "<jim>";
$line =~ s/$orig/$num++ . "-jim"/ge;
------------------------------
Date: Thu, 28 Apr 2005 12:37:50 GMT
From: Geoff Cox <geoff.cox@notquitecorrectfreeuk.com>
Subject: Re: how to count matches?
Message-Id: <g7m171h3bmdtfg8m454fpe1phhd4c1ospg@4ax.com>
On 28 Apr 2005 10:48:10 GMT, John Bokma <john@castleamber.com> wrote:
>> $num = 1;
>>
>> if ($line =~ /<jim>/) {
>> my $orig = "<jim>";
>> my $new = "$num-jim";
>> $line =~ s/$orig/$new/g;
>> $num++;
ThanksJOhn - I have it working now.
Cheers
Geoff
>
>Not tested: s{$orig}{ $num++ . "-jim" }ge;
>
>note the e.
>
>Also note that the first part of your posting and your actual code are in
>disagreement.
------------------------------
Date: Thu, 28 Apr 2005 12:38:26 GMT
From: Geoff Cox <geoff.cox@notquitecorrectfreeuk.com>
Subject: Re: how to count matches?
Message-Id: <s8m171l95ltlprdsr1bkt11d5tmp84it07@4ax.com>
On 28 Apr 2005 12:51:38 +0200, Arndt Jonasson <do-not-use@invalid.net>
wrote:
>Use the 'e' modifier:
>
> if ($line =~ /<jim>/) {
> my $orig = "<jim>";
> $line =~ s/$orig/$num++ . "-jim"/ge;
Thanks Arndt - all is well!
Cheers
Geoff
------------------------------
Date: Thu, 28 Apr 2005 09:01:36 +0200
From: "Bernard El-Hagin" <bernard.el-haginDODGE_THIS@lido-tech.net>
Subject: Re: one-time and one-line programs
Message-Id: <Xns96465BD341899elhber1lidotechnet@62.89.127.66>
Wes Groleau <groleau+news@freeshell.org> wrote:
> Anno Siegel wrote:
>> Abigail <abigail@abigail.nl> wrote in comp.lang.perl.misc:
>>
>>> perl -pe '}{*_=*.}{' file # wc -l
>>
>>
>> Nice. Why {*_=*.} and not {$_=$.}? Just obfuscation?
>
> Speaking of obfuscation, what's "nice" about using
> fifteen characters, over half punctuation, to replace five?
It's a *Perl* one-liner. Duh.
--
Cheers,
Bernard
------------------------------
Date: Thu, 28 Apr 2005 14:47:00 +0100
From: "Donal K. Fellows" <donal.k.fellows@manchester.ac.uk>
Subject: Re: Re ANN: A new scripting language Tao 0.9.0 beta released!
Message-Id: <d4qpd2$1bco$1@godfrey.mcc.ac.uk>
Fu Limin wrote:
>>In what ways is your language distinctive?
>
> Well, I think the most distinctive feature is the integration of powerful
> text processing ability and powerful numerical computation with a
> simple syntax for them. I also try to support a simple and transparent
> interface to C++. Though there is still a long way to go, I think the
> status of Tao is promising, in fact I already benefit from it in my
> research where both text processing and numerically computation are
> important, for which I "suffered" a lot from using Perl in the beginning,
> which drived me to design my own one ^_^
Ah, the dilemma between whether to create your own or extend what is
already there. Myself, I prefer the second option because that lets more
people use it with a lower overhead; the barrier to entry on an entire
programming language is quite a bit higher even if the integration it
affords you is more closely tuned.
>> How does developing your own
>> language (as opposed to making stronger one of the languages whose
>> newsgroups you've posted to) make the sum of human experience richer?
>
> For the existing languages, I think they have already stabilized a kind of
> philosophy, and something have already got complicated. So I think it
> would be more difficult for me to dive into them. Anyway, I believe it
> worth trying to design a new language, if I found it more usesful in some
> work than the other languages, somebody will also find this. A new option
> is not bad thing.
Every programming language is really a different way of thinking about
programming problems. The hard bit is determining whether you're either
covering some new fraction of the noosphere, or if not, covering some
useful distinct subset of it instead. It is also a curiosity that
sometimes reducing the number of things you do can make a language more
suitable for some kinds of tasks (security often seems to work out that
way for some reason).
[Below, where I've not replied to some point made, it's because I don't
feel I have anything more to say on the matter at the moment.]
>>Odd things I've noticed in a quick trawl through your docs:
>> * What no bit-ops? And why give ^ a non C/C++ meaning?
>
> My using of ^ as power is influenced by Matlab. In the beginning I didn't
> know it is also used in C/C++.
An alternative would be to use ** for exponentiation, deriving from
Fortran, and give the bit-ops their conventional C/C++ meanings.
>> * What sort of characters are you using?
>
> For strings, I use stl::string, so I didn't care about this.
> But for byte type numeric array, I used unsigned char.
The real Q here is chars vs. characters. :^)
>> * How does passing by reference square with constant arguments?
>
> In Tao constants are also represented by objects, their passing by reference
> are the same as other types of objects.
So, that would mean that it is possible for a constant to be altered? My
goodness me, that reminds me of some awfully buggy code I've seen in the
past! Please make sure that you can't ever change the actual constants
that come from your program (setting 0 to 1 is a good way to make things
go belly up!)
>> * Your I/O interface has a /long/ way to go!
>
> Right. I haven't concentrated on this part. The status of IO support
> deosn't conform to the version number which increased too rapidly :-)
Concentrate on producing a system for externally-defined loadable
modules and put (most of) your IO code in one of those. :^) Some people
have had real success with doing that.
>> * Given that you are using mutable objects, do you have a way for
>> someone to force a duplicate of an arbitrary object? That would make
>> doing things like security separation much easier.
>
> If you mean data object, yes, but it's not available for users yet. Now it is
> mainly used for passing Tao data types to C++ modules, where each Tao
> object can be copied and converted into its C++ equivalent object
> (though internally they are all C++ objects, but the object used in C++
> modules is different from the one used in Tao interpreter, e.g. TaoHash is
> used in the interpreter, while TcHash is used in C++ module, the conversion
> or copying keep its structure including cyclic ones).
Firstly, control of when things get duplicated is very important,
because doing a deep copy of a large object graph can be expensive. The
easiest way to handle this would be to put in a monadic operator that
duplicates an object.
Secondly, is it really necessary to have TaoHash and TcHash as separate
things? Unifying your entity system might be a big win over the longer term.
Donal.
------------------------------
Date: Thu, 28 Apr 2005 12:16:18 +0200
From: Luciano Tolomei <luciano@tolomei.name>
Subject: regular expression
Message-Id: <d4qcrl$sar$1@newsreader.mailgate.org>
i have to match some content in an html file.
the file is a single line.
i have to get some cell content from it.
i have build an expression but it match a lot of <tr></tr> together instead of matching on a time...
the line is more complex (with rowspan... ) and i have to retrieve a lot of cell's and there are a lot of rows.
but here a simplified example:
<tr><td><font color=\"#EFAD00\">xxxx</font></td></tr><tr><td><font color=\"#EFAD00\">yyyy</font></td></tr>
<tr>(.*)<font color=\"#EFAD00\">([^<]*)(.*)</tr>
it match yyyy, i need to match xxxx and yyyy.
i think that i have to change (.*) to make it match everything but not the </tr>
but i do not know how to do it.
------------------------------
Date: 28 Apr 2005 10:27:40 GMT
From: John Bokma <john@castleamber.com>
Subject: Re: regular expression
Message-Id: <Xns96463728C82F0castleamber@130.133.1.4>
Luciano Tolomei wrote:
> i have to match some content in an html file.
So put this in your subject. A regexp might not be the answer.
Look at File::Slurp to read the file in one go.
[ snip ]
> i think that i have to change (.*) to make it match everything but not
> the </tr> but i do not know how to do it.
I think you have to have a peek at HTML::TreeBuilder
see:
http://johnbokma.com/perl/froogle-script.html
http://johnbokma.com/perl/phpbb-remote-backup.html
for examples.
--
John Small Perl scripts: http://johnbokma.com/perl/
Perl programmer available: http://castleamber.com/
Happy Customers: http://castleamber.com/testimonials.html
------------------------------
Date: Thu, 28 Apr 2005 14:33:16 +0200
From: Fabian Pilkowski <pilkowsk@informatik.uni-marburg.de>
Subject: Re: regular expression
Message-Id: <3dc3fcF6rk66nU1@individual.net>
* Luciano Tolomei schrieb:
>
> i have to match some content in an html file. the file is a single
> line. i have to get some cell content from it. i have build an
> expression but it match a lot of <tr></tr> together instead of
> matching on a time...
> the line is more complex (with rowspan... ) and i have to retrieve
> a lot of cell's and there are a lot of rows. but here a simplified
> example:
>
> <tr><td><font color=\"#EFAD00\">xxxx</font></td></tr><tr><td><font
> color=\"#EFAD00\">yyyy</font></td></tr>
>
> <tr>(.*)<font color=\"#EFAD00\">([^<]*)(.*)</tr>
>
> it match yyyy, i need to match xxxx and yyyy. i think that i have to
> change (.*) to make it match everything but not the </tr> but i do
> not know how to do it.
It seems you want to strip all the HTML tags from your data. Once you
were doing that with your example above the string "xxxx yyyy" remains.
Have a look for HTML::Strip on CPAN. This module is for stripping all
the HTML stuff. Afterwards you could try to split() your data as usual.
regards,
fabian
------------------------------
Date: Thu, 28 Apr 2005 07:59:14 -0500
From: Tad McClellan <tadmc@augustmail.com>
Subject: Re: regular expression
Message-Id: <slrnd71nh2.9gs.tadmc@magna.augustmail.com>
[ Please limit your line lengths to the conventional 70-72 characters. ]
Luciano Tolomei <luciano@tolomei.name> wrote:
> Subject: regular expression
That is not the Right Tool for your job.
> i have to match some content in an html file.
You should use a module that understands HTML for processing HTML data.
> i have to get some cell content from it.
> the line is more complex (with rowspan... ) and i have to retrieve
> a lot of cell's and there are a lot of rows.
There is a module that can handle that for you, no need to
reinvent that wheel.
> i think that i have to change (.*) to make it match everything
I think you have to change to:
use HTML::TableExtract;
:-)
--
Tad McClellan SGML consulting
tadmc@augustmail.com Perl programming
Fort Worth, Texas
------------------------------
Date: Thu, 28 Apr 2005 15:30:45 +0200
From: Martijn Lievaart <m@remove.this.part.rtij.nl>
Subject: Re: regular expression
Message-Id: <5557k2-cfr.ln1@news.rtij.nl>
On Thu, 28 Apr 2005 12:16:18 +0200, Luciano Tolomei wrote:
> i think that i have to change (.*) to make it match everything but not the </tr>
> but i do not know how to do it.
Have a look at (.*?) instead of (.*). Bu as others already noted, regexps
are not the best way to tackle this.
M4
--
Redundancy is a great way to introduce more single points of failure.
------------------------------
Date: Thu, 28 Apr 2005 16:45:18 +0200
From: Luciano Tolomei <luciano@tolomei.name>
Subject: Re: regular expression
Message-Id: <d4qsk0$gv1$1@newsreader.mailgate.org>
Tad McClellan wrote:
> I think you have to change to:
> use HTML::TableExtract;
> :-)
i do
really thanks.
------------------------------
Date: 28 Apr 2005 06:11:43 -0700
From: mik3l3374@gmail.com (justme)
Subject: using LWP to get a very large file
Message-Id: <e3202a16.0504280511.435094f8@posting.google.com>
hi
I have a remote machine running some application that generates very
large
log files in terms of Mb...say average 60Mb.
Normally, we need to connect to this remote machine by keying in a URL
in a browser such as http://remote:123/logs. Then the browser will
display tab delimited columns of data.
What i wanted to do use the perl LWP module to get this log as i do
not want to go to the physical machine to use the browser to get the
logs. Then according to some filtering parameters, display the logs
according to the filters. For example, if filter by a certain date,
then it will display the contents based on that date.
Questions:
1) this log file is very big ,60Mb at least. So , is LWP the one to
use..or is there a better module to deal with large files..
2) While getting the log file, is it better in terms of memory usage
to parse the data "on the fly" or get the whole file and do the
parsing afterwards..
3) becos i am not at the physical machine, i can't really do something
lika a "tail" feature which displays the data "realtime"... any ways
to do a "tail" on the log file remotely?
thanks..
------------------------------
Date: 28 Apr 2005 13:20:56 GMT
From: John Bokma <john@castleamber.com>
Subject: Re: using LWP to get a very large file
Message-Id: <Xns9646548962A2Acastleamber@130.133.1.4>
justme wrote:
> hi
>
> I have a remote machine running some application that generates very
> large
> log files in terms of Mb...say average 60Mb.
> Normally, we need to connect to this remote machine by keying in a URL
> in a browser such as http://remote:123/logs. Then the browser will
> display tab delimited columns of data.
>
> What i wanted to do use the perl LWP module to get this log as i do
> not want to go to the physical machine to use the browser to get the
> logs. Then according to some filtering parameters, display the logs
> according to the filters. For example, if filter by a certain date,
> then it will display the contents based on that date.
>
> Questions:
> 1) this log file is very big ,60Mb at least. So , is LWP the one to
> use..or is there a better module to deal with large files..
> 2) While getting the log file, is it better in terms of memory usage
> to parse the data "on the fly" or get the whole file and do the
> parsing afterwards..
> 3) becos i am not at the physical machine, i can't really do something
> lika a "tail" feature which displays the data "realtime"... any ways
> to do a "tail" on the log file remotely?
I use plink for stuff like this (part of PuTTY), e.g.:
plink -ssh -pw password user@example.com gzip -9 -c logs/error_log |
gzip -d > site/logs/error_log
But this only works if you can get the log via SSH.
My log is uncompressed around 130 MB :-)
--
John Small Perl scripts: http://johnbokma.com/perl/
Perl programmer available: http://castleamber.com/
Happy Customers: http://castleamber.com/testimonials.html
------------------------------
Date: Thu, 28 Apr 2005 16:47:09 +0200
From: Mark Clements <mark.clementsREMOVETHIS@wanadoo.fr>
Subject: Re: using LWP to get a very large file
Message-Id: <4270f770$0$3128$8fcfb975@news.wanadoo.fr>
John Bokma wrote:
> justme wrote:
>>I have a remote machine running some application that generates very
>>large
>>log files in terms of Mb...say average 60Mb.
>>Normally, we need to connect to this remote machine by keying in a URL
>>in a browser such as http://remote:123/logs. Then the browser will
>>display tab delimited columns of data.
<snip>
>>Questions:
>>1) this log file is very big ,60Mb at least. So , is LWP the one to
>>use..or is there a better module to deal with large files..
>>2) While getting the log file, is it better in terms of memory usage
>>to parse the data "on the fly" or get the whole file and do the
>>parsing afterwards..
>>3) becos i am not at the physical machine, i can't really do something
>>lika a "tail" feature which displays the data "realtime"... any ways
>>to do a "tail" on the log file remotely?
>
>
> I use plink for stuff like this (part of PuTTY), e.g.:
>
> plink -ssh -pw password user@example.com gzip -9 -c logs/error_log |
> gzip -d > site/logs/error_log
>
> But this only works if you can get the log via SSH.
>
> My log is uncompressed around 130 MB :-)
>
Numbered answers:
1. You might be better off using something like wget if you want to do
this, not because LWP can't handle files of that size, but because it
gives you a bunch of functionality (eg resume) without doing any
programming.
2. it depends on the structure of the data and the amount of memory on
the parsing machine. 60MB isn't a vast amount of data to suck in in one
go, but bear in mind that any in-memory data structure you build will
take more space than this. Check out perldoc -q memory.
3. Well: if he can get SSH access, then he could run tail that way.
Mark
------------------------------
Date: 6 Apr 2001 21:33:47 GMT (Last modified)
From: Perl-Users-Request@ruby.oce.orst.edu (Perl-Users-Digest Admin)
Subject: Digest Administrivia (Last modified: 6 Apr 01)
Message-Id: <null>
Administrivia:
#The Perl-Users Digest is a retransmission of the USENET newsgroup
#comp.lang.perl.misc. For subscription or unsubscription requests, send
#the single line:
#
# subscribe perl-users
#or:
# unsubscribe perl-users
#
#to almanac@ruby.oce.orst.edu.
NOTE: due to the current flood of worm email banging on ruby, the smtp
server on ruby has been shut off until further notice.
To submit articles to comp.lang.perl.announce, send your article to
clpa@perl.com.
#To request back copies (available for a week or so), send your request
#to almanac@ruby.oce.orst.edu with the command "send perl-users x.y",
#where x is the volume number and y is the issue number.
#For other requests pertaining to the digest, send mail to
#perl-users-request@ruby.oce.orst.edu. Do not waste your time or mine
#sending perl questions to the -request address, I don't have time to
#answer them even if I did know the answer.
------------------------------
End of Perl-Users Digest V10 Issue 8023
***************************************
