[15796] in Perl-Users-Digest
Perl-Users Digest, Issue: 3209 Volume: 9
daemon@ATHENA.MIT.EDU (Perl-Users Digest)
Tue May 30 14:15:54 2000
Date: Tue, 30 May 2000 11:15:25 -0700 (PDT)
From: Perl-Users Digest <Perl-Users-Request@ruby.OCE.ORST.EDU>
To: Perl-Users@ruby.OCE.ORST.EDU (Perl-Users Digest)
Message-Id: <959710525-v9-i3209@ruby.oce.orst.edu>
Content-Type: text
Perl-Users Digest Tue, 30 May 2000 Volume: 9 Number: 3209
Today's topics:
server overload <diab.litoNOdiSPAM@usa.net.invalid>
Re: server overload (Kragen Sitaker)
Re: server overload <andkaha@my-deja.com>
Re: server overload nobull@mail.com
Re: server overload <diab.litoNOdiSPAM@usa.net.invalid>
Strategies for determining if there's a memory leak? <dwhaskin@earthlink.net>
Re: Translate foreign characters to English <lr@hpl.hp.com>
Re: Waxing Philosophical (Jerome O'Neil)
Re: which perl module for http perl script <tina@streetmail.com>
Re: which perl module for http perl script (Kragen Sitaker)
Re: which perl module for http perl script nobull@mail.com
Digest Administrivia (Last modified: 16 Sep 99) (Perl-Users-Digest Admin)
----------------------------------------------------------------------
Date: Tue, 30 May 2000 08:14:59 -0700
From: Diablito <diab.litoNOdiSPAM@usa.net.invalid>
Subject: server overload
Message-Id: <06252180.9aed4508@usw-ex0105-038.remarq.com>
I have this code on a script:
$a{'1'} = "aaa";
$a{'2'} = "bbb";
$a{'3'} = "ccc";
$a{'4'} = "ddd";
..
$a{'999'} = "abc";
$a{'1000'} = "xyz";
..
$t=a number;
$m=$a{$t};
What happens on the server?it has to load all the 1000 variables
in memory or just the one that will be used ($a{$t})?
If it loads all the variables,is that a problem(the script is
300 Kb and runs about 1000 times a day)?
If there is a best way to do this thing,I would like to
know.Thanks everyone.
Mario
* Sent from RemarQ http://www.remarq.com The Internet's Discussion Network *
The fastest and easiest way to search and participate in Usenet - Free!
------------------------------
Date: Tue, 30 May 2000 16:18:15 GMT
From: kragen@dnaco.net (Kragen Sitaker)
Subject: Re: server overload
Message-Id: <bRRY4.126214$681.2376921@news-east.usenetserver.com>
In article <06252180.9aed4508@usw-ex0105-038.remarq.com>,
Diablito <diab.litoNOdiSPAM@usa.net.invalid> wrote:
>I have this code on a script:
>
>$a{'1'} = "aaa";
>$a{'2'} = "bbb";
>$a{'3'} = "ccc";
>$a{'4'} = "ddd";
>..
>$a{'999'} = "abc";
>$a{'1000'} = "xyz";
>..
>$t=a number;
>$m=$a{$t};
>
>What happens on the server?it has to load all the 1000 variables
>in memory or just the one that will be used ($a{$t})?
All 1000.
>If it loads all the variables,is that a problem(the script is
>300 Kb and runs about 1000 times a day)?
It might be a problem. A 300K script is going to take a while to
parse, but probably not more than a few seconds. But that might be
unacceptably slow response, depending on your application. Also, it
will use a fair bit of memory.
You might look into FastCGI or mod_perl if it is a problem.
>If there is a best way to do this thing,I would like to
>know.Thanks everyone.
Perhaps store %a in a DBM file?
--
<kragen@pobox.com> Kragen Sitaker <http://www.pobox.com/~kragen/>
The Internet stock bubble didn't burst on 1999-11-08. Hurrah!
<URL:http://www.pobox.com/~kragen/bubble.html>
The power didn't go out on 2000-01-01 either. :)
------------------------------
Date: Tue, 30 May 2000 16:08:19 GMT
From: Andreas Kahari <andkaha@my-deja.com>
Subject: Re: server overload
Message-Id: <8h0p13$f0h$1@nnrp1.deja.com>
In article <06252180.9aed4508@usw-ex0105-038.remarq.com>,
Diablito <diab.litoNOdiSPAM@usa.net.invalid> wrote:
> I have this code on a script:
>
> $a{'1'} = "aaa";
> $a{'2'} = "bbb";
> $a{'3'} = "ccc";
> $a{'4'} = "ddd";
> ..
> $a{'999'} = "abc";
> $a{'1000'} = "xyz";
> ..
It looks as if an array would be of more use here (if the indicies are
always integer). A hash has larger storage requirements and is a tiny
bit slower when accessing elements.
/A
--
# Andreas Kähäri, <URL:http://hello.to/andkaha/>.
# All junk e-mail is reported to the
# appropriate authorities, no exceptions.
Sent via Deja.com http://www.deja.com/
Before you buy.
------------------------------
Date: 30 May 2000 17:43:27 +0100
From: nobull@mail.com
Subject: Re: server overload
Message-Id: <u9k8gc56q8.fsf@wcl-l.bham.ac.uk>
Diablito <diab.litoNOdiSPAM@usa.net.invalid> writes:
> I have this code on a script:
>
> $a{'1'} = "aaa";
> $a{'2'} = "bbb";
> $a{'3'} = "ccc";
> $a{'4'} = "ddd";
> ..
> $a{'999'} = "abc";
> $a{'1000'} = "xyz";
> ..
> $t=a number;
> $m=$a{$t};
>
> What happens on the server?it has to load all the 1000 variables
> in memory or just the one that will be used ($a{$t})?
Perl does not contain a precient optomizer. Statements are complied
and executed even if later events mean it wasn't necessary.
> If it loads all the variables,is that a problem(the script is
> 300 Kb and runs about 1000 times a day)?
It is a problem if and only if your users and/or sysop tell you that
it's a problem.
> If there is a best way to do this thing,I would like to
> know.
No there probably is no single best way but using Apache+mod_perl,
making %a global and putting the initialisation of %a in a BEGIN {}
block may help.
--
\\ ( )
. _\\__[oo
.__/ \\ /\@
. l___\\
# ll l\\
###LL LL\\
------------------------------
Date: Tue, 30 May 2000 10:23:21 -0700
From: Diablito <diab.litoNOdiSPAM@usa.net.invalid>
Subject: Re: server overload
Message-Id: <04870b98.37b350ee@usw-ex0106-047.remarq.com>
I am wondering if is it a good idea to create a *.txt file for
each variable then including just the one that is used?
Mario
In article <bRRY4.126214$681.2376921@news-
east.usenetserver.com>, kragen@dnaco.net (Kragen Sitaker) wrote:
>In article <06252180.9aed4508@usw-ex0105-038.remarq.com>,
>Diablito <diab.litoNOdiSPAM@usa.net.invalid> wrote:
>>I have this code on a script:
>>
>>$a{'1'} = "aaa";
>>$a{'2'} = "bbb";
>>$a{'3'} = "ccc";
>>$a{'4'} = "ddd";
>>..
>>$a{'999'} = "abc";
>>$a{'1000'} = "xyz";
>>..
>>$t=a number;
>>$m=$a{$t};
>>
>>What happens on the server?it has to load all the 1000
variables
>>in memory or just the one that will be used ($a{$t})?
>
>All 1000.
>
>>If it loads all the variables,is that a problem(the script is
>>300 Kb and runs about 1000 times a day)?
>
>It might be a problem. A 300K script is going to take a while
to
>parse, but probably not more than a few seconds. But that
might be
>unacceptably slow response, depending on your application.
Also, it
>will use a fair bit of memory.
>
>You might look into FastCGI or mod_perl if it is a problem.
>
>>If there is a best way to do this thing,I would like to
>>know.Thanks everyone.
>
>Perhaps store %a in a DBM file?
>--
><kragen@pobox.com> Kragen Sitaker
<http://www.pobox.com/~kragen/>
>The Internet stock bubble didn't burst on 1999-11-08. Hurrah!
><URL:http://www.pobox.com/~kragen/bubble.html>
>The power didn't go out on 2000-01-01 either. :)
>
>
* Sent from RemarQ http://www.remarq.com The Internet's Discussion Network *
The fastest and easiest way to search and participate in Usenet - Free!
------------------------------
Date: Tue, 30 May 2000 16:52:41 GMT
From: Denis Haskin <dwhaskin@earthlink.net>
Subject: Strategies for determining if there's a memory leak?
Message-Id: <3933F141.3BC80F7C@earthlink.net>
Environment: Perl 5.6, compiled with -DDEBUGGING, on Solaris 2.6.
I have a server process (not a web server, although it does communicate
with clients over TCP/IP) that I suspect has a memory leak. I'm looking
for suggestions as to how I would best go about investigating this and
(hopefully) correcting it, assuming the leak isn't in some external
module I'm using.
The reason I think I have a leak is that, after each request to the
server, the resident set size (rss) goes up by about a meg, and the
virtual size (vsz) goes up by more like 1.5-2 megs. I would not expect
this to happen, and needless to say, the server can't service that many
requests before memory is exhausted.
The server is designed quite OO and uses quite a number of existing
modules (see below). I use a lot of local-scoped variables (and I have
'use strict' turned on), including locally-scoped objects (which I am
assuming get correctly unreferenced when they go out of scope).
As I indicated above, I tried compiling with -DDEBUGGING and running it
with -Dm, but once the thread of execution goes into one of my (own)
packages, the malloc/free messages stop appearing. Hmm.
I'm currently investigating the Devel::Leak and Devel::Symdump modules,
but it's not yet clear to me whether/how these would identify memory
that isn't getting correctly freed (and I actually haven't built
Devel::Symdump on this box yet).
I wonder if PERL_DESTRUCT_LEVEL might help, but (as others have already
observed) there's very little doc on what exactly that does. I guess
it's time to dive into the C code.
I've done lots of newsgroup searching and checked the perldbug db, and
there are some indications that perhaps some of the modules I'm using
(e.g. MLDBM) may have leaks. Agh! But again, how would I best use the
tools I've mentioned above (or others?) to determine this for certain?
Apologies for a sort of vague posting, but much thanks in advance for
any tips/pointers anyone can provide.
dwh
Modules I'm using:
use Carp;
use DB_File;
use Date::Manip;
use Digest::MD5 qw(md5_hex);
use File::Basename;
use FileHandle;
use HTTP::Date;
use HTTP::Request::Common;
use IO::Select;
use IO::Socket;
use LWP::UserAgent;
use MLDBM qw(GDBM_File Storable);
use POSIX;
use Storable qw(nfreeze thaw);
use Time::HiRes
use Time::Local;
use URI::Escape;
use XML::DOM;
use utf8;
------------------------------
Date: Tue, 30 May 2000 10:38:08 -0700
From: Larry Rosler <lr@hpl.hp.com>
Subject: Re: Translate foreign characters to English
Message-Id: <MPG.139d9c56fbcba33598ab02@nntp.hpl.hp.com>
In article <3933d596.14969140@news.uni-stuttgart.de> on Tue, 30 May 2000
12:45:50 GMT, Lou Hevly <lou@visca.com> says...
> nospam.newton@gmx.li (Philip 'Yes, that's my address' Newton) wrote:
>
> >On Mon, 29 May 2000 09:12:36 GMT, lou@visca.com (Lou Hevly) wrote:
> >
> >> $ascii =~ s/æ|Æ/ae/g;
> >> $ascii =~ s/Á|Â|À|Å|Ã|Ä|á|â|à|å|ã|ä/a/g;
> >> $ascii =~ s/Ç|ç/c/g;
> >> $ascii =~ s/Ð|ð/d/g;
> >> $ascii =~ s/É|Ê|È|Ë|é|ê|è|ë/e/g;
> >> $ascii =~ s/Í|Î|Ì|Ï|í|î|ì|ï/i/g;
> >[etc.]
> >
> >Why don't you use character classes?
> >
> > $ascii =~ s/[ÁÂÀ...ä]/a/g;
> >
> >and so on.
>
> True. Hadn't thought of it. Thanks for the tip!
A better tip: use translation instead of substitution.
$ascii =~ tr/ÁÂÀ...ä/a/;
An even better tip: Read the following article:
International Sorting with Perl's sort
Sean M. Burke
The Perl Journal, Vol. 4, No. 2, p. 70 (Summer 1999)
--
(Just Another Larry) Rosler
Hewlett-Packard Laboratories
http://www.hpl.hp.com/personal/Larry_Rosler/
lr@hpl.hp.com
------------------------------
Date: Tue, 30 May 2000 16:51:55 GMT
From: jerome@activeindexing.com (Jerome O'Neil)
Subject: Re: Waxing Philosophical
Message-Id: <LkSY4.135$lg1.2848@news.uswest.net>
Elaine Ashton <elaine@chaos.wustl.edu> elucidates:
> This is no amateur and yet all of you keep coming back with the stick being
> completely unable to discern blatant troll and having an irrepressible need
> to correct them. You can't teach this dog tricks it already knows.
Lets assume for a moment that you are correct. The troll is actualy a
very astute programmer, and knows exactly the errors it is posting,
and does so with deliberate intent.
If this is the case, as opposed to the troll being mentaly unbalanced
and truly incapable of understanding (what I belive, FWIW), then it's
motive isn't ignorance, but malice, and therefore opposition should
be even more rigorus.
------------------------------
Date: 30 May 2000 15:29:35 GMT
From: Tina Mueller <tina@streetmail.com>
Subject: Re: which perl module for http perl script
Message-Id: <8h0mov$268tc$3@fu-berlin.de>
hi,
In article <3933d3ed.2795635@news.state.mn.us> you wrote:
> I am writing a perl script to access a given web page, then
> check any link found on the page to insure that it still is up --
> i.e. that if you access that link that it returns the given page
> with no errors.
hm, i think you should look for LWP at CPAN, that's for
accessing webpages. and try out HTML::Parser for
extracting links.
tina
--
http://www.tinita.de \ enter__| |__the___ _ _ ___
tina's moviedatabase \ / _` / _ \/ _ \ '_(_-< of
search & add comments \ \ _,_\ __/\ __/_| /__/ perception
------------------------------
Date: Tue, 30 May 2000 16:15:50 GMT
From: kragen@dnaco.net (Kragen Sitaker)
Subject: Re: which perl module for http perl script
Message-Id: <WORY4.126194$681.2376555@news-east.usenetserver.com>
In article <8h0mov$268tc$3@fu-berlin.de>, Tina Mueller <me@tinita.de> wrote:
>In article <3933d3ed.2795635@news.state.mn.us> you wrote:
>> I am writing a perl script to access a given web page, then
>> check any link found on the page to insure that it still is up --
>> i.e. that if you access that link that it returns the given page
>> with no errors.
>
>hm, i think you should look for LWP at CPAN, that's for
>accessing webpages. and try out HTML::Parser for
>extracting links.
Good advice, Tina.
I'll be even more specific and recommend LWP::Simple for simply
fetching web pages, and HTML::LinkExtor for extracting links.
--
<kragen@pobox.com> Kragen Sitaker <http://www.pobox.com/~kragen/>
The Internet stock bubble didn't burst on 1999-11-08. Hurrah!
<URL:http://www.pobox.com/~kragen/bubble.html>
The power didn't go out on 2000-01-01 either. :)
------------------------------
Date: 30 May 2000 17:55:52 +0100
From: nobull@mail.com
Subject: Re: which perl module for http perl script
Message-Id: <u9d7m4565j.fsf@wcl-l.bham.ac.uk>
replytousenet@usenet.com (Beetlejuice) writes:
> I am writing a perl script to access a given web page, then
> check any link found on the page to insure that it still is up --
> i.e. that if you access that link that it returns the given page
> with no errors.
>
> I have downloaded a ton of www related modules, but would
> appreciate if someone could narrow down which ones
> would be useful.
If you look on CPAN you'll see a one-liner description of each module.
Alternatively go to deja and see how the "checking broken link"
question was answered the last couple of dozen times it as asked here.
Note: some web sites redirect page-not-found errors to a "friendly"
error page that returns with "OK" status. They do this because a
certain well known HTTP client doesn't (by default) display error
pages that return with the proper status. There's no way for a robot
to identify a bad link to a page on such a site.
--
\\ ( )
. _\\__[oo
.__/ \\ /\@
. l___\\
# ll l\\
###LL LL\\
------------------------------
Date: 16 Sep 99 21:33:47 GMT (Last modified)
From: Perl-Users-Request@ruby.oce.orst.edu (Perl-Users-Digest Admin)
Subject: Digest Administrivia (Last modified: 16 Sep 99)
Message-Id: <null>
Administrivia:
The Perl-Users Digest is a retransmission of the USENET newsgroup
comp.lang.perl.misc. For subscription or unsubscription requests, send
the single line:
subscribe perl-users
or:
unsubscribe perl-users
to almanac@ruby.oce.orst.edu.
| NOTE: The mail to news gateway, and thus the ability to submit articles
| through this service to the newsgroup, has been removed. I do not have
| time to individually vet each article to make sure that someone isn't
| abusing the service, and I no longer have any desire to waste my time
| dealing with the campus admins when some fool complains to them about an
| article that has come through the gateway instead of complaining
| to the source.
To submit articles to comp.lang.perl.announce, send your article to
clpa@perl.com.
To request back copies (available for a week or so), send your request
to almanac@ruby.oce.orst.edu with the command "send perl-users x.y",
where x is the volume number and y is the issue number.
For other requests pertaining to the digest, send mail to
perl-users-request@ruby.oce.orst.edu. Do not waste your time or mine
sending perl questions to the -request address, I don't have time to
answer them even if I did know the answer.
------------------------------
End of Perl-Users Digest V9 Issue 3209
**************************************
