[21961] in Perl-Users-Digest
Perl-Users Digest, Issue: 4183 Volume: 10
daemon@ATHENA.MIT.EDU (Perl-Users Digest)
Tue Nov 26 18:10:53 2002
Date: Tue, 26 Nov 2002 15:10:15 -0800 (PST)
From: Perl-Users Digest <Perl-Users-Request@ruby.OCE.ORST.EDU>
To: Perl-Users@ruby.OCE.ORST.EDU (Perl-Users Digest)
Perl-Users Digest Tue, 26 Nov 2002 Volume: 10 Number: 4183
Today's topics:
Out Of Memory while parsing CVS rlog output (Jacob)
Re: perlcc and perl modules <ian@WINDOZEdigiserv.net>
Re: snmptrapd handler in perl? (John Ramsden)
Re: splice() is fast when shrinking and slow when growi <usenet@atrixnet.com>
Wow! Differences between "unless" and if (!) <apollock11@hotmail.com>
Re: Wow! Differences between "unless" and if (!) (Walter Roberson)
Re: Wow! Differences between "unless" and if (!) <apollock11@hotmail.com>
Re: Wow! Differences between "unless" and if (!) <uri@stemsystems.com>
Digest Administrivia (Last modified: 6 Apr 01) (Perl-Users-Digest Admin)
----------------------------------------------------------------------
Date: 26 Nov 2002 12:28:03 -0800
From: Jacob.Schroeder@latitude.com (Jacob)
Subject: Out Of Memory while parsing CVS rlog output
Message-Id: <4497f1ad.0211261228.43e4f735@posting.google.com>
I'm new to perl but I've been working on a script that will parse a
large
amount of text and while it's going through it, it will store data in
a few
different hashes I have (a one dimensional hash, and two-two
dimensional
hashes). Once I read in all the data, I then sort the hashes and
output all
of this to a set of log files.
Here's the main chunk of my code that start the text coming in...
# Build up the command string appropriately, depending on what
options
# have been set.
my $command =
($rlog_module ne "") ? "cvs -n -d $cvsdir rlog $rlog_module" : "cvs
log";
print "Executing \"$command\"\n" if $debug;
open (CVSLOG, "$command |") || die "Couldn't execute
\"$command\"";
while (<CVSLOG>)
{
if ($search_file == 1)
{
# Need to locate the name of the working file
if (/^RCS file: (.*),v$/)
{
$working_file = $1;
$working_file =~ s/Attic\///g;
$relative_working_file = "";
$directory = $working_file;
$directory =~ s/\//\\/g;
$directory =~ /.*\\srcroot(.*)\\.*$/;
$directory = $1;
print "Directory: \"$directory\"\n" if $debug;
# Check if this file is to be included or not.
if (include_file($working_file))
{
# Yep, search for more details on this file.
$search_file = 0;
if ($branch_tag eq "")
{
# Main branch to be investigated only.
$file_branch_number{$working_file} = "1";
$file_number_branch_revisions{$working_file} = 0;
}
print "Including file \"$working_file\"\n" if $debug;
}
else
{
print "Excluding file \"$working_file\"\n" if $debug;
}
}
}
else
{
# Collective the relative part for those runs that don't use
# -rlog.
if (/^Working file: (.*)$/)
{
$relative_working_file = $1;
}
# If we are collecting statistics on a branch, determine the
magic
# branch number for this file.
elsif ( (! defined $file_branch_number{$working_file}) &&
(/^\s*${branch_tag}: ([\d\.]+)\.0\.(\d+)$/) )
{
$file_branch_number{$working_file} = "${1}.${2}";
$file_number_branch_revisions{$working_file} = 0;
if ($debug)
{
print "Got branch $file_branch_number{$working_file}";
print " for file \"$working_file\"\n";
}
}
elsif (/^keyword substitution: b$/)
{
# This is a binary file, ignore it.
undef($file_branch_number{$working_file});
undef($file_number_branch_revisions{$working_file});
$search_file = 1;
print "Excluding binary file \"$working_file\"\n" if $debug;
}
elsif (/^=============================================================================$/)
{
# End of the log entry for this file, start parsing for the
# next file.
$search_file = 1;
next;
}
elsif (/^----------------------------$/)
{
# Matched the description separator. If a branch has been
# specified, but this file doesn't exist on it, skip this file.
if (($branch_tag ne "") &&
(! defined $file_branch_number{$working_file}))
{
if ($debug)
{
print "File \"$working_file\" not on branch\n";
}
$search_file = 1;
next;
}
# Read the revision line, and record the appropriate
# information.
$_ = <CVSLOG>;
if (/^revision ([\d\.]+)$/)
{
# Record the revision, and whether it is part of the tag
# of interest.
$revision = $1;
if ($revision =~
/^$file_branch_number{$working_file}\.\d+$/)
{
$file_on_branch = 1;
$file_number_branch_revisions{$working_file}++;
}
else
{
$file_on_branch = 0;
}
if ($debug)
{
print "Got branch number: $file_branch_number{$working_file} rev
$revision on branch: $file_on_branch\n";
}
}
else
{
# Problem in parsing, skip it.
print "Couldn't parse line: $_\n";
$search_file = 1;
next;
}
$_ = <CVSLOG>; # Read the "date" line.
if (/^date: (\d\d\d\d\/\d\d\/\d\d \d\d:\d\d:\d\d);.*author: (.*);.*
state: (.*);.*lines: \+(\d+) \-(\d+)$/)
{
# Note for some CVS clients, state dead is presented in
# this this way, as the following pattern.
$date = $1;
$author = $2;
$state = $3;
$lines_added = $4;
$lines_removed = $5;
$number_lines = $lines_added - $lines_removed;
$lines_modified = $lines_added + $lines_removed;
$file_version_delta{$working_file}{$revision} =
$number_lines;
$file_version_state{$working_file}{$revision} = $state;
if ($file_on_branch)
{
# This revision lives on the branch of interest.
$line_stats{$date}{$working_file} += $number_lines;
$state_stats{$date}{$working_file} = $state;
$revision_stats{$date}{$working_file} = $revision;
#User stats
$user_stats_master{$author}{$date} = $directory;
$user_stats{$author}{$date} += $lines_modified;
$user_stats_summary{$author} += $lines_modified;
}
}
elsif (/^date: (\d\d\d\d\/\d\d\/\d\d \d\d:\d\d:\d\d); .* state:
dead;$/)
{
# File has been removed.
$date = $1;
$file_version_delta{$working_file}{$revision} = 0;
$file_version_state{$working_file}{$revision} = "dead";
if ($file_on_branch)
{
$line_stats{$date}{$working_file} = 0;
$state_stats{$date}{$working_file} = "dead";
$revision_stats{$date}{$working_file} = $revision;
}
}
elsif (/^date: (\d\d\d\d\/\d\d\/\d\d \d\d:\d\d:\d\d);.*author:
(.*);.* state: Exp;$/)
{
$date = $1;
$author = $2;
# Unfortunately, cvs log doesn't indicate the number of
# lines an initial revision is created with, so find this
# out using the following cvs command.
my $lccmd = "";
if ($rlog_module ne "")
{
print "1.working_file = $working_file\n";
print "2.working_cvsdir = $working_cvsdir\n";
$working_file =~ /^${working_cvsdir}\/(.*)$/;
print "3.working_file = $working_file\n";
#TODO:FIX THIS LINE
$lccmd = "cvs -d $cvsdir co -r $revision -p \"$1\"";
}
else
{
$lccmd = "cvs update -r $revision -p \"$relative_working_file\"";
}
print "Executing $lccmd\n" if $debug;
$number_lines = `$lccmd 2>/dev/null | wc -l`;
chop $number_lines;
$number_lines =~ s/ //g;
print "$working_file 1.1 = $number_lines lines\n" if $debug;
$file_version_delta{$working_file}{$revision} =
$number_lines;
$file_version_state{$working_file}{$revision} = "Exp";
if ($file_on_branch)
{
$line_stats{$date}{$working_file} += $number_lines;
$state_stats{$date}{$working_file} = "Exp";
$revision_stats{$date}{$working_file} = $revision;
#User stats
$user_stats_master{$author}{$date} = $directory;
$user_stats{$author}{$date} += $number_lines;
$user_stats_summary{$author} += $number_lines;
}
}
else
{
print "Couldn't parse: $_";
}
if ($debug)
{
print "File \"$working_file\" rev $revision ";
print "delta $file_version_delta{$working_file}{$revision} ";
print "state $file_version_state{$working_file}{$revision}\n";
}
}
}
}
What happens is that when this is running, after maybe 30 minutes I
get an error that just says "Out Of Memory!" and that's it. It
probably has something to do with the above loop because it never gets
out of that loop before it runs out of memory (i know this by watching
my debug print statements). My CVS root directory is nearly 550 MB
with thousands of files and folders. If I run this command:
cvs -n -d \\server\cvs\srcroot -rlog . > rlog_output.txt
then rlog_output.txt is about 32.5 MB, which is large, but that may
provide some information.
Let me know if you see something obviously wrong, I'm working with the
code for CVSPlot.pl which can be found here:
http://cvsstat.sourceforge.net/ (i have made a few changes, but even
before my changes, it still crashes with the out of Memory error).
Here's the stats on the machine running this code:
OS: Win2k Advanced Server
RAM: 256MB
Processor: Pentium III ~733 MHz
I'm running the mks toolkit (so I can have access to unix commands)
I'm new so I don't really know what this could be or where to look for
help,
thanks in advance,
Jacob
------------------------------
Date: Tue, 26 Nov 2002 20:25:09 GMT
From: "Ian.H [dS]" <ian@WINDOZEdigiserv.net>
Subject: Re: perlcc and perl modules
Message-Id: <eml7uu8ti10h9n7ljlsbh4t25b87img61t@4ax.com>
Keywords: Remove WINDOZE to reply
-----BEGIN xxx SIGNED MESSAGE-----
Hash: SHA1
In a fit of excitement on Tue, 26 Nov 2002 13:09:26 GMT,
helgi@decode.is (Helgi Briem) managed to scribble:
> On Tue, 26 Nov 2002 11:34:48 GMT, "Ian.H [dS]"
> <ian@WINDOZEdigiserv.net> wrote:
>
> >> Has anyone ever managed to get that thing to work?
> >
> >I did with something (I forget now) but it didn't include any
> >modules.
> >
> >I was hoping to compile something which included the File::Find
> >module, but alas as you state... it was not a job well done. I also
> >found that Perl2Exe (for windoze) works very well in most cases,
> >except when
> >including modules, which to me, makes it somewhat useless for any
> >real usable script.. or a _lot_ of work to code "around" using a
> >module.
>
> I have written lots of programs utlising modules and
> packed them successfully using perl2exe. In fact I
> don't think I have ever written a real program that
> didn't use modules. Why should that be a problem?
Helgi,
I'm not sure if you're misinterpreted my post (maybe I've
misinterpreted this reply).
I must correct something in my OP though, IIR now, I was attempting to
compile a Perl/Tk script and it caused problems (although this may well
have been an issue with me not RTFM correctly).
Regarding "real program", this wasn't supposed to be a statement of any
depth. From the way I read your reply, it's as though I had an issue
with using modules or something, I haven't at all, thus being no
problem.
> The manual tells you *precisely* how to do it.
I guess I should RTFM again.
Regards,
Ian
-----BEGIN xxx SIGNATURE-----
Version: PGP Personal Privacy 6.5.3
iQA/AwUBPePY9Wfqtj251CDhEQL8TACdEVozS+rfy71+6zVKwV3pct39Pe4AoNAn
rz3nEsyogBLS6AJFKU7P2B7u
=EKS1
-----END PGP SIGNATURE-----
--
Ian.H [Design & Development]
digiServ Network - Web solutions
www.digiserv.net | irc.digiserv.net | forum.digiserv.net
Scripting, Web design, development & hosting.
------------------------------
Date: 26 Nov 2002 14:56:35 -0800
From: john_ramsden@sagitta-ps.com (John Ramsden)
Subject: Re: snmptrapd handler in perl?
Message-Id: <d27434e.0211261456.32bb1370@posting.google.com>
Dave Shield <D.T.Shield@csc.liv.ac.uk> wrote:
>
> John Ramsden wrote:
> >
> > * SNMP by Joe Marzot
> >
> > Sits on top of the Net-SNMP library; seems to me like the most
> > complete Perl implementation
>
> You're probably already aware of this, but it's worth emphasising
> that you should always use the version of the perl modules that are
> shipped with the source Net-SNMP (or UCD-SNMP) distribution.
>
> The internals of the Net-SNMP library do change from one distribution
> to the next, and the perl module is sufficiently closely interlinked
> that this can cause problems. The danger with using a perl module
> picked up from CPAN, is that you may end up working with a subtly
> incompatible combination of Perl code and C library.
>
> Apologies if this is old news, but I felt it was worth mentioning,
> just in case.
Thanks for the prompt reply, Dave. Uh, no I wasn't aware of that,
although I should have guessed and I'll try using the Net-SNMP
version of Perl tomorrow.
I tend to go for the latest stablish versions of the software,
and in this case I've been working with Net-SNMP v5.0.5 and
Perl v5.8.0 (and the latest SNMP modules from CPAN - I don't
have their versions to hand).
The odd thing is that the Perl modules SNMP_Session and Job Marzot's
SNMP _both_ fail to process any traps on my system (Solaris v2.8
with software versions as above), although the Net-SNMP 'snmptrapd'
utility seems to handle them fine using the same SNMP configuration
and MIBs.
Actually, SNMP core-dumps after receiving a simple Veritas 'coldStart'
trap and half way through parsing it. I've been looking at debug output
from the depths of the Net-SNMP library, but don't have time to go into
this further, and in any case I've been able to work round the problem
by setting up a snmptrapd.conf file so that for each trap 'snmptrapd'
receives it executes a default trap handler written in (guess what) Perl.
That way the handler can just read the trap details in 'cooked' ASCII
form via stdin - Not as elegant as receiving traps directly in Perl,
as planned; but it will have to do for now.
SNMP seems to be _hellishly_ complicated for what it does, which is
basically just shuffling data around using UDP. What's more, for some
baffling reason, SNMP traps and especially trap receiving seem to have
been added on as a half-hearted afterthought. (I even came across a
remark to the effect that many people involved in the early days of
developing SNMP were strongly opposed to including traps at all.
Why ever not?!)
Cheers
John Ramsden (john_ramsden AT sagitta-ps DOT com)
------------------------------
Date: Tue, 26 Nov 2002 14:04:07 -0600
From: Tommy Butler <usenet@atrixnet.com>
Subject: Re: splice() is fast when shrinking and slow when growing an array. Why?
Message-Id: <3de3d132$1_6@news.teranews.com>
Patrick Ditchen wrote:
> What exactly does splice() do with an array? I thougt, an array is a
> list of pointers to skalar-values. After a splice()-command, Perl
> should have to rebuild the whole list, no matter if shrinking or
> growing. But when I test its behaviour, splice() with shrinking size
> is done at once while splice() with growing size takes a lot of time.
> Why doesn't Perl have to copy the array in the first case?
It simply takes more computation and memory. It's faster to assign to the array
when growing it. See perldoc -f unshift
--
-Tommy Butler
see if I'm online »http://ooopps.sourceforge.net/online
Tommy Butler <tommy@atrixnet.com>
phone: (817)-468-7716
6711 Forest Park Dr
Arlington, TX
76001-8403
Download my résumé in PDF, MS Word, HTML, text
http://www.atrixnet.com/
the ooOPps Code Library
http://ooopps.sourceforge.net/pub/
------------------------------
Date: Tue, 26 Nov 2002 12:33:26 -0800
From: Arvin Portlock <apollock11@hotmail.com>
Subject: Wow! Differences between "unless" and if (!)
Message-Id: <as0lqo$2la9$1@agate.berkeley.edu>
Something very similar to this bit me today. Can anybody explain
why these two scripts behave differently?
(Using ActiveState perl 5.6.1)
## This first doesn't print anything
my $sub = &TargetSubs;
my $string = $sub->("Hello", "there!");
print "$string\n" if $string;
sub TargetSubs {
my $sub = sub {
my ($param1, $param2) = @_;
if (!$param1) {
return $param2;
}
};
return $sub;
}
## But this one prints "Hello" !!!!
my $sub = &TargetSubs;
my $string = $sub->("Hello", "there!");
print "$string\n" if $string;
sub TargetSubs {
my $sub = sub {
my ($param1, $param2) = @_;
unless ($param1) {
return $param2;
}
};
return $sub;
}
------------------------------
Date: 26 Nov 2002 21:12:05 GMT
From: roberson@ibd.nrc.ca (Walter Roberson)
Subject: Re: Wow! Differences between "unless" and if (!)
Message-Id: <as0o35$dle$1@canopus.cc.umanitoba.ca>
In article <as0lqo$2la9$1@agate.berkeley.edu>,
Arvin Portlock <apollock11@hotmail.com> wrote:
:Something very similar to this bit me today. Can anybody explain
:why these two scripts behave differently?
:(Using ActiveState perl 5.6.1)
:sub TargetSubs {
: my $sub = sub {
: my ($param1, $param2) = @_;
: if (!$param1) {
: return $param2;
: }
: };
: return $sub;
:}
Notice that if $param1 is "true" then you do not explicitly return
anything from the subroutine. When you do not return anything
explicitly, the return value is, if I recall correctly, the last
expression evaluated. In this case, the last express evaluated
is !$param1 which in your example is going to be !"Hello" which
is going to be null-false. In the variant that uses unless ($param1)
the last expression evaluated would be $param1 which in your
example is "Hello" so that's going to be returned and printed.
--
Strange but true: there are entire WWW pages devoted to listing
programs designed to obfuscate HTML.
------------------------------
Date: Tue, 26 Nov 2002 14:30:03 -0800
From: Arvin Portlock <apollock11@hotmail.com>
Subject: Re: Wow! Differences between "unless" and if (!)
Message-Id: <as0sld$2o1f$1@agate.berkeley.edu>
Aha! I have forgotten about the last expression evaluated deal.
Thanks! Before I could believe you I had to write my own little
test script to see for myself. Indeed,
print !"Hello world!\n";
Doesn't print anything.
I always knew you were supposed to have only a single return
in a subroutine and that it should be last. Now I know why.
Something like "$return = $param2 unless $param1; return $return;"
would probably work.
This also explains why the "unless" script works if I add an
} else { return 0; } to it.
Thanks for your help!
Arvin
Walter Roberson wrote:
> In article ,
> Arvin Portlock wrote:
> :Something very similar to this bit me today. Can anybody explain
> :why these two scripts behave differently?
> :(Using ActiveState perl 5.6.1)
>
> :sub TargetSubs {
> : my $sub = sub {
> : my ($param1, $param2) = @_;
> : if (!$param1) {
> : return $param2;
> : }
> : };
> : return $sub;
> :}
>
> Notice that if $param1 is "true" then you do not explicitly return
> anything from the subroutine. When you do not return anything
> explicitly, the return value is, if I recall correctly, the last
> expression evaluated. In this case, the last express evaluated
> is !$param1 which in your example is going to be !"Hello" which
> is going to be null-false. In the variant that uses unless ($param1)
> the last expression evaluated would be $param1 which in your
> example is "Hello" so that's going to be returned and printed.
> --
> Strange but true: there are entire WWW pages devoted to listing
> programs designed to obfuscate HTML.
------------------------------
Date: Tue, 26 Nov 2002 22:39:08 GMT
From: Uri Guttman <uri@stemsystems.com>
Subject: Re: Wow! Differences between "unless" and if (!)
Message-Id: <x7r8d8552s.fsf@mail.sysarch.com>
>>>>> "AP" == Arvin Portlock <apollock11@hotmail.com> writes:
please don't top post.
AP> This also explains why the "unless" script works if I add an
AP> } else { return 0; } to it.
the real lesson to learn is to use explicit returns when you return
values. and if you do an early return, then the fall though should also
be explicit.
uri
--
Uri Guttman ------ uri@stemsystems.com -------- http://www.stemsystems.com
----- Stem and Perl Development, Systems Architecture, Design and Coding ----
Search or Offer Perl Jobs ---------------------------- http://jobs.perl.org
------------------------------
Date: 6 Apr 2001 21:33:47 GMT (Last modified)
From: Perl-Users-Request@ruby.oce.orst.edu (Perl-Users-Digest Admin)
Subject: Digest Administrivia (Last modified: 6 Apr 01)
Message-Id: <null>
Administrivia:
The Perl-Users Digest is a retransmission of the USENET newsgroup
comp.lang.perl.misc. For subscription or unsubscription requests, send
the single line:
subscribe perl-users
or:
unsubscribe perl-users
to almanac@ruby.oce.orst.edu.
To submit articles to comp.lang.perl.announce, send your article to
clpa@perl.com.
To request back copies (available for a week or so), send your request
to almanac@ruby.oce.orst.edu with the command "send perl-users x.y",
where x is the volume number and y is the issue number.
For other requests pertaining to the digest, send mail to
perl-users-request@ruby.oce.orst.edu. Do not waste your time or mine
sending perl questions to the -request address, I don't have time to
answer them even if I did know the answer.
------------------------------
End of Perl-Users Digest V10 Issue 4183
***************************************
