[21822] in Perl-Users-Digest
Perl-Users Digest, Issue: 4026 Volume: 10
daemon@ATHENA.MIT.EDU (Perl-Users Digest)
Fri Oct 25 00:06:03 2002
Date: Thu, 24 Oct 2002 21:05:10 -0700 (PDT)
From: Perl-Users Digest <Perl-Users-Request@ruby.OCE.ORST.EDU>
To: Perl-Users@ruby.OCE.ORST.EDU (Perl-Users Digest)
Perl-Users Digest Thu, 24 Oct 2002 Volume: 10 Number: 4026
Today's topics:
Re: compile perl 5.8 (Dave Saunders)
Re: Difference between @foo = <`some program`> and @foo (Jay Tilton)
Re: Difference between @foo = <`some program`> and @foo (Jay Tilton)
Re: Embeded Documentation <wksmith@optonline.net>
Re: File access in CGI (Michael Puskar)
finding all local ip addresses <mat_trash@hotmail.com>
Re: Help: Installing GD module <jason@baugher.pike.il.us>
Re: Help: Installing GD module <bryan@akanta.com>
Re: Help: Installing GD module <jason@baugher.pike.il.us>
Re: Matching a null in [ -~\t]? <bwalton@rochester.rr.com>
Re: Missing WRITE_DAC event to audit in Registry.pm (Jay Tilton)
Re: PL_Sv error <rgarciasuarez@free.fr>
Re: pulling out text from between two strings <jkeen@concentric.net>
Re: pulling out text from between two strings (Tad McClellan)
question about line of code <phil.latio@f-in-stupid.co.uk>
Re: question about line of code <tony_curtis32@yahoo.com>
Re: question about line of code <phil.latio@f-in-stupid.co.uk>
Re: RegEx switcheroo <tony_curtis32@yahoo.com>
Re: regular expression <pinyaj@rpi.edu>
Slow code, improvements requested (Jan Fure)
Re: Slow code, improvements requested <uri@stemsystems.com>
Still can't get CPAN to install anything for me. <kerry@shetline.com>
Re: Still can't get CPAN to install anything for me. <kerry@shetline.com>
Digest Administrivia (Last modified: 6 Apr 01) (Perl-Users-Digest Admin)
----------------------------------------------------------------------
Date: 24 Oct 2002 17:59:01 -0700
From: drs8h@virginia.edu (Dave Saunders)
Subject: Re: compile perl 5.8
Message-Id: <30d24638.0210241659.40f170a6@posting.google.com>
Bob et al,
Was there ever a resolution to this? I'm experiencing a similar problem:
Solaris 2.8
gcc 3.2
gdbm 1.8.3
perl 5.6.1 & perl 5.8.0
gcc 3.2 successfully compiles gdbm and is then utilized during the configure,
which fails on libgdbm with a similar message: "No such file or directory".
Thanks for your help,
Dave Saunders
"Bob Morse" <Robert_Morse@brown.edu> wrote in message news:<an27bn$k63$1@saturn.services.brown.edu>...
> Hi,
>
> I am finally getting around to compiling perl 5.8. I took
> all of the defaults just to see if it would compile. I am running
> this on a solaris 2.8 machine. I get this error:
>
>
> I used the command:
>
> gcc -B/usr/ccs/bin/ -o
> try -O -fno-strict-aliasing -I/usr/local/include -I/opt/local/include -L/usr
> /local/lib -L/opt/local/lib try.c -lsfio -lsocket -lnsl -ldb -ldl -lm -lc
> ./try
>
> and I got the following output:
>
> ld.so.1: ./try: fatal: libdb.so: open failed: No such file or directory
> Killed
> The program compiled OK, but exited with status 137.
> (The supplied flags or libraries might be incorrect.)
>
> You have a problem. Shall I abort Configure [y]
>
>
>
> I believe the libraries are correct, even though the program is saying
> they might be wrong. Anybody see this error before? Any ideas?
>
> Thanks.
>
> -----
> Robert Morse
> Senior Systems Programmer
> Brown University
> 401 863-7426
------------------------------
Date: Fri, 25 Oct 2002 00:56:34 GMT
From: tiltonj@erols.com (Jay Tilton)
Subject: Re: Difference between @foo = <`some program`> and @foo = `some program` ??
Message-Id: <3db895e1.549127346@news.erols.com>
jandirb@hotmail.com (Tipu) wrote:
: Subject: Difference between @foo = <`some program`> and @foo = `some program` ??
: Anyone know how the two are different??
One feeds the return of `some program` into glob(),
H>perl -MO=Deparse -e "@foo = <`some program`>"
@foo = CORE::GLOBAL::glob('`some program`', 0);
-e syntax OK
The other does not,
H>perl -MO=Deparse -e "@foo = `some program`"
@foo = `some program`;
-e syntax OK
Big difference if the return from contains things meaningful to
glob(), no difference if it doesn't. Code using <`some program`> has
probably been dodging bullets without you even knowing.
: I used to use the former but
: for certain situations only the latter works.
Unless there are certain situations where only the former produces the
desired result, stick with the latter.
------------------------------
Date: Fri, 25 Oct 2002 01:05:55 GMT
From: tiltonj@erols.com (Jay Tilton)
Subject: Re: Difference between @foo = <`some program`> and @foo = `some program` ??
Message-Id: <3db897f6.549660643@news.erols.com>
tiltonj@erols.com (Jay Tilton) wrote:
: One feeds the return of `some program` into glob(),
:
: H>perl -MO=Deparse -e "@foo = <`some program`>"
: @foo = CORE::GLOBAL::glob('`some program`', 0);
^ ^
^ ^
Er...rather, it globs the string '`some program`' without ever doing
the backticks thing.
: Big difference if the return from contains things meaningful to
: glob(), no difference if it doesn't.
Big difference, period.
Sorry about that.
------------------------------
Date: Fri, 25 Oct 2002 03:14:35 GMT
From: "Bill Smith" <wksmith@optonline.net>
Subject: Re: Embeded Documentation
Message-Id: <vG2u9.7889$D63.6898@news4.srv.hcvlny.cv.net>
"Morrison Davis" <morry.davis@sun.com> wrote in message
news:3DB84C5C.DA949BF5@sun.com...
> Where can I find documentation or How-To so I can run perldoc
> <scriptname>
> and get the documentation on it. Need to know the syntax to put in the
> script for this to work.
I assume that you are asking how to put your own documentation in a script
so perldoc can display it.
For the specification of POD, see perldoc perlpod.
For the specification of the contents of a manpage,
see the notes section of perldoc pod2man
For examples, refer to the source of nearly any module.
Checkout the module Usage.pm to display subsets of your POD when usage
errors are detected.
Note: Paragraphs in POD are terminated by a blank line. This implies a lot
more blank lines than I first realized. Pay attention to the use of blank
lines in examples. They are probably necessary, and certainly do not hurt.
Bill
------------------------------
Date: 24 Oct 2002 15:24:43 -0700
From: puskarm@eighth-avenue.com (Michael Puskar)
Subject: Re: File access in CGI
Message-Id: <3e7726f4.0210241424.58094039@posting.google.com>
> My problem is that everything seems to be fine, but the script just won't open the file.
What does the web server error log say?
------------------------------
Date: Fri, 25 Oct 2002 01:29:09 +0200
From: Mathieu Jondet <mat_trash@hotmail.com>
Subject: finding all local ip addresses
Message-Id: <3DB88245.12446D07@hotmail.com>
hi all,
i'd like to know how i can find all the ip addresses currently
available on a server. This means the ip aliases available on all the
ethernet cards.
Is it possible without parsing the output of the ifconfig command ? I
also would like to use it under mod_perl.
Does anyone have a clue ?
Thanks in advance.
Mathieu
------------------------------
Date: Fri, 25 Oct 2002 00:20:31 GMT
From: Jason Baugher <jason@baugher.pike.il.us>
Subject: Re: Help: Installing GD module
Message-Id: <Xns92B1C4C1EF3FBjasonbaugherpikeilus@209.242.76.10>
Bryan <bryan@akanta.com> wrote in comp.lang.perl.misc:
> I just installed GD for about the 10th time. Make sure you read the
> documents -very- carefully in the GD source.
>
> Here's what you need to do:
> 1. Get the latest libgd from boutrell.com.
> 2. From the GD source dir, copy patch_gd.pl to an -untouched- libgd
> source dir.
> 3. in libgd source run ./configure --enable-jpeg --enable-freetype
> --enable-xpm
> 4. run make, make install
> 5. Now go to GD dir and GD should compile properly and install
>
> Worked for me. Give it a shot.
>
> B
>
>
>
>
I ran into the same problem the OP did.
I got gd 2.0.4 from boutrell.com (latest stable version. Compiled and
installed. Please note, this version should not need patch_gd.pl, and
does not come with it in the source distribution. And the documentation
for this version does not mention it at all.
I installed GD-2.02 using CPAN, and got the same errors as the OP. I
found the problem, and I have emailed this to the maintainer of GD
(Lincoln Stein) as well.
What I found was that in 6 places in GD.c (created from GD.xs), there is
a structure of a type defined in /usr/include/gd_io.h that has an element
named "free". However, the element in the actual gd_io.h file is named
"gd_free". I have no idea when this would have changed, or why, but when
I changed the 6 occurrances of this, I was able to compile.
Make sure you make the changes in GD.xs, not GD.c.
--
Jason Baugher
Virtual Adept Professional Consulting Services
1406 Adams Street, Quincy, IL 62301 - (217) 221-5406
jason@baugher.pike.il.us - http://baugher.pike.il.us/virtualadept
------------------------------
Date: Fri, 25 Oct 2002 00:47:56 GMT
From: Bryan <bryan@akanta.com>
Subject: Re: Help: Installing GD module
Message-Id: <3DB894B9.5070902@akanta.com>
>
> I ran into the same problem the OP did.
>
> I got gd 2.0.4 from boutrell.com (latest stable version. Compiled and
> installed. Please note, this version should not need patch_gd.pl, and
> does not come with it in the source distribution. And the documentation
> for this version does not mention it at all.
patch_gd.pl comes with the GD (perl) package, not libgd which would
explain why there is no documentation for it in the libgd package.
>
> I installed GD-2.02 using CPAN, and got the same errors as the OP. I
> found the problem, and I have emailed this to the maintainer of GD
> (Lincoln Stein) as well.
>
> What I found was that in 6 places in GD.c (created from GD.xs), there is
> a structure of a type defined in /usr/include/gd_io.h that has an element
> named "free". However, the element in the actual gd_io.h file is named
> "gd_free". I have no idea when this would have changed, or why, but when
> I changed the 6 occurrances of this, I was able to compile.
Was this with or without freetype(truetype) support? Sounds like libgd
was not compiled with it, and the GD package didn't like that.
> Make sure you make the changes in GD.xs, not GD.c.
>
For what its worth, I have NEVER gotten GD to just install via CPAN. It
has always required some tweaking, although I have never had to do the
tweak you mentioned. But I will note it for future reference since I
have to install this stuff all the time and I never know what will happen.
B
------------------------------
Date: Fri, 25 Oct 2002 02:09:29 GMT
From: Jason Baugher <jason@baugher.pike.il.us>
Subject: Re: Help: Installing GD module
Message-Id: <Xns92B1D73AAC180jasonbaugherpikeilus@209.242.76.10>
Bryan <bryan@akanta.com> wrote in comp.lang.perl.misc:
>
>>
>> I ran into the same problem the OP did.
>>
>> I got gd 2.0.4 from boutrell.com (latest stable version. Compiled
>> and installed. Please note, this version should not need
>> patch_gd.pl, and does not come with it in the source distribution.
>> And the documentation for this version does not mention it at all.
>
> patch_gd.pl comes with the GD (perl) package, not libgd which would
> explain why there is no documentation for it in the libgd package.
>
Yep, you're right on that point. I checked through the patch though, and
it wouldn't have resolved this particular problem.
Also, the README that comes with GD-2.02 from CPAN seems to indicate that
the patch_gd.pl is optional, and is for libgd 2.0.1. In fact, it states
that it may not even work with later versions of libgd.
>>
>> I installed GD-2.02 using CPAN, and got the same errors as the OP. I
>> found the problem, and I have emailed this to the maintainer of GD
>> (Lincoln Stein) as well.
>>
>> What I found was that in 6 places in GD.c (created from GD.xs), there
>> is a structure of a type defined in /usr/include/gd_io.h that has an
>> element named "free". However, the element in the actual gd_io.h
>> file is named "gd_free". I have no idea when this would have
>> changed, or why, but when I changed the 6 occurrances of this, I was
>> able to compile.
>
> Was this with or without freetype(truetype) support? Sounds like
> libgd was not compiled with it, and the GD package didn't like that.
My Makefile for libgd, generated by configure, shows that it is including
freetype in both the libs and includes:
CFLAGS=-g -DHAVE_LIBPNG -DHAVE_LIBZ -DHAVE_LIBJPEG -DHAVE_LIBFREETYPE -
DHAVE_LIBXPM -DHAVE_LIBX11
LIBS=-lgd -lpng -lz -ljpeg -lfreetype -lXpm -lX11 -lm
INCLUDEDIRS=-I. -I/usr/include/freetype2 -I/usr/local/include/freetype2 -
I/usr/include/X11 -I/usr/X11R6/include/X11 -I/usr/local/include
>
>> Make sure you make the changes in GD.xs, not GD.c.
>>
>
> For what its worth, I have NEVER gotten GD to just install via CPAN.
> It has always required some tweaking, although I have never had to do
> the tweak you mentioned. But I will note it for future reference
> since I have to install this stuff all the time and I never know what
> will happen.
>
Also, I don't see how this problem could be affected by anything other
than a basic mismatch between GD and libgd. Since the gd_io.h file is
direct from the libgd package, and not a compiled object, there isn't a
chance it was compiled wrong. And the GD.xs file from the GD perl module
is as-is from the GD package, so it isn't dependent on anything either.
I'm betting the libgd people changed something and the GD people haven't
reflected that change yet. Perhaps if I could have gotten an earlier
version of libgd, this would not have been a problem.
--
Jason Baugher
Virtual Adept Professional Consulting Services
1406 Adams Street, Quincy, IL 62301 - (217) 221-5406
jason@baugher.pike.il.us - http://baugher.pike.il.us/virtualadept
------------------------------
Date: Fri, 25 Oct 2002 04:01:32 GMT
From: Bob Walton <bwalton@rochester.rr.com>
Subject: Re: Matching a null in [ -~\t]?
Message-Id: <3DB8C0FA.8040802@rochester.rr.com>
Tim Hogard wrote:
> I've got a bit of code that should work much like the unix strings(1)
> command but only prints strings that have a null or newline at the end.
> Or at least thats the theory.
>
> The problem is the regex matches ' [NULL]' where NULL is ascii character
> code 0. That 's "20 5B 00 5D" in the hexeditor.
> Can someone please explain why the null is ending up in $1?
> perl -v says: This is perl, v5.6.0 built for i386-linux.
>
> #!/usr/bin/perl
> #works like strings -t x -n $cnt file file...
>
> $addr=0;
> $cnt=4;
> while(<>) {
> $x=length($_);
> while($_=~ /([ -~\t]{$cnt,})[\0\n]/g) {
> printf("%7lx $1\n",$addr+pos($_)-length($1)-1);
> }
> $addr+=$x;
> } continue {
> $addr=0 if eof;
> }
>
> -tim
...
Hmmmm...according to my tests (AS Perl build 633, Windoze 98SE), the
null character is not included in $1, nor is the sequence " \0" matched
by the regex. What makes you think it is? Here is a complete
standalone test program (following after your program) showing that:
$cnt=4;
$_="aaaaa\0bbbbb\0cccc\n \0dddd\0";
$x=length($_);
print "length=$x\n";
while($_=~ /([ -~\t]{$cnt,})[\0\n]/g) {
$keep=$1;
print "match occurred...\$1 is ".
length($keep).
" long, shown below...\n";
for $i(split //,$keep){
print "ord=".ord($i)."\n";
}
}
Result:
D:\junk>perl junk218.pl
length=24
match occurred...$1 is 5 long, shown below...
ord=97
ord=97
ord=97
ord=97
ord=97
match occurred...$1 is 5 long, shown below...
ord=98
ord=98
ord=98
ord=98
ord=98
match occurred...$1 is 4 long, shown below...
ord=99
ord=99
ord=99
ord=99
match occurred...$1 is 4 long, shown below...
ord=100
ord=100
ord=100
ord=100
D:\junk>
--
Bob Walton
------------------------------
Date: Fri, 25 Oct 2002 00:52:14 GMT
From: tiltonj@erols.com (Jay Tilton)
Subject: Re: Missing WRITE_DAC event to audit in Registry.pm
Message-Id: <3db89080.547750497@news.erols.com>
a246456@fmr.com wrote:
: I am trying to audit registry Events for "Write DAC" using
: Registry.pm.
The name of the package/module is more useful than the filename.
There could be any number of files named Registry.pm, each with a
different purpose.
------------------------------
Date: 24 Oct 2002 22:06:07 GMT
From: Rafael Garcia-Suarez <rgarciasuarez@free.fr>
Subject: Re: PL_Sv error
Message-Id: <slrnargrt8.r0.rgarciasuarez@rafael.example.com>
Gregg Palmer wrote in comp.lang.perl.misc :
> Hello. I am trying to run an app written in perl and am getting the
> following
> error:[/usr/sbin]> collgui
>
> Can't load '/usr/opt/col/lib/site_perl/5.6.0/auto/Tk/Event/Event.so'
> for module Tk::Event: dlopen:
> /usr/opt/col/lib/site_perl/5.6.0/auto/Tk/Event/Event.so: symbol
^^^^^
> "PL_Sv" unresolved at
> /usr/local/lib/perl5/5.005/alpha-dec_osf-thread/DynaLoader.pm line
^^^^^
> 168.
You're trying to run a Tk application, using the Tk modules compiled
for perl 5.6.0, which a perl 5.005. This is not, as far as I know,
supported. Use perl 5.6.0 instead. If you don't have perl 5.6.0
on your machine, you should consider cleaning it up and reinstalling
perl.
------------------------------
Date: 24 Oct 2002 22:15:39 GMT
From: "James E Keenan" <jkeen@concentric.net>
Subject: Re: pulling out text from between two strings
Message-Id: <ap9reb$kob@dispatch.concentric.net>
"shambolic" <johnston.jay@mtvne.com> wrote in message
news:f333df78.0210241343.3e83c23e@posting.google.com...
> Hi,
>
> i have a whois webpage in scalar $doc, and want to pull out the text
> between the strings "netname" and RIPE.
>
> i know this is fairly standard, but having pondered the FAQ at
>
http://theoryx5.uwinnipeg.ca/CPAN/perl/pod/perlfaq6/How_can_I_pull_out_lines
_between_two_pat.html
> i am still none the wiser.
>
> (i'll confess to being fairly green, but this particular FAQ is a
> little light on explanation).
>
Agreed. Here are two references that may help:
http://groups.yahoo.com/group/perlsemny/message/174
http://history.perl.org/oneliners/filters/markers.html
Let us know if they're useful.
------------------------------
Date: Thu, 24 Oct 2002 22:12:56 -0500
From: tadmc@augustmail.com (Tad McClellan)
Subject: Re: pulling out text from between two strings
Message-Id: <slrnarhdlo.22f.tadmc@magna.augustmail.com>
shambolic <johnston.jay@mtvne.com> wrote:
>
> i have a whois webpage in scalar $doc, and want to pull out the text
> between the strings "netname" and RIPE.
if ( $doc =~ /netname(.*?)RIPE/s ) {
print "$1\n";
}
> i know this is fairly standard, but having pondered the FAQ at
It is also on your hard disk already.
> http://theoryx5.uwinnipeg.ca/CPAN/perl/pod/perlfaq6/How_can_I_pull_out_lines_between_two_pat.html
> i am still none the wiser.
The FAQ is about text between two lines.
Your subject is about text between two strings.
Which is it, lines or strings?
--
Tad McClellan SGML consulting
tadmc@augustmail.com Perl programming
Fort Worth, Texas
------------------------------
Date: Fri, 25 Oct 2002 01:28:32 GMT
From: "Phil Latio" <phil.latio@f-in-stupid.co.uk>
Subject: question about line of code
Message-Id: <471u9.1428910$P95.141987@post-03.news.easynews.com>
The below piece of code already works (it's from a news publishing script)
but I don't understand what the 5th line does: $yourlength--;
I come from a Java background and it's already declared but wondering
what -- means.
my $yourlength = 100;
my $SText = $Text;
if (length($SText) > $yourlength)
{
$yourlength--;
$SText =~ s/<br>/ /g;
if (length($SText) > $yourlength)
{
$SText = substr($SText,0,index($SText,' ',$yourlength));
$SText .= qq~...~;
}
}
$newshtml .= $SText;
------------------------------
Date: Thu, 24 Oct 2002 20:40:36 -0500
From: Tony Curtis <tony_curtis32@yahoo.com>
Subject: Re: question about line of code
Message-Id: <87hefbthwr.fsf@limey.hpcc.uh.edu>
>> On Fri, 25 Oct 2002 01:28:32 GMT,
>> "Phil Latio" <phil.latio@f-in-stupid.co.uk> said:
> The below piece of code already works (it's from a news
> publishing script) but I don't understand what the 5th
> line does: $yourlength--;
perldoc perlop:
Auto-increment and Auto-decrement
"++" and "--" work as in C. That is, if placed before a
variable, they increment or decrement the variable before
returning the value, and if placed after, increment or
decrement the variable after returning the value.
hth
t
------------------------------
Date: Fri, 25 Oct 2002 01:57:50 GMT
From: "Phil Latio" <phil.latio@f-in-stupid.co.uk>
Subject: Re: question about line of code
Message-Id: <yy1u9.1429939$w17.143181@post-02.news.easynews.com>
"Tony Curtis" <tony_curtis32@yahoo.com> wrote in message
news:87hefbthwr.fsf@limey.hpcc.uh.edu...
> >> On Fri, 25 Oct 2002 01:28:32 GMT,
> >> "Phil Latio" <phil.latio@f-in-stupid.co.uk> said:
>
> > The below piece of code already works (it's from a news
> > publishing script) but I don't understand what the 5th
> > line does: $yourlength--;
>
> perldoc perlop:
>
> Auto-increment and Auto-decrement
>
> "++" and "--" work as in C. That is, if placed before a
> variable, they increment or decrement the variable before
> returning the value, and if placed after, increment or
> decrement the variable after returning the value.
>
> hth
> t
Thanks very much. A bit thick of me really as same in Java.
------------------------------
Date: Thu, 24 Oct 2002 17:05:04 -0500
From: Tony Curtis <tony_curtis32@yahoo.com>
Subject: Re: RegEx switcheroo
Message-Id: <87znt3sdbj.fsf@limey.hpcc.uh.edu>
>> On Thu, 24 Oct 2002 21:12:55 GMT,
>> "TBN" <ihave@noemail.com> said:
> I would be grateful for the formula for taking a string
> such as: Bob.Smith
> and turning it into
> Smith, Bob
my $name = 'Bob.Smith';
my $reverso = join ', ', reverse split /\./, $name;
------------------------------
Date: Thu, 24 Oct 2002 18:46:00 -0400
From: Jeff 'japhy' Pinyan <pinyaj@rpi.edu>
To: "David K. Wall" <usenet@dwall.fastmail.fm>
Subject: Re: regular expression
Message-Id: <Pine.SGI.3.96.1021024184522.58641A-100000@vcmr-64.server.rpi.edu>
[posted & mailed]
On Thu, 24 Oct 2002, David K. Wall wrote:
>> It turns out that hybrid1 is the fastest solution (given my data set),
>> just because it's the most explicit.
>
>Are you sure you don't have a typo? When I ran your program hybrid2 was
>fastest. That's certainly not what I would have expected. I tried different
>variations of data, too, some that had lots of matches, and some that had no
>matches. Same result: /d.*?d.*?d.*?d/ was always the fastest.
>
>Anyway, maybe I need to reread Friedl's book. :-)
hybrid1 was the fastest on my machine (running 5.6.1).
--
Jeff "japhy" Pinyan RPI Acacia Brother #734 2002 Acacia Senior Dean
"And I vos head of Gestapo for ten | Michael Palin (as Heinrich Bimmler)
years. Ah! Five years! Nein! No! | in: The North Minehead Bye-Election
Oh. Was NOT head of Gestapo AT ALL!" | (Monty Python's Flying Circus)
------------------------------
Date: 24 Oct 2002 18:33:14 -0700
From: jan_may2002_fure@attbi.com (Jan Fure)
Subject: Slow code, improvements requested
Message-Id: <e47a84bf.0210241733.2a3e81d3@posting.google.com>
Hi;
I just wrote a script which:
1. Reads in a text file.
2. Sorts it. (The heading is stripped prior to sort)
3. Goes through each line, and captures the contents of the second
column.
4. Stores the line in a temporary array (@tempdata).
5. Every time the value of second column changes, the temporary array
is appended to a textfile (with the heading), unless the line number
exceeds a set limit, in which case a new file is created.
6. Steps 1-5 are repeated on all files with .txt extension in the
directory of the script.
Effectively, I break the file into smalled volumes which can be
handled by a software which is limited to 32000 lines file size.
The problem is that the script runs in no time on 1MB files, but it is
really slow on 5MB files.
I appreciate any suggestions about how to make the code more
efficient!
I suspect my sort operation takes most of the computer resources.
Jan
PS, the OS is WinNT4.0, ActiveState perl 5.6
With slow I mean 15-30 minutes for a 5MB file, whereas a 1MB file ran
in 10 seconds. The computer memory seems to not be exceeded, perl
process has grown from 30 to 40MB over the last hour: 320MB total
memory utilized, physical RAM of 384MB. Perl CPU utilization near
100%.
Code pasted below:
my $linelimit=31998;
opendir(DIR, ".");
my @tfiles = sort(grep(/.txt/, readdir(DIR)));
closedir(DIR);
foreach (<@tfiles>) {
print "$_";
print "\n";
}
foreach (<@tfiles>) {
my $INPUT_FILE = $_;
($name, $extent) = /(\w+).(\w+)/;
open INPUT_FILE, "$INPUT_FILE" or die "can't open $INPUT_FILE";
@array = <INPUT_FILE>;
my $heading=shift(@array);
print "Heading is $heading!\n";
my @sorted = sort @array;
print $name, $extent, "\n";
my $n1=0; my $n2=0;my $n3=0;my $newone=0;my $old=0;
my ($lot, @other);
my $firstrow=shift(@sorted);
($lot,$newone,@other)=split /\t/, $firstrow;
unshift @sorted, $firstrow;
open(CURRENT,">$name$n3.tx");
print CURRENT "$heading";
close CURRENT;
@tempdata = [];
foreach (@sorted) {
$n1++;
$old=$newone;
($lot,$newone,@other)=split /\t/;
if ($newone ne $old){
if ($n1<$linelimit){
shift @tempdata;
open(CURRENT,">>$name$n3.tx");
print CURRENT @tempdata;
close CURRENT;
print "$n2\n";
}
else {
$n3++;
shift @tempdata;
open (CURRENT,">$name$n3.tx");
print CURRENT "$heading";
print CURRENT @tempdata;
close CURRENT;
$n1=$n1 - $linelimit;
}
$n2++;
@tempdata = [];
}
push @tempdata, $_
}
if ($n1>$linelimit){
$n3++;
shift @tempdata;
open(CURRENT,">$name$n3.tx");
print CURRENT "$heading";
print CURRENT @tempdata;
close CURRENT;
}
else {
shift @tempdata;
open(CURRENT,">>$name$n3.tx");
print CURRENT @tempdata;
close CURRENT;
}
}
------------------------------
Date: Fri, 25 Oct 2002 02:57:07 GMT
From: Uri Guttman <uri@stemsystems.com>
Subject: Re: Slow code, improvements requested
Message-Id: <x7vg3r1b0c.fsf@mail.sysarch.com>
>>>>> "JF" == Jan Fure <jan_may2002_fure@attbi.com> writes:
regardless of speed, your code needs lots of improvements. found many
bugs and poor perl coding technique so this will be a general critique.
JF> my $linelimit=31998;
JF> opendir(DIR, ".");
always check the result of open/opendir
JF> my @tfiles = sort(grep(/.txt/, readdir(DIR)));
JF> closedir(DIR);
JF> foreach (<@tfiles>) {
why the <>? another recent thread has that oddity. you already have the
list of file so why glob them again?
JF> print "$_";
JF> print "\n";
print "$_\n" ;
JF> }
JF> foreach (<@tfiles>) {
why do the same loop again? just print the file in here and save a loop
JF> my $INPUT_FILE = $_;
better to use that variable in the foreach line. and lower case it. file
handles are upper cased, not the file name variables.
foreach my $INPUT_FILE (@tfiles) {
JF> ($name, $extent) = /(\w+).(\w+)/;
that . matches any char. it needs escaping but it happens to work there
because (real) . isn't a \w char.
JF> open INPUT_FILE, "$INPUT_FILE" or die "can't open $INPUT_FILE";
JF> @array = <INPUT_FILE>;
JF> my $heading=shift(@array);
JF> print "Heading is $heading!\n";
JF> my @sorted = sort @array;
JF> print $name, $extent, "\n";
JF> my $n1=0; my $n2=0;my $n3=0;my $newone=0;my $old=0;
blech. use one line for each of those. multiple statements on one line
is frowned upon.
JF> my ($lot, @other);
why is are those declared here? you can just declare them at first use below.
JF> my $firstrow=shift(@sorted);
JF> ($lot,$newone,@other)=split /\t/, $firstrow;
JF> unshift @sorted, $firstrow;
here is a bad line. why shift and then unshift the line back? just
access it with $sorted[0] and process it.
JF> open(CURRENT,">$name$n3.tx");
always check for errors with open
JF> print CURRENT "$heading";
JF> close CURRENT;
this open/close thing is very clumsy. i am not even going to attempt to
clear up the logic of how you split the file into smaller files. the
spec is unclear, the file format is not shown (with sample data) and the
code is confusing.
JF> @tempdata = [];
JF> foreach (@sorted) {
JF> $n1++;
that is a very bad variable name. same with its relatives. pick names
with meaning to others, not just to you.
JF> $old=$newone;
JF> ($lot,$newone,@other)=split /\t/;
JF> if ($newone ne $old){
JF> if ($n1<$linelimit){
JF> shift @tempdata;
JF> open(CURRENT,">>$name$n3.tx");
JF> print CURRENT @tempdata;
JF> close CURRENT;
JF> print "$n2\n";
JF> }
JF> else {
JF> $n3++;
JF> shift @tempdata;
JF> open (CURRENT,">$name$n3.tx");
JF> print CURRENT "$heading";
useless use of quotes.
JF> print CURRENT @tempdata;
JF> close CURRENT;
JF> $n1=$n1 - $linelimit;
JF> }
JF> $n2++;
JF> @tempdata = [];
gack! that does not do what you think it does. it assigns a ref to an
anon array, it doesn't empty the array.
JF> }
JF> push @tempdata, $_
JF> }
JF> if ($n1>$linelimit){
JF> $n3++;
JF> shift @tempdata;
JF> open(CURRENT,">$name$n3.tx");
JF> print CURRENT "$heading";
JF> print CURRENT @tempdata;
JF> close CURRENT;
JF> }
JF> else {
ok, if the chosen file is still the same, why reopen it? i don't know
the logic but i am sure you would only need to close and open a new file
if the output needs changing. all those open/close pairs are slow.
JF> shift @tempdata;
JF> open(CURRENT,">>$name$n3.tx");
JF> print CURRENT @tempdata;
JF> close CURRENT;
JF> }
JF> }
you need to describe your logic needs much better so you can clean up
the open/close repetition. as for the sorting of the files, 5Mb is not
that much and shouldn't slow down the program by that much. without real
data it is hard to see where it is slow but i think it is in the logic.
uri
--
Uri Guttman ------ uri@stemsystems.com -------- http://www.stemsystems.com
----- Stem and Perl Development, Systems Architecture, Design and Coding ----
Search or Offer Perl Jobs ---------------------------- http://jobs.perl.org
------------------------------
Date: Fri, 25 Oct 2002 02:23:38 GMT
From: Kerry Shetline <kerry@shetline.com>
Subject: Still can't get CPAN to install anything for me.
Message-Id: <KW1u9.89062$zE6.306158@rwcrnsc51.ops.asp.att.net>
I'm trying to install Perl modules using my web hosting service, an
Apache server running on Linux, for which I do not have root access.
I got some help yesterday (thanks!) figuring out how to manually set up
a MyConfig.pm file, since CPAN refused to even start up before that was
done -- preventing me from using CPAN itself to create my original
configuration!
But now I can't get past this:
------------------------------------------------------------
cpan> install Mail::Sendmail
Going to read ~/.cpan/sources/authors/01mailrc.txt.gz
/home/u2/shetline2/.cpan/sources/authors/01mailrc.txt.gz: No such file
or directory
Going to read ~/.cpan/sources/modules/02packages.details.txt.gz
/home/u2/shetline2/.cpan/sources/modules/02packages.details.txt.gz: No
such file or directory
Warning: Your ~/.cpan/sources/modules/02packages.details.txt.gz does not
contain a Line-Count header.
Please check the validity of the index file by comparing it to more
than one CPAN mirror. I'll continue but problems seem likely to
happen.
Warning: Your ~/.cpan/sources/modules/02packages.details.txt.gz does not
contain a Last-Updated header.
Please check the validity of the index file by comparing it to more
than one CPAN mirror. I'll continue but problems seem likely to
happen.
Going to read ~/.cpan/sources/modules/03modlist.data.gz
/home/u2/shetline2/.cpan/sources/modules/03modlist.data.gz: No such file
or directory
Can't locate object method "data" via package "CPAN::Modulelist"
(perhaps you forgot to load "CPAN::Modulelist"?) at (eval 33) line 1.
CPAN::Index::rd_modlist('CPAN::Index',
'~/.cpan/sources/modules/03modlist.data.gz') called at
/usr/local/lib/perl5/5.6.1/CPAN.pm line 3002
CPAN::Index::reload('CPAN::Index') called at
/usr/local/lib/perl5/5.6.1/CPAN.pm line 620
CPAN::exists('CPAN=HASH(0x8462b30)', 'CPAN::Module',
'Mail::Sendmail') called at /usr/local/lib/perl5/5.6.1/CPAN.pm line 1735
CPAN::Shell::expandany('CPAN::Shell', 'Mail::Sendmail') called
at /usr/local/lib/perl5/5.6.1/CPAN.pm line 1969
CPAN::Shell::rematein('CPAN::Shell', 'install',
'Mail::Sendmail') called at /usr/local/lib/perl5/5.6.1/CPAN.pm line 2056
CPAN::Shell::install('CPAN::Shell', 'Mail::Sendmail') called at
/usr/local/lib/perl5/5.6.1/CPAN.pm line 183
eval {...} called at /usr/local/lib/perl5/5.6.1/CPAN.pm line 183
CPAN::shell() called at -e line 1
------------------------------------------------------------
I have no idea what's wrong here. Nothing I've read in any CPAN
documentation leads me to believe I should have had to set up anything
more that what I've already set up, yet now it looks like there's a
whole lot of stuff CPAN expects to find in ~.cpan/sources/ that just
isn't there :(
Help!
------------------------------
Date: Fri, 25 Oct 2002 03:11:46 GMT
From: Kerry Shetline <kerry@shetline.com>
Subject: Re: Still can't get CPAN to install anything for me.
Message-Id: <SD2u9.66642$Hj7.34081@rwcrnsc53>
Mystery solved!
I noticed that my home directory contained a new directory... named "~"!
Somehow, someway, but only part of the time, the ~ character in
"~/.cpan" got interpreted as a literal name for a directory, rather than
the path to my home directory.
Once I threw out this unwanted "~" directory and reconfigured
MyConfig.pm using "/home/u2/myusername/whatever" instead of
"~/whatever", everything worked fine.
-Kerry
Kerry Shetline wrote:
> I'm trying to install Perl modules using my web hosting service, an
> Apache server running on Linux, for which I do not have root access.
>
> I got some help yesterday (thanks!) figuring out how to manually set up
> a MyConfig.pm file, since CPAN refused to even start up before that was
> done -- preventing me from using CPAN itself to create my original
> configuration!
>
> But now I can't get past this:
>
> ------------------------------------------------------------
>
> cpan> install Mail::Sendmail
> Going to read ~/.cpan/sources/authors/01mailrc.txt.gz
> /home/u2/shetline2/.cpan/sources/authors/01mailrc.txt.gz: No such file
> or directory
> Going to read ~/.cpan/sources/modules/02packages.details.txt.gz
> /home/u2/shetline2/.cpan/sources/modules/02packages.details.txt.gz: No
> such file or directory
> Warning: Your ~/.cpan/sources/modules/02packages.details.txt.gz does not
> contain a Line-Count header.
...etc, etc...
------------------------------
Date: 6 Apr 2001 21:33:47 GMT (Last modified)
From: Perl-Users-Request@ruby.oce.orst.edu (Perl-Users-Digest Admin)
Subject: Digest Administrivia (Last modified: 6 Apr 01)
Message-Id: <null>
Administrivia:
The Perl-Users Digest is a retransmission of the USENET newsgroup
comp.lang.perl.misc. For subscription or unsubscription requests, send
the single line:
subscribe perl-users
or:
unsubscribe perl-users
to almanac@ruby.oce.orst.edu.
To submit articles to comp.lang.perl.announce, send your article to
clpa@perl.com.
To request back copies (available for a week or so), send your request
to almanac@ruby.oce.orst.edu with the command "send perl-users x.y",
where x is the volume number and y is the issue number.
For other requests pertaining to the digest, send mail to
perl-users-request@ruby.oce.orst.edu. Do not waste your time or mine
sending perl questions to the -request address, I don't have time to
answer them even if I did know the answer.
------------------------------
End of Perl-Users Digest V10 Issue 4026
***************************************
