[26258] in Perl-Users-Digest
Perl-Users Digest, Issue: 8442 Volume: 10
daemon@ATHENA.MIT.EDU (Perl-Users Digest)
Thu Sep 22 03:05:34 2005
Date: Thu, 22 Sep 2005 00:05:04 -0700 (PDT)
From: Perl-Users Digest <Perl-Users-Request@ruby.OCE.ORST.EDU>
To: Perl-Users@ruby.OCE.ORST.EDU (Perl-Users Digest)
Perl-Users Digest Thu, 22 Sep 2005 Volume: 10 Number: 8442
Today's topics:
Re: *fastest* was to get a large directory listing in P <1usa@llenroc.ude.invalid>
Re: *fastest* was to get a large directory listing in P xhoster@gmail.com
Re: *fastest* was to get a large directory listing in P <1usa@llenroc.ude.invalid>
Analyzing many $scalars for match - then action <rbutcher.nospam@hotmail.com>
Re: Analyzing many $scalars for match - then action <1usa@llenroc.ude.invalid>
Re: Analyzing many $scalars for match - then action <djames@thehub.com.au>
Re: Integer conversion and back again <noreply@gunnar.cc>
Re: newbie, what is the best way to include common sett nospam@geniegate.com
Problem in Executing system command in a forking server <vikrantREMOVE@DELETEsaysnetsoft.com>
Digest Administrivia (Last modified: 6 Apr 01) (Perl-Users-Digest Admin)
----------------------------------------------------------------------
Date: Wed, 21 Sep 2005 22:22:02 GMT
From: "A. Sinan Unur" <1usa@llenroc.ude.invalid>
Subject: Re: *fastest* was to get a large directory listing in Perl
Message-Id: <Xns96D8BAD449E5Fasu1cornelledu@127.0.0.1>
"Seth Brundle" <brundlefly76@hotmail.com> wrote in
news:Y9adnSgxcfb5SqzeRVn-jA@comcast.com:
> There are several methods of getting a large directory listing (3000+
> files in a single directory) in Perl, but all the methods I've tried
> (<*>, readdir) are vastly slower in my usage then using readdir in C.
>
> This doesnt seem to make sense, since I imagine perl is just making
> the same system call.
Maybe you are using readdir incorrectly?
What is your notion of fast & slow?
How do you measure fast and slow?
D:\Home\asu1\UseNet\clpmisc\r> dir
...
09/21/2005 06:14 PM 0 file998
09/21/2005 06:14 PM 0 file999
09/21/2005 06:14 PM 241 myt.pl
09/21/2005 06:18 PM 266 test.pl
3002 File(s) 507 bytes
D:\Home\asu1\UseNet\clpmisc\r> cat test.pl
#!/usr/bin/perl
use strict;
use warnings;
use Benchmark;
sub ls {
opendir my $dir, '.' or die "Cannot opendir '.': $!";
my @files = readdir $dir;
closedir $dir or die "Cannot closedir '.': $!";
}
timethese -1, { ls => \&ls };
__END__
D:\Home\asu1\UseNet\clpmisc\r> test
Benchmark: running ls for at least 1 CPU seconds...
ls: 1 wallclock secs ( 0.76 usr + 0.31 sys = 1.08 CPU) @ 99.35/s
(n=107)
This is on Windows XPSP2, AMD64 running at 1.8Ghz, 1Gb RAM, about 402 MB
allocated.
What is the equivalent C program you tested?
Sinan
--
A. Sinan Unur <1usa@llenroc.ude.invalid>
(reverse each component and remove .invalid for email address)
comp.lang.perl.misc guidelines on the WWW:
http://mail.augustmail.com/~tadmc/clpmisc/clpmisc_guidelines.html
------------------------------
Date: 21 Sep 2005 22:33:07 GMT
From: xhoster@gmail.com
Subject: Re: *fastest* was to get a large directory listing in Perl
Message-Id: <20050921183307.446$2c@newsreader.com>
"Seth Brundle" <brundlefly76@hotmail.com> wrote:
> There are several methods of getting a large directory listing (3000+
> files in a single directory) in Perl, but all the methods I've tried
> (<*>, readdir) are vastly slower in my usage then using readdir in C.
>
> This doesnt seem to make sense, since I imagine perl is just making the
> same system call.
Perl first has to determine if readdir is in a list or a scalar context and
has to unwrap the stack. Then it has to make the same system call as C
does. Then it has to copy the contents of the char* "foo.d_name" someplace
safe (unlike C's readdir), and package that up into a perl scalar, and push
that onto the return stack. And in a list context, it has to do that
repeatedly.
> Opinions appreciated...
It is possible you are doing something silly, like calling the underlying
system call 9,000,000+ times. If you posted code (both C and Perl would be
nice, if you want us to do the comparison) (and actual time measurements,
rather than just "vastly slower") we could offer more informed opinions.
Xho
--
-------------------- http://NewsReader.Com/ --------------------
Usenet Newsgroup Service $9.95/Month 30GB
------------------------------
Date: Thu, 22 Sep 2005 00:23:07 GMT
From: "A. Sinan Unur" <1usa@llenroc.ude.invalid>
Subject: Re: *fastest* was to get a large directory listing in Perl
Message-Id: <Xns96D8CF5E91FA6asu1cornelledu@127.0.0.1>
"A. Sinan Unur" <1usa@llenroc.ude.invalid> wrote in
news:Xns96D8BAD449E5Fasu1cornelledu@127.0.0.1:
> "Seth Brundle" <brundlefly76@hotmail.com> wrote in
> news:Y9adnSgxcfb5SqzeRVn-jA@comcast.com:
>
>> There are several methods of getting a large directory listing (3000+
>> files in a single directory) in Perl, but all the methods I've tried
>> (<*>, readdir) are vastly slower in my usage then using readdir in C.
...
snip Perl code
...
> D:\Home\asu1\UseNet\clpmisc\r> test
> Benchmark: running ls for at least 1 CPU seconds...
> ls: 1 wallclock secs ( 0.76 usr + 0.31 sys = 1.08 CPU) @ 99.35/s
> (n=107)
>
> This is on Windows XPSP2, AMD64 running at 1.8Ghz, 1Gb RAM, about 402
> MB allocated.
So we get about 100 readdirs in list context per second.
> What is the equivalent C program you tested?
The following C program is really not the equivalent of the Perl program
I posted, but it does copy the names, and creates a list of file names
etc.
I first ran a do-nothing version which called an empty ls() function 100
times to get a baseline timing. The time reported by the Windows'
timethis utility reported an average of 0.16 seconds.
Then I wrote the following:
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/types.h>
#include <dirent.h>
void ls(size_t num_files) {
struct dirent *ent;
DIR *dir;
size_t f;
char **list = malloc(1 + num_files * sizeof(*list) );
if( !list ) {
fprintf(stderr, "Memory allocation error\n");
exit(EXIT_FAILURE);
}
dir = opendir(".");
if( !dir ) {
perror("Cannot open '.'");
}
for(f = 0; f != num_files; ++f) {
char *d_name;
ent = readdir(dir);
if( !ent ) {
break;
}
d_name = malloc(1 + strlen(ent->d_name));
if( !d_name ) {
break;
}
strcpy(d_name, ent->d_name);
list[f] = d_name;
}
list[f] = NULL;
}
int main(void) {
struct dirent *ent;
size_t num_files = 0;
DIR *dir = opendir(".");
if( !dir ) {
perror("Cannot open '.'");
}
while(ent = readdir(dir)) {
++num_files;
}
if( closedir(dir) ) {
perror("Cannot close '.'");
}
{
int i;
for(i = 0; i != 100; ++i) {
ls(num_files);
}
}
return 0;
}
D:\Home\asu1\UseNet\clpmisc\r> gcc -Wall -O2 r.c -o r.exe
D:\Home\asu1\UseNet\clpmisc\r> timethis r.exe
TimeThis : Command Line : r.exe
TimeThis : Start Time : Wed Sep 21 20:20:45 2005
TimeThis : End Time : Wed Sep 21 20:20:47 2005
TimeThis : Elapsed Time : 00:00:01.640
So, again, we get about 100 readdirs per second in list context (so to
speak). Now, clearly, I am not a great C programmer, but I would be
interested to see the C program that generates the vastly superior
timings.
Sinan
--
A. Sinan Unur <1usa@llenroc.ude.invalid>
(reverse each component and remove .invalid for email address)
comp.lang.perl.misc guidelines on the WWW:
http://mail.augustmail.com/~tadmc/clpmisc/clpmisc_guidelines.html
------------------------------
Date: Thu, 22 Sep 2005 00:59:36 GMT
From: "Robert" <rbutcher.nospam@hotmail.com>
Subject: Analyzing many $scalars for match - then action
Message-Id: <YlnYe.539209$s54.79301@pd7tw2no>
Hi,
I'm building a small subroutine that will check many variables at once for
certain matches and then take action based on it's results. Currently I am
doing it like this:
if ($name =~ /XXX/ || $email =~ /XXX/ || $inquiry =~ /XXX/ || $phone =~
/XXX/ || $comments =~ /XXX/) { &terminate; }
The the above check the 5 variables for "XXX" and if ANY of them contain
"XXX" goes to sub routine &terminate. I would like to know if there is a
more efficient/cleaner way of doing this with less code. While the above
checks 5 variables, my real script checks 22 which maks for a really long
line of code. Is there a better way to do this? For example (not real code):
if ($name,$email,$inquiry,$phone,$comments =~ /XXX/) { &terminate; }
That of course isn't real code but its cleaner and easier to manage. Thats
what I'm looking for, a better way to match many variables.
Thanx all, much appriciated.
Robert
------------------------------
Date: Thu, 22 Sep 2005 01:04:15 GMT
From: "A. Sinan Unur" <1usa@llenroc.ude.invalid>
Subject: Re: Analyzing many $scalars for match - then action
Message-Id: <Xns96D8D65718331asu1cornelledu@127.0.0.1>
"Robert" <rbutcher.nospam@hotmail.com> wrote in
news:YlnYe.539209$s54.79301@pd7tw2no:
> I'm building a small subroutine that will check many variables at once
> for certain matches and then take action based on it's results.
> Currently I am doing it like this:
>
> if ($name =~ /XXX/ || $email =~ /XXX/ || $inquiry =~ /XXX/ || $phone
> =~ /XXX/ || $comments =~ /XXX/) { &terminate; }
...
> For example (not real code):
>
> if ($name,$email,$inquiry,$phone,$comments =~ /XXX/) { &terminate; }
<UNTESTED>
for( $name, $email, $inquiry, $phone, $comments ) {
terminate if /XXX/;
}
</UNTESTED>
Sinan
--
A. Sinan Unur <1usa@llenroc.ude.invalid>
(reverse each component and remove .invalid for email address)
comp.lang.perl.misc guidelines on the WWW:
http://mail.augustmail.com/~tadmc/clpmisc/clpmisc_guidelines.html
------------------------------
Date: Thu, 22 Sep 2005 11:34:10 +1000
From: Damian James <djames@thehub.com.au>
Subject: Re: Analyzing many $scalars for match - then action
Message-Id: <slrndj42gh.7ec.djames@stumble.qimr.edu.au>
On Thu, 22 Sep 2005 00:59:36 GMT, Robert said:
> I'm building a small subroutine that will check many variables at once for
> certain matches and then take action based on it's results. Currently I am
> doing it like this:
>
> if ($name =~ /XXX/ || $email =~ /XXX/ || $inquiry =~ /XXX/ || $phone =~
> /XXX/ || $comments =~ /XXX/) { &terminate; }
>
> The the above check the 5 variables for "XXX" and if ANY of them contain
> "XXX" goes to sub routine &terminate. I would like to know if there is a
> more efficient/cleaner way of doing this with less code. While the above
> checks 5 variables, my real script checks 22 which maks for a really long
> line of code. Is there a better way to do this? For example (not real code):
>
> if ($name,$email,$inquiry,$phone,$comments =~ /XXX/) { &terminate; }
>
> That of course isn't real code but its cleaner and easier to manage. Thats
> what I'm looking for, a better way to match many variables.
>
> Thanx all, much appriciated.
[untested]
sub check {
my $pattern = shift;
for my $term ( @_ ) {
return 1 if $term =~ /$pattern/;
}
return 0
}
&terminate if check( 'XXX', $name, $email, $inquiry, $phone, $comments);
--Damian
------------------------------
Date: Thu, 22 Sep 2005 00:47:26 +0200
From: Gunnar Hjalmarsson <noreply@gunnar.cc>
Subject: Re: Integer conversion and back again
Message-Id: <3pe684F9p1c7U1@individual.net>
cp wrote:
> I have a database field to hold a string that is represented as a
> series of digits separated by dashes. The string will be input by a
> user. The format of the string is 1-3 digits followed by a three digit
> sequence, follow by a three digit sequence.
>
> Users being what they are, many are dropping the leading zeros in the
> middle and end set. I need to treat the user input of 060-090-014 the
> same as 60-090-14 and the same as 60-90-14.
>
> My thought was to convert the string to an integer, and store it in the
> database (in the example, as 60090014). Since the last three digits
> represent the sequence in which the numbers where issued, that also
> makes the strings easier to sort.
>
> I adapted the add commas re from perl FAQ and came up with the
> following code. I'd appreciate any other ideas.
Well, I find your approach rather complicated.
> Error checking to make
> sure that the number contains two dashes, etc. has been omitted.
> Eventually the final three digits will go to four digits. Can I get a
> suggestion for a solution that deals with that?
This would print a validated string in its desired format, and also take
care of error checking (to some degree) and the four digit requirement:
my @groups = $n =~ /^(\d{1,3})-(\d{1,3})-(\d{1,4})$/
or die "Wrong input format";
printf '%d-%03d-%04d', @groups;
--
Gunnar Hjalmarsson
Email: http://www.gunnar.cc/cgi-bin/contact.pl
------------------------------
Date: Thu, 22 Sep 2005 00:44:10 GMT
From: nospam@geniegate.com
Subject: Re: newbie, what is the best way to include common settings.conf values
Message-Id: <Lucy112734268039480xfc06e8@air.tunestar.net>
In: <3FaYe.10292$st1.5292@newsfe3-gui.ntli.net>, "Pritchie" <info2005@remove-this-including-dot.bigbunker.com> wrote:
>Hi
>What is the best way to include common settings/values into a perl script?
There isn't a best way really, it can be ultra simple but it can
get really complex. (Especially complex when configurations
need to be maintained across multiple servers)
The real catch is that you may not want to have settings stored
in multiple files for multiple programs.
>in shell script you can have a file called "common-vars.sh" with
>sMyVar="My string"
>
>and call it from "MyProgram.sh" by
>.. ./common-vars.sh
>echo ${sMyVar}
>
>so when you run ./MyProgram.sh you get echoed back
>My String
Do you need the same variables within shell scripts? What I mean is,
does it need to work across shell -and- perl?
If so, your best bet is probably just to have a very simple
"common-vars.sh" with: NAME=VALUE and then parse it. (probably
wise to place a magic '#--BEGIN--CONF and #--END-CONF some place
in the shell script in case the script changes)
If the shell needs them quoted, look into Test::ParseWords to
help with that.
This has the advantage of being readable by shell AND perl, parsing easy.
Disadvantage in that if the shell needs to start doing things to compute the
values, for example locating a directory:
P=`dirname $0`
cd $P # something/bin/my_conf.sh
cd .. # something/
#--BEGIN-CONF
CONF_DIR=`pwd`
#--END-CONF
It won't work, perl would likely see CONF_DIR as a literal `pwd`. (you could, I
suppose always set up the script to be executed and dump conf to stdout, that
works, it's flexible, but has overhead)
If you'll be needing more advanced configuration(s), look into
Config::General, it's "apache-like" support does a lot more
stuff.
Trouble is, we start out with simple NAME=VALUE pairs or my favorite, a global
%CFG... then, as time goes on, run into situations where it just doesn't work.
At least with a hash, you can use 'tie' on it later if you need to.
Jamie
--
http://www.geniegate.com Custom web programming
guhzo_42@lnubb.pbz (rot13) User Management Solutions
------------------------------
Date: Thu, 22 Sep 2005 11:38:23 +0530
From: vikrant <vikrantREMOVE@DELETEsaysnetsoft.com>
Subject: Problem in Executing system command in a forking server
Message-Id: <dgthsc$98j$1@domitilla.aioe.org>
hi
I am trying to store the return value of system command after execution in a forking server.
The command executes successfully but returns "-1". In case of failure, it returns the same value
"-1", like when i am replacing "date" with "data" in system command. From my understanding, it
should return -1 only in case of failure. How do go about distinguishing between a success and
failure call while using the system commnad?
Code:-
-----------------------------------------------------------------------------------
#!/usr/bin/perl -w
use strict;
use IO::Socket;
use IO::Select;
$SIG{CHLD} = 'IGNORE';
my $sock = new IO::Socket::INET(
LocalHost => '10.0.0.23',
LocalPort => '36545',
Proto => 'tcp',
Listen => SOMAXCONN,
Reuse => 1);
$sock or die "no socket :$!";
REQUEST:
while (my $sNew_Socket = $sock->accept())
{
my $kid=fork();
if ($kid)
{
close $sNew_Socket;
next REQUEST;
}
close $sock;
my $obSelected_Socket = IO::Select->new($sNew_Socket);
my $sData_Recevied;
while( $obSelected_Socket->can_read(20))
{
my $sBuffer;
sysread($sNew_Socket,$sBuffer,1<<10);
$sData_Recevied.=$sBuffer;
if($sBuffer =~/\/END>/)
{
last;
}
}
my $sDatecmd=system("date");
$sNew_Socket->send($sDatecmd);
exit;
}
-----------------------------------------------------------------------------------------------------
Thanks
Vikrant
------------------------------
Date: 6 Apr 2001 21:33:47 GMT (Last modified)
From: Perl-Users-Request@ruby.oce.orst.edu (Perl-Users-Digest Admin)
Subject: Digest Administrivia (Last modified: 6 Apr 01)
Message-Id: <null>
Administrivia:
#The Perl-Users Digest is a retransmission of the USENET newsgroup
#comp.lang.perl.misc. For subscription or unsubscription requests, send
#the single line:
#
# subscribe perl-users
#or:
# unsubscribe perl-users
#
#to almanac@ruby.oce.orst.edu.
NOTE: due to the current flood of worm email banging on ruby, the smtp
server on ruby has been shut off until further notice.
To submit articles to comp.lang.perl.announce, send your article to
clpa@perl.com.
#To request back copies (available for a week or so), send your request
#to almanac@ruby.oce.orst.edu with the command "send perl-users x.y",
#where x is the volume number and y is the issue number.
#For other requests pertaining to the digest, send mail to
#perl-users-request@ruby.oce.orst.edu. Do not waste your time or mine
#sending perl questions to the -request address, I don't have time to
#answer them even if I did know the answer.
------------------------------
End of Perl-Users Digest V10 Issue 8442
***************************************
