[29741] in Perl-Users-Digest
Perl-Users Digest, Issue: 985 Volume: 11
daemon@ATHENA.MIT.EDU (Perl-Users Digest)
Sun Oct 28 16:09:43 2007
Date: Sun, 28 Oct 2007 13:09:06 -0700 (PDT)
From: Perl-Users Digest <Perl-Users-Request@ruby.OCE.ORST.EDU>
To: Perl-Users@ruby.OCE.ORST.EDU (Perl-Users Digest)
Perl-Users Digest Sun, 28 Oct 2007 Volume: 11 Number: 985
Today's topics:
Re: configurable variables in own file? <bik.mido@tiscalinet.it>
Re: configurable variables in own file? <tzz@lifelogs.com>
Re: configurable variables in own file? <bik.mido@tiscalinet.it>
Re: FAQ 5.11 How can I write() into a string? <hjp-usenet2@hjp.at>
Re: how to redirect error to variable <stoupa@practisoft.cz>
new CPAN modules on Sun Oct 28 2007 (Randal Schwartz)
Re: packing a C structure <mark.clementsREMOVETHIS@wanadoo.fr>
Re: perl time function <Peter@PSDT.com>
Re: perl time function <hjp-usenet2@hjp.at>
Re: reading a directory, first files the newest ones <noreply@gunnar.cc>
Re: reading a directory, first files the newest ones <Juha.Laiho@iki.fi>
Re: reading a directory, first files the newest ones <hjp-usenet2@hjp.at>
Re: reading a directory, first files the newest ones <hjp-usenet2@hjp.at>
Re: reading a directory, first files the newest ones <mark.clementsREMOVETHIS@wanadoo.fr>
Digest Administrivia (Last modified: 6 Apr 01) (Perl-Users-Digest Admin)
----------------------------------------------------------------------
Date: Sun, 28 Oct 2007 10:34:20 +0100
From: Michele Dondi <bik.mido@tiscalinet.it>
Subject: Re: configurable variables in own file?
Message-Id: <6vk8i3h6040g9kv41gf4bpg9p5921ik086@4ax.com>
On Sat, 27 Oct 2007 10:28:14 -0500, Ted Zlatanov <tzz@lifelogs.com>
wrote:
>On 26 Oct 2007 18:27:03 GMT xhoster@gmail.com wrote:
>
>x> I already know what a hash looks like in Perl. Why spend the
>x> effort to learn what a hash looks like in some other language, too,
>x> without some compelling reason?
>
>You're joking, I hope. We write software for the users, not for
No he's not! I think that your POVs are *complementary* rather than
conflicting, and describing two possible situations which may be
considered extremes such that one approach is well suited for one of
them and not that well suited for the other, and vice versa. Between
these two extremes there may be a continuous spectrum of situations in
which things are not that clear. But to dismiss his well reasoned
arguments a "jokes" is not fair.
Michele
--
{$_=pack'B8'x25,unpack'A8'x32,$a^=sub{pop^pop}->(map substr
(($a||=join'',map--$|x$_,(unpack'w',unpack'u','G^<R<Y]*YB='
.'KYU;*EVH[.FHF2W+#"\Z*5TI/ER<Z`S(G.DZZ9OX0Z')=~/./g)x2,$_,
256),7,249);s/[^\w,]/ /g;$ \=/^J/?$/:"\r";print,redo}#JAPH,
------------------------------
Date: Sun, 28 Oct 2007 06:30:15 -0500
From: Ted Zlatanov <tzz@lifelogs.com>
Subject: Re: configurable variables in own file?
Message-Id: <m2ejffpjco.fsf@lifelogs.com>
On Sun, 28 Oct 2007 10:34:20 +0100 Michele Dondi <bik.mido@tiscalinet.it> wrote:
MD> On Sat, 27 Oct 2007 10:28:14 -0500, Ted Zlatanov <tzz@lifelogs.com>
MD> wrote:
>> On 26 Oct 2007 18:27:03 GMT xhoster@gmail.com wrote:
>>
x> I already know what a hash looks like in Perl. Why spend the
x> effort to learn what a hash looks like in some other language, too,
x> without some compelling reason?
>>
>> You're joking, I hope. We write software for the users, not for
MD> No he's not! I think that your POVs are *complementary* rather than
MD> conflicting, and describing two possible situations which may be
MD> considered extremes such that one approach is well suited for one of
MD> them and not that well suited for the other, and vice versa. Between
MD> these two extremes there may be a continuous spectrum of situations in
MD> which things are not that clear. But to dismiss his well reasoned
MD> arguments a "jokes" is not fair.
I was objecting to the use of a Perl programmer as the standard for what
a configuration editor user should know. To me, asking users to learn
Perl in order to edit a configuration file is completely unreasonable
and quite funny based on my experience. Try proposing that to the
users:
"You will configure everything in Perl."
"Is there a GUI to do this?"
"Well no, you should just learn Perl."
"OK, we'll try it. Why are some things in single and others in double
quotes?"
"That's OK, the double quotes do variable interpolation but otherwise
they are the same. You need a backslash inside double quotes for some
things, so it's best to avoid them unless blah blah blah..." (the user
has drifted off at this point)
"Grr. OK, I want to make this message multiple lines."
"Just use a here-doc. Oh, to disable interpolation you have to quote
the end-of-document marker. Or you could put it in a separate file and
write this input loop..."
"Are you serious?" (this sentence probably comes sooner in the dialog)
Ted
------------------------------
Date: Sun, 28 Oct 2007 12:37:15 +0100
From: Michele Dondi <bik.mido@tiscalinet.it>
Subject: Re: configurable variables in own file?
Message-Id: <v0t8i3tri1m3k5n6o2b32qjbaqvsmrr2d0@4ax.com>
On Sun, 28 Oct 2007 06:30:15 -0500, Ted Zlatanov <tzz@lifelogs.com>
wrote:
>I was objecting to the use of a Perl programmer as the standard for what
>a configuration editor user should know. To me, asking users to learn
>Perl in order to edit a configuration file is completely unreasonable
I agree.
>and quite funny based on my experience. Try proposing that to the
That is not what was being proposed. I think both the OP and x had in
mind a situation in which users have some Perl experience a priori.
Michele
--
{$_=pack'B8'x25,unpack'A8'x32,$a^=sub{pop^pop}->(map substr
(($a||=join'',map--$|x$_,(unpack'w',unpack'u','G^<R<Y]*YB='
.'KYU;*EVH[.FHF2W+#"\Z*5TI/ER<Z`S(G.DZZ9OX0Z')=~/./g)x2,$_,
256),7,249);s/[^\w,]/ /g;$ \=/^J/?$/:"\r";print,redo}#JAPH,
------------------------------
Date: Sun, 28 Oct 2007 11:37:03 +0100
From: "Peter J. Holzer" <hjp-usenet2@hjp.at>
Subject: Re: FAQ 5.11 How can I write() into a string?
Message-Id: <slrnfi8pif.i6q.hjp-usenet2@zeno.hjp.at>
On 2007-10-27 19:17, brian d foy <brian.d.foy@gmail.com> wrote:
> In article <slrnfi6683.77r.hjp-usenet2@zeno.hjp.at>, Peter J. Holzer
><hjp-usenet2@hjp.at> wrote:
>>
>> How can I open a file handle to a string?
>
> That would be a good question, although you might want to phrase is
> "How do I print to a string instead of a file?".
That was my first idea, too. I changed it because it works for
reading a string as well as writing.
> Thanks, :)
You're welcome.
hp
--
_ | Peter J. Holzer | It took a genius to create [TeX],
|_|_) | Sysadmin WSR | and it takes a genius to maintain it.
| | | hjp@hjp.at | That's not engineering, that's art.
__/ | http://www.hjp.at/ | -- David Kastrup in comp.text.tex
------------------------------
Date: Sun, 28 Oct 2007 18:48:06 +0100
From: "Petr Vileta" <stoupa@practisoft.cz>
Subject: Re: how to redirect error to variable
Message-Id: <fg2if5$2s55$1@ns.felk.cvut.cz>
Mumia W. wrote:
> On 10/27/2007 08:30 AM, Petr Vileta wrote:
>> [...]
>> This work as expected. I tried to use IO::Scalar but without success.
>> What is wrong in this code?
>>
>> use IO::Scalar;
>> $SIG{'PIPE'}=sub {
>> my $err='';
>> $SH=new IO::Scalar \$err;
>> open STDERR,,">&SH";
>> cluck("SIG-PIPE ");
>> print $err;
>> exit;
>> };
>>
>
> I don't think it'll work with IO::Scalar. Why not do it the right way?
>
> my $err = '';
>
> $SIG{PIPE} = sub {
> $err .= Carp::shortmess('SIG-PIPE ');
> print $err;
> exit 1;
> };
>
> I doubt you want $err to be lexically scoped within the sub, and I
> think you need to read "man Carp" again.
Hmm, thanks for direct me to right way. I use Perl 5.6.1 and my perldoc Carp
say nothing about shortmess() or longmess().
---CUT---
D:\Perl>perldoc Carp
NAME
carp - warn of errors (from perspective of caller)
cluck - warn of errors with stack backtrace (not exported by default)
croak - die of errors (from perspective of caller)
confess - die of errors with stack backtrace
---CUT---
and on CPAN these functions are documented for Perl 5.8.x and later. But it
works in my perl version too.
--
Petr Vileta, Czech republic
(My server rejects all messages from Yahoo and Hotmail. Send me your mail
from another non-spammer site please.)
------------------------------
Date: Sun, 28 Oct 2007 04:42:17 GMT
From: merlyn@stonehenge.com (Randal Schwartz)
Subject: new CPAN modules on Sun Oct 28 2007
Message-Id: <JqLuEH.J30@zorch.sf-bay.org>
The following modules have recently been added to or updated in the
Comprehensive Perl Archive Network (CPAN). You can install them using the
instructions in the 'perlmodinstall' page included with your Perl
distribution.
Buffalo-G54-0.02
http://search.cpan.org/~mschilli/Buffalo-G54-0.02/
Limited scraping API for Buffalo WBR2-G54 routers
----
Business-ISBN-2.03_01
http://search.cpan.org/~bdfoy/Business-ISBN-2.03_01/
work with International Standard Book Numbers
----
Business-ISBN-Data-1.17
http://search.cpan.org/~bdfoy/Business-ISBN-Data-1.17/
data pack for Business::ISBN
----
Crypt-CCM-0.02
http://search.cpan.org/~oyama/Crypt-CCM-0.02/
CCM Mode for symmetric key block ciphers
----
Crypt-GCM-0.02
http://search.cpan.org/~oyama/Crypt-GCM-0.02/
Galois/Counter Mode (GCM)
----
Data-Page-Balanced-1.0.0
http://search.cpan.org/~kimahl/Data-Page-Balanced-1.0.0/
A data pager that will balance the number of entries per page.
----
EV-0.01
http://search.cpan.org/~mlehmann/EV-0.01/
perl interface to libevent, monkey.org/~provos/libevent/
----
Games-Tournament-Swiss-0.12
http://search.cpan.org/~drbean/Games-Tournament-Swiss-0.12/
FIDE Swiss Same-Rank Contestant Pairing
----
Games-Tournament-Swiss-0.13
http://search.cpan.org/~drbean/Games-Tournament-Swiss-0.13/
FIDE Swiss Same-Rank Contestant Pairing
----
Games-Tournament-Swiss-0.14
http://search.cpan.org/~drbean/Games-Tournament-Swiss-0.14/
FIDE Swiss Same-Rank Contestant Pairing
----
HTML-Element-Tiny-0.001
http://search.cpan.org/~hdp/HTML-Element-Tiny-0.001/
lightweight DOM-like elements
----
IncPatch-0.01
http://search.cpan.org/~sartak/IncPatch-0.01/
incrementally apply diffs
----
IncPatch-0.02
http://search.cpan.org/~sartak/IncPatch-0.02/
incrementally apply diffs with patch, a la darcs
----
KinoSearch-0.162
http://search.cpan.org/~creamyg/KinoSearch-0.162/
search engine library
----
KinoSearch-0.20_05
http://search.cpan.org/~creamyg/KinoSearch-0.20_05/
Search engine library.
----
Lingua-EN-Conjugate-0.305
http://search.cpan.org/~rwg/Lingua-EN-Conjugate-0.305/
Conjugation of English verbs
----
Net-LDAP-Server-0.4
http://search.cpan.org/~aar/Net-LDAP-Server-0.4/
LDAP server side protocol handling
----
Net-MAC-Vendor-1.18
http://search.cpan.org/~bdfoy/Net-MAC-Vendor-1.18/
look up the vendor for a MAC
----
Net-TiVo-0.09
http://search.cpan.org/~boumenot/Net-TiVo-0.09/
Perl interface to TiVo.
----
Parse-Marpa-0.001_024
http://search.cpan.org/~jkegl/Parse-Marpa-0.001_024/
Earley's Algorithm, with improvements
----
Pod-Multi-0.08
http://search.cpan.org/~jkeenan/Pod-Multi-0.08/
pod2man, pod2text, pod2html simultaneously
----
Pod-Multi-0.09
http://search.cpan.org/~jkeenan/Pod-Multi-0.09/
pod2man, pod2text, pod2html simultaneously
----
Pod-ProjectDocs-0.34
http://search.cpan.org/~lyokato/Pod-ProjectDocs-0.34/
generates CPAN like pod pages
----
TRL-Microarray-0.063
http://search.cpan.org/~cjones/TRL-Microarray-0.063/
A Perl module for creating and manipulating microarray objects
----
Test-Data-1.21
http://search.cpan.org/~bdfoy/Test-Data-1.21/
test functions for particular variable types
----
Test-Deep-0.099_a1
http://search.cpan.org/~fdaly/Test-Deep-0.099_a1/
Extremely flexible deep comparison
----
Test-File-1.19
http://search.cpan.org/~bdfoy/Test-File-1.19/
test file attributes
----
Test-HTTPStatus-1.07
http://search.cpan.org/~bdfoy/Test-HTTPStatus-1.07/
check an HTTP status
----
Test-ISBN-2.01
http://search.cpan.org/~bdfoy/Test-ISBN-2.01/
Check International Standard Book Numbers
----
Test-Manifest-1.22
http://search.cpan.org/~bdfoy/Test-Manifest-1.22/
interact with a t/test_manifest file
----
Text-AutoLink-0.03000
http://search.cpan.org/~dmaki/Text-AutoLink-0.03000/
Automatically Linkfy
----
Tie-Cycle-1.16
http://search.cpan.org/~bdfoy/Tie-Cycle-1.16/
Cycle through a list of values via a scalar.
----
Tie-Filehandle-Preempt-Stdin-0.02
http://search.cpan.org/~jkeenan/Tie-Filehandle-Preempt-Stdin-0.02/
Preempt STDIN during testing.
----
Tk-Enscript-1.09
http://search.cpan.org/~srezic/Tk-Enscript-1.09/
a text-to-postscript converter using Tk::Canvas
If you're an author of one of these modules, please submit a detailed
announcement to comp.lang.perl.announce, and we'll pass it along.
This message was generated by a Perl program described in my Linux
Magazine column, which can be found on-line (along with more than
200 other freely available past column articles) at
http://www.stonehenge.com/merlyn/LinuxMag/col82.html
print "Just another Perl hacker," # the original
--
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<merlyn@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!
------------------------------
Date: Sun, 28 Oct 2007 17:50:05 +0100
From: Mark Clements <mark.clementsREMOVETHIS@wanadoo.fr>
Subject: Re: packing a C structure
Message-Id: <4724bdbd$0$25921$ba4acef3@news.orange.fr>
haairam@gmail.com wrote:
> hi ,,,
>
> iam trying to pack a C structure to send via socket..i would like to
> know how to frame the template for the C structure..
>
> this is my structure
>
> typedef struct buffer_t {
> uint32_t a;
> char b[16];
> uint32_t c;
> uint32_t d;
> char e[6];
> char f[8];
> } buffer_t;
>
> kindly suggest me the format for the pack function call..this will
> help me to solve my issue...
>
Have you checked out Convert::Binary::C?
http://search.cpan.org/~mhx/Convert-Binary-C-0.68/lib/Convert/Binary/C.pm
Mark
------------------------------
Date: Sun, 28 Oct 2007 09:03:09 GMT
From: Peter Scott <Peter@PSDT.com>
Subject: Re: perl time function
Message-Id: <pan.2007.10.28.09.03.08.933841@PSDT.com>
On Sat, 27 Oct 2007 17:00:42 +0200, Peter J. Holzer wrote:
> On 2007-10-27 11:48, Ben Morrow <ben@morrow.me.uk> wrote:
>> [Spurious Perl connection: this is similar to the hobbit's system of
>> treating leap days as not part of the year... :)]
>
> What does the hobbit's system have to do with Perl?
Take a look at the comments at the beginnings of .c files in the Perl
source distribution.
--
Peter Scott
http://www.perlmedic.com/
http://www.perldebugged.com/
------------------------------
Date: Sun, 28 Oct 2007 11:28:33 +0100
From: "Peter J. Holzer" <hjp-usenet2@hjp.at>
Subject: Re: perl time function
Message-Id: <slrnfi8p2h.i6q.hjp-usenet2@zeno.hjp.at>
On 2007-10-28 09:03, Peter Scott <Peter@PSDT.com> wrote:
> On Sat, 27 Oct 2007 17:00:42 +0200, Peter J. Holzer wrote:
>> On 2007-10-27 11:48, Ben Morrow <ben@morrow.me.uk> wrote:
>>> [Spurious Perl connection: this is similar to the hobbit's system of
>>> treating leap days as not part of the year... :)]
>>
>> What does the hobbit's system have to do with Perl?
>
> Take a look at the comments at the beginnings of .c files in the Perl
> source distribution.
Ah, I get it now. Thanks.
hp
--
_ | Peter J. Holzer | It took a genius to create [TeX],
|_|_) | Sysadmin WSR | and it takes a genius to maintain it.
| | | hjp@hjp.at | That's not engineering, that's art.
__/ | http://www.hjp.at/ | -- David Kastrup in comp.text.tex
------------------------------
Date: Sun, 28 Oct 2007 05:29:36 +0100
From: Gunnar Hjalmarsson <noreply@gunnar.cc>
Subject: Re: reading a directory, first files the newest ones
Message-Id: <5oihhqFmnsm1U1@mid.individual.net>
xhoster@gmail.com wrote:
> Gunnar Hjalmarsson <noreply@gunnar.cc> wrote:
>> jordilin wrote:
>>> On Oct 28, 2:02 am, Gunnar Hjalmarsson <nore...@gunnar.cc> wrote:
>>>> Maybe you should let the system do the desired sorting. On *nix that
>>>> might be:
>>>>
>>>> chomp( my @files = qx(ls -t $dir) );
>>>> foreach my $file (@files) {
>>>> last if -M "$dir/$file" > 2/24;
>>>> print "$file\n";
>>>> }
> ...
>>> The solution would be reading the newest ones first.
>> And that's what the -t option achieves...
>
> No, the -t option tells ls to *present* the newest ones first, not to
> read them first. To present them in that order, it first needs to read all
> of the directory entries in whatever order the file system deigns to
> deliver them, stat them all, and sort the results based on time.
So far I agree, but ...
> There is no reason to think that ls is going to be meaningfully
> faster about this than perl will.
... my benchmark (see below) indicates otherwise. The difference seems
to increase when the directory size increases.
$ cat sortdir.pl
use Benchmark 'cmpthese';
my $dir = '/usr/lib';
cmpthese -5, {
Linux => sub {
chomp( my @files = qx(ls -t $dir) );
},
Perl => sub {
chdir $dir;
opendir( my $DH, '.' );
my @files = map { $_->[0] }
sort { $a->[1] <=> $b->[1] } map { [ $_, -M ] }
grep substr($_, 0, 1) ne '.', readdir $DH;
},
};
$ perl sortdir.pl
Rate Perl Linux
Perl 174/s -- -75%
Linux 693/s 297% --
--
Gunnar Hjalmarsson
Email: http://www.gunnar.cc/cgi-bin/contact.pl
------------------------------
Date: Sun, 28 Oct 2007 09:17:03 GMT
From: Juha Laiho <Juha.Laiho@iki.fi>
Subject: Re: reading a directory, first files the newest ones
Message-Id: <fg1k07$8fm$1@ichaos2.ichaos-int>
jordilin <jordilin@gmail.com> said:
>On Oct 28, 2:04 am, "Jürgen Exner" <jurge...@hotmail.com> wrote:
>> jordilin wrote:
>> > On Oct 28, 1:36 am, Gunnar Hjalmarsson <nore...@gunnar.cc> wrote:
>> >> You may want to use grep() to assign to an array the files you are
>> >> interested in.
>>
>> >> my @files = grep -M $_ <= 2/24, readdir DIR;
>
>Yeah, it seems that this would be a solution.
If you're concerned over the memory use, you wouldn't use this -- it'll
implicitly _first_ load all the directory entries into the memory, and
will start filtering the modification dates only after the directory
scan has been completed.
A small benchmark, to run stat() on all files in a directory (a directory
with one million files, names 000000 .. 999999):
$ time perl -e 'opendir($DH,"."); while ($f=readdir($DH)) { stat($f); }; closedir($DH);'
real 8m0.728s
user 0m2.367s
sys 0m21.523s
While running this, the memory usage (according to "top") was rather
constant 7MB.
Another way to do the same - which would behave like the "grep" example:
$ time perl -e 'opendir($DH,"."); foreach $f (readdir($DH)) { stat($f); }; closedir($DH);'
real 11m9.247s
user 0m1.957s
sys 0m21.953s
With this approach, the memory usage initially climbed to reach approx. 50MB,
and remained there until the completion.
--
Wolf a.k.a. Juha Laiho Espoo, Finland
(GC 3.0) GIT d- s+: a C++ ULSH++++$ P++@ L+++ E- W+$@ N++ !K w !O !M V
PS(+) PE Y+ PGP(+) t- 5 !X R !tv b+ !DI D G e+ h---- r+++ y++++
"...cancel my subscription to the resurrection!" (Jim Morrison)
------------------------------
Date: Sun, 28 Oct 2007 12:03:30 +0100
From: "Peter J. Holzer" <hjp-usenet2@hjp.at>
Subject: Re: reading a directory, first files the newest ones
Message-Id: <slrnfi8r42.i6q.hjp-usenet2@zeno.hjp.at>
On 2007-10-28 02:18, jordilin <jordilin@gmail.com> wrote:
> On Oct 28, 2:02 am, Gunnar Hjalmarsson <nore...@gunnar.cc> wrote:
>> jordilin wrote:
>> > When I read a huge directory with opendir,
>> > opendir(DIR,"dirname");
>> > my $file;
>> > while($file=readdir(DIR))
>> > whatever...
>> > it loads the oldest ones first. I would like the newest files first,
>> > instead of the oldest. Taking into account that I am only interested
>> > in the newest files, this takes a lot of time, as the directory is
>> > really huge. I am talking about thousands and thousands of files. I
>> > need to process the files that are two hours old from now. I am not
>> > interested in those older than two hours ago.
>>
>> Maybe you should let the system do the desired sorting. On *nix that
>> might be:
>>
>> chomp( my @files = qx(ls -t $dir) );
>> foreach my $file (@files) {
>> last if -M "$dir/$file" > 2/24;
>> print "$file\n";
>> }
>
> With this code, and taking into account that the directory is huge,
> memory usage would be a problem as we are going to use a huge array
> @files, and the Unix server is a very important one.
That would be easily remedied by reading from a pipe. But I don't think
Gunnar's suggestion is really faster. It needs to stat read the
directory and stat all the files (which takes the same time as your
code), *then* it needs to sort them (which takes additional time),
*then* your code needs to read the sorted list.
> Don't know if that could be achieved by means of a while. The real
> problem is having to process many files before arriving to the
> interesting ones. The solution would be reading the newest ones first.
> I think there is no solution.
The solution, as Xho suggested, is to come up with a better directory
structure.
If you can't do that:
Is there any way you can deduce the age of the files from the file name?
If you can avoid stat'ing all these files it will be a lot faster. You
don't need the exact age - if you can determine, from the filename
alone, that a file is surely older than two hours you don't have to stat
it.
Do these files get written once, or are they constantly updated? If it's
the former, you can cache their last-modified-dates. Reading them from
a file or memcached is likely to be a lot faster than a stat.
hp
--
_ | Peter J. Holzer | It took a genius to create [TeX],
|_|_) | Sysadmin WSR | and it takes a genius to maintain it.
| | | hjp@hjp.at | That's not engineering, that's art.
__/ | http://www.hjp.at/ | -- David Kastrup in comp.text.tex
------------------------------
Date: Sun, 28 Oct 2007 12:10:50 +0100
From: "Peter J. Holzer" <hjp-usenet2@hjp.at>
Subject: Re: reading a directory, first files the newest ones
Message-Id: <slrnfi8rhq.i6q.hjp-usenet2@zeno.hjp.at>
On 2007-10-28 04:29, Gunnar Hjalmarsson <noreply@gunnar.cc> wrote:
> xhoster@gmail.com wrote:
>> There is no reason to think that ls is going to be meaningfully
>> faster about this than perl will.
>
> ... my benchmark (see below) indicates otherwise. The difference seems
> to increase when the directory size increases.
>
> $ cat sortdir.pl
> use Benchmark 'cmpthese';
> my $dir = '/usr/lib';
> cmpthese -5, {
> Linux => sub {
> chomp( my @files = qx(ls -t $dir) );
> },
> Perl => sub {
> chdir $dir;
> opendir( my $DH, '.' );
> my @files = map { $_->[0] }
> sort { $a->[1] <=> $b->[1] } map { [ $_, -M ] }
> grep substr($_, 0, 1) ne '.', readdir $DH;
> },
> };
>
> $ perl sortdir.pl
> Rate Perl Linux
> Perl 174/s -- -75%
> Linux 693/s 297% --
>
Your benchmark isn't valid: You are processing the complete directory
several hundred times per second, which indicates that it fits
completely into the buffer cache. After the first time you are measuring
mostly the processing time of ls and perl, not disk accesses.
Jordilin wrote that it takes about 10 minutes to process the directory
just once, which indicates that it either doesn't fit into the cache, or
that it is evicted from the cache between runs (which is quite likely on
a busy system), so he does have to access the disk for every file.
hp
--
_ | Peter J. Holzer | It took a genius to create [TeX],
|_|_) | Sysadmin WSR | and it takes a genius to maintain it.
| | | hjp@hjp.at | That's not engineering, that's art.
__/ | http://www.hjp.at/ | -- David Kastrup in comp.text.tex
------------------------------
Date: Sun, 28 Oct 2007 17:58:46 +0100
From: Mark Clements <mark.clementsREMOVETHIS@wanadoo.fr>
Subject: Re: reading a directory, first files the newest ones
Message-Id: <4724bfc6$0$27398$ba4acef3@news.orange.fr>
jordilin wrote:
> When I read a huge directory with opendir,
> opendir(DIR,"dirname");
> my $file;
> while($file=readdir(DIR))
> whatever...
> it loads the oldest ones first. I would like the newest files first,
> instead of the oldest. Taking into account that I am only interested
> in the newest files, this takes a lot of time, as the directory is
> really huge. I am talking about thousands and thousands of files. I
> need to process the files that are two hours old from now. I am not
> interested in those older than two hours ago. I know that because I
> check the modification time with stat.
> any idea?
> Thanks in advance
>
Maybe you could do something with File::Monitor, although I know nothing
of its efficiency with large directories. FAM may also be worth a look.
Mark
------------------------------
Date: 6 Apr 2001 21:33:47 GMT (Last modified)
From: Perl-Users-Request@ruby.oce.orst.edu (Perl-Users-Digest Admin)
Subject: Digest Administrivia (Last modified: 6 Apr 01)
Message-Id: <null>
Administrivia:
#The Perl-Users Digest is a retransmission of the USENET newsgroup
#comp.lang.perl.misc. For subscription or unsubscription requests, send
#the single line:
#
# subscribe perl-users
#or:
# unsubscribe perl-users
#
#to almanac@ruby.oce.orst.edu.
NOTE: due to the current flood of worm email banging on ruby, the smtp
server on ruby has been shut off until further notice.
To submit articles to comp.lang.perl.announce, send your article to
clpa@perl.com.
#To request back copies (available for a week or so), send your request
#to almanac@ruby.oce.orst.edu with the command "send perl-users x.y",
#where x is the volume number and y is the issue number.
#For other requests pertaining to the digest, send mail to
#perl-users-request@ruby.oce.orst.edu. Do not waste your time or mine
#sending perl questions to the -request address, I don't have time to
#answer them even if I did know the answer.
------------------------------
End of Perl-Users Digest V11 Issue 985
**************************************
