[28335] in Perl-Users-Digest
Perl-Users Digest, Issue: 9699 Volume: 10
daemon@ATHENA.MIT.EDU (Perl-Users Digest)
Thu Sep 7 14:10:16 2006
Date: Thu, 7 Sep 2006 11:10:08 -0700 (PDT)
From: Perl-Users Digest <Perl-Users-Request@ruby.OCE.ORST.EDU>
To: Perl-Users@ruby.OCE.ORST.EDU (Perl-Users Digest)
Perl-Users Digest Thu, 7 Sep 2006 Volume: 10 Number: 9699
Today's topics:
Perl and du output difference. <george.e.sullivan@saic.com>
Re: Perl and du output difference. <george.e.sullivan@saic.com>
Re: Perl and du output difference. <thepoet_nospam@arcor.de>
Re: Perl and du output difference. <someone@example.com>
Re: Perl and du output difference. <someone@example.com>
Re: Perl and du output difference. (Randal L. Schwartz)
Re: Perl and du output difference. <bik.mido@tiscalinet.it>
Re: Perl and du output difference. <george.e.sullivan@saic.com>
Re: Problems detecting multiple clients xhoster@gmail.com
Re: time icrement based loop <deadpickle@gmail.com>
TLS script christophergraber@gmail.com
Re: TLS script <tadmc@augustmail.com>
Re: WHY NOT FINDING HASH KEY???? <tadmc@augustmail.com>
Re: WHY NOT FINDING HASH KEY???? <1usa@llenroc.ude.invalid>
Digest Administrivia (Last modified: 6 Apr 01) (Perl-Users-Digest Admin)
----------------------------------------------------------------------
Date: 7 Sep 2006 07:21:43 -0700
From: "george.e.sullivan@saic.com" <george.e.sullivan@saic.com>
Subject: Perl and du output difference.
Message-Id: <1157638903.095919.40580@h48g2000cwc.googlegroups.com>
In a closed thread Mr. John W. Kahn posted a script to add up directory
usage per user that produces simple output of such as:
userA 112345
userB 57389293
userC 323
and so forth
Here is Mr. Kahn's script:
perl -MFile::Find -le '($m) = stat( $d = shift ); find( sub{ @s =
lstat; $m == $s[0] and $u{ getpwuid $s[4] } += $s[7]}, $d ); printf
"%-5s %d\n", $_, $u{$_} for sort { $u{$b} <=> $u{$a} } keys %u' .
The above is a cut and paste from there.
Output on one of my larger directories produces almost a 1 gigabyte
difference between this script and the du -ks command syntax.
du -ks = 37,928,180,000
script = 38,641,548,183
Is there any minute error in the script that would cause this or is the
script actually reading deeper into the file/directory structure and
accounting for unused blocks on the hard drive or other overhead types?
Thanks to all.
------------------------------
Date: 7 Sep 2006 08:36:38 -0700
From: "george.e.sullivan@saic.com" <george.e.sullivan@saic.com>
Subject: Re: Perl and du output difference.
Message-Id: <1157643397.945508.222770@h48g2000cwc.googlegroups.com>
george.e.sullivan@saic.com wrote:
> In a closed thread Mr. John W. Kahn posted a script to add up directory
> usage per user that produces simple output of such as:
>
> userA 112345
> userB 57389293
> userC 323
>
> and so forth
> Here is Mr. Kahn's script:
>
> perl -MFile::Find -le '($m) = stat( $d = shift ); find( sub{ @s =
> lstat; $m == $s[0] and $u{ getpwuid $s[4] } += $s[7]}, $d ); printf
> "%-5s %d\n", $_, $u{$_} for sort { $u{$b} <=> $u{$a} } keys %u' .
>
> The above is a cut and paste from there.
>
> Output on one of my larger directories produces almost a 1 gigabyte
> difference between this script and the du -ks command syntax.
>
> du -ks = 37,928,180,000
> script = 38,641,548,183
>
> Is there any minute error in the script that would cause this or is the
> script actually reading deeper into the file/directory structure and
> accounting for unused blocks on the hard drive or other overhead types?
>
> Thanks to all.
Also, if anyone knows how to modify the above script so that it sums
the individual totals that would be helpful also. Thanks to all.
------------------------------
Date: Thu, 07 Sep 2006 17:48:52 +0200
From: Christian Winter <thepoet_nospam@arcor.de>
Subject: Re: Perl and du output difference.
Message-Id: <45003f64$0$18486$9b4e6d93@newsspool3.arcor-online.net>
george.e.sullivan@saic.com wrote:
> In a closed thread Mr. John W. Kahn
Actually, his last name is spelled Krahn (with an "r").
> posted a script to add up directory
> usage per user that produces simple output of such as:
>
> userA 112345
> userB 57389293
> userC 323
>
> and so forth
> Here is Mr. Kahn's script:
>
> perl -MFile::Find -le '($m) = stat( $d = shift ); find( sub{ @s =
> lstat; $m == $s[0] and $u{ getpwuid $s[4] } += $s[7]}, $d ); printf
> "%-5s %d\n", $_, $u{$_} for sort { $u{$b} <=> $u{$a} } keys %u' .
>
> The above is a cut and paste from there.
>
> Output on one of my larger directories produces almost a 1 gigabyte
> difference between this script and the du -ks command syntax.
>
> du -ks = 37,928,180,000
What version of du on what OS are you using? On my systems
(Linux, GNU coreutils) the "-k" option means "give result in 1k blocks".
> script = 38,641,548,183
It works completely fine here in a test directory, giving
script = 176814243
du -sb = 176814243
> Is there any minute error in the script that would cause this or is the
> script actually reading deeper into the file/directory structure and
> accounting for unused blocks on the hard drive or other overhead types?
Could it be that your directories contain hard linked files? I just
checked it, and it seems that File::Find doesn't remember if a hard
linked file is accessed more than once. But that can be mended by
checking if the inode number has already been found before:
perl -MFile::Find -le '($m) = stat( $d = shift ); find( sub{ @s = lstat;
$m == $s[0] and !$seen{$s[1]}++ and $u{ getpwuid $s[4] } += $s[7]}, $d);
printf "%-5s %d\n", $_, $u{$_} for sort { $u{$b} <=> $u{$a} } keys %u' .
To get the total you just have to sum up the values of %u:
perl -MFile::Find -le '($m) = stat( $d = shift ); find( sub{ @s = lstat;
$m == $s[0] and !$seen{$s[1]}++ and $u{ getpwuid $s[4] } += $s[7]}, $d )
; printf "%-5s %d\n", $_, $u{$_} for sort { $u{$b} <=> $u{$a} } keys %u;
printf "Total %d\n", eval join "+", values %u' .
HTH
-Chris
------------------------------
Date: Thu, 07 Sep 2006 16:20:06 GMT
From: "John W. Krahn" <someone@example.com>
Subject: Re: Perl and du output difference.
Message-Id: <WEXLg.11079$Hr1.9419@clgrps12>
george.e.sullivan@saic.com wrote:
> In a closed thread Mr. John W. Kahn posted a script to add up directory
> usage per user that produces simple output of such as:
>
> userA 112345
> userB 57389293
> userC 323
>
> and so forth
> Here is Mr. Kahn's script:
>
> perl -MFile::Find -le '($m) = stat( $d = shift ); find( sub{ @s =
> lstat; $m == $s[0] and $u{ getpwuid $s[4] } += $s[7]}, $d ); printf
> "%-5s %d\n", $_, $u{$_} for sort { $u{$b} <=> $u{$a} } keys %u' .
>
> The above is a cut and paste from there.
>
> Output on one of my larger directories produces almost a 1 gigabyte
> difference between this script and the du -ks command syntax.
>
> du -ks = 37,928,180,000
> script = 38,641,548,183
>
> Is there any minute error in the script that would cause this or is the
> script actually reading deeper into the file/directory structure and
> accounting for unused blocks on the hard drive or other overhead types?
du checks for hard linked files but that script doesn't so you could have some
hard linked files that are counted multiple times by the script. Try du with
the -l switch and see if they produce the same results.
If you want the script to ignore hard linked files:
perl -MFile::Find -le'($m) = stat( $d = shift ); find( sub{ @s = lstat; $m ==
$s[0] and !$seen{$s[1]}++ and $u{ getpwuid $s[4] } += $s[7] }, $d ); printf
"%-5s %d\n", $_, $u{$_} for sort { $u{$b} <=> $u{$a} } keys %u' .
John
--
use Perl;
program
fulfillment
------------------------------
Date: Thu, 07 Sep 2006 16:24:03 GMT
From: "John W. Krahn" <someone@example.com>
Subject: Re: Perl and du output difference.
Message-Id: <DIXLg.11089$Hr1.10642@clgrps12>
george.e.sullivan@saic.com wrote:
> george.e.sullivan@saic.com wrote:
>>
>>perl -MFile::Find -le '($m) = stat( $d = shift ); find( sub{ @s =
>>lstat; $m == $s[0] and $u{ getpwuid $s[4] } += $s[7]}, $d ); printf
>>"%-5s %d\n", $_, $u{$_} for sort { $u{$b} <=> $u{$a} } keys %u' .
>
> Also, if anyone knows how to modify the above script so that it sums
> the individual totals that would be helpful also. Thanks to all.
perl -MFile::Find -le'($m) = stat( $d = shift ); find( sub{ @s = lstat; $m ==
$s[0] and $u{ getpwuid $s[4] } += $s[7] }, $d ); printf "%-5s %d\n", $_,
$u{$_}, $t += $u{$_} for sort { $u{$b} <=> $u{$a} } keys %u; print "Total: $t"' .
John
--
use Perl;
program
fulfillment
------------------------------
Date: 07 Sep 2006 09:37:25 -0700
From: merlyn@stonehenge.com (Randal L. Schwartz)
To: "george.e.sullivan@saic.com" <george.e.sullivan@saic.com>
Subject: Re: Perl and du output difference.
Message-Id: <86y7svmy4a.fsf@blue.stonehenge.com>
>>>>> "george" == george e sullivan@saic com <george.e.sullivan@saic.com> writes:
george> perl -MFile::Find -le '($m) = stat( $d = shift ); find( sub{ @s =
george> lstat; $m == $s[0] and $u{ getpwuid $s[4] } += $s[7]}, $d ); printf
george> "%-5s %d\n", $_, $u{$_} for sort { $u{$b} <=> $u{$a} } keys %u' .
This is adding $s[7] (highest byte in use), when it really should be adding
$s[11] * $s[12] (block size * number of blocks), which correctly deals with
indirect blocks and file holes. See the source for "du".
For example:
perl -e 'open X, ">somefile"; seek X, 2**30, 0 or die "$!"; print X "x"'
creates a file that is "1 gigabyte", but du reports as only a few
dozen blocks (the indirect blocks).
--
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<merlyn@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!
--
Posted via a free Usenet account from http://www.teranews.com
------------------------------
Date: 7 Sep 2006 18:44:18 +0200
From: Michele Dondi <bik.mido@tiscalinet.it>
Subject: Re: Perl and du output difference.
Message-Id: <8ui0g2h8eha6j54mbnp5rg3hvejb21j23s@4ax.com>
On 7 Sep 2006 07:21:43 -0700, "george.e.sullivan@saic.com"
<george.e.sullivan@saic.com> wrote:
>In a closed thread Mr. John W. Kahn posted a script to add up directory
>usage per user that produces simple output of such as:
[snip]
>Output on one of my larger directories produces almost a 1 gigabyte
>difference between this script and the du -ks command syntax.
I just gave a very quick peek into the thread and I see that stuff
like hard links is being mentioned. All this may well be relevant. But
also take into account that the *disk usage* of a file is generally
different from its exact size, and the former depends on the block
size of the device. This circumstance affects most commonly used osen.
Michele
--
{$_=pack'B8'x25,unpack'A8'x32,$a^=sub{pop^pop}->(map substr
(($a||=join'',map--$|x$_,(unpack'w',unpack'u','G^<R<Y]*YB='
.'KYU;*EVH[.FHF2W+#"\Z*5TI/ER<Z`S(G.DZZ9OX0Z')=~/./g)x2,$_,
256),7,249);s/[^\w,]/ /g;$ \=/^J/?$/:"\r";print,redo}#JAPH,
------------------------------
Date: 7 Sep 2006 10:17:01 -0700
From: "george.e.sullivan@saic.com" <george.e.sullivan@saic.com>
Subject: Re: Perl and du output difference.
Message-Id: <1157649421.412608.59240@m73g2000cwd.googlegroups.com>
Michele Dondi wrote:
> On 7 Sep 2006 07:21:43 -0700, "george.e.sullivan@saic.com"
> <george.e.sullivan@saic.com> wrote:
>
> >In a closed thread Mr. John W. Kahn posted a script to add up directory
> >usage per user that produces simple output of such as:
> [snip]
> >Output on one of my larger directories produces almost a 1 gigabyte
> >difference between this script and the du -ks command syntax.
>
> I just gave a very quick peek into the thread and I see that stuff
> like hard links is being mentioned. All this may well be relevant. But
> also take into account that the *disk usage* of a file is generally
> different from its exact size, and the former depends on the block
> size of the device. This circumstance affects most commonly used osen.
>
>
> Michele
> --
> {$_=pack'B8'x25,unpack'A8'x32,$a^=sub{pop^pop}->(map substr
> (($a||=join'',map--$|x$_,(unpack'w',unpack'u','G^<R<Y]*YB='
> .'KYU;*EVH[.FHF2W+#"\Z*5TI/ER<Z`S(G.DZZ9OX0Z')=~/./g)x2,$_,
> 256),7,249);s/[^\w,]/ /g;$ \=/^J/?$/:"\r";print,redo}#JAPH,
Oh heck... a typo. I am so sorry about the name misspelling. Thanks
to Christian Winter for pointing that out. And my apology Mr. Krahn.
Your script, however, is going to be very useful and I thank you for
taking the time to post it along with the updates.
The du I am using is on a SGI Altix 350 running their version of Redhat
Linux. I believe it is based on Advanced Server 3.0. SGI boxed it and
added their on bells and whistles. Also thank for the advice on
adding the individual numbers.
The "l" and "s" both product more comparible numbers. Thanks for that
tip. I know...man pages...they are there for a reason. Especially for
us who find old habits hard to break. :) :) :)
Randal... I will also try fields 11 and 12 so thanks for that tip also.
And Michele...I was worried about the actual space being used, blocking
factors, and such also. Thanks.
I am grateful for all the information each of you has provided. This
is truely a great community of users.
------------------------------
Date: 07 Sep 2006 16:25:54 GMT
From: xhoster@gmail.com
Subject: Re: Problems detecting multiple clients
Message-Id: <20060907122620.604$mb@newsreader.com>
"samasama" <bryan@worldspice.net> wrote:
> > It would probably make sense to have %ip_conns be permanently reversed,
> > i.e. have the IP be the key and socket be the value (or the value could
> > just be "1" or undef, since the socket part which is stored in that
> > hash is never used anyway (and indeed probably can't be used--hash keys
> > are stringifed and thus lose their magic.))
> >
>
> I had a "Oh yeah..." moment... One of the main reasons I'm keying by
> $CLIENT is because $CLIENT will be unique every connection. If I were
> to key by ip address, this may not be true, two connections from
> 127.0.0.1 will not get two seperate keys...
I had thought that duplicate connections from the same IP were disallowed,
so that that wouldn't be an issue. Still, if you are allowing multiple
connections per IP, you could always make a hash mapping IP to count.
> AFAIK. I need it to be
> unique every time, be able to create multiple keys for the same ip
> address. So with that, keying by $CLIENT does the job I want.
>
> Though, now I'm trying to figure out out how to remove a single
> instance...
>
> my $pid = waitpid(-1, WNOHANG); # Wait for children to die
>
> foreach my $keys (keys %children) {
> my $value = $children{$keys};
> if ($pid == $keys) {
There is no reason to loop over all keys if you only do anything with
one of those keys.
if (exists $children{$pid}) {
my $value = $children{$pid};
#...
> my %temp = reverse %ip_conns;
> delete $temp{$value};
> %ip_conns = reverse %temp;
If you are going to keep %ip_conns like it is, then you should
store the socket (rather than the IP) in %children, then you could just do:
delete $ip_conns{$value};
Rather than all the reversing of entire hashes (which has added benefit of
not breaking if you allow multiple connects per IP, as you describe below).
> }
> }
>
> This works perfectly for right now, by down the road I might want to
> allow two connections from the same host at once. In this case I
> believe I would need to delete only one instance of $value.
> I just started thinking about that so... I'm sure the solution is
> quite easy.
> Perhaps I could key %ip_conns by $kidpid, hmmm, have to think about
> that.
I still think you should change %ip_conns to be IP => count, rather than
socket=>IP, but if you keep it like it is you should store the socket,
rather than the IP, in %children.
Xho
--
-------------------- http://NewsReader.Com/ --------------------
Usenet Newsgroup Service $9.95/Month 30GB
------------------------------
Date: 7 Sep 2006 06:57:21 -0700
From: "deadpickle" <deadpickle@gmail.com>
Subject: Re: time icrement based loop
Message-Id: <1157637441.510292.205230@i3g2000cwc.googlegroups.com>
deadpickle wrote:
> Tad McClellan wrote:
> > deadpickle <deadpickle@gmail.com> wrote:
> >
> > > I get a bunch of errors:
> > > Use of uninitilized value in string eq/ne at
> > > C:/PXPerl/lib/Net/FileShare.pm
> >
> >
> > That is not an error message.
> >
> > It is a warning message.
> >
> > (BTW: An "error" is different from an "error message".)
> >
> >
> > > I have no clue as to whats going on.
> >
> >
> > You are using an undef value when you don't want to be.
> >
> Is there a way to make this value defined, i tried installing the
> module Net::Fileshare withh activestate's PPM but I guess it did not
> work.
I got Net::Fileshare to be defined and now I dont get the uninitalized
error any more but I still get the packet error. Is this because the
connection did not close? If I enter "close ($fh)'" nothing happens,
how can I get the client to close its connection?
------------------------------
Date: 7 Sep 2006 07:53:47 -0700
From: christophergraber@gmail.com
Subject: TLS script
Message-Id: <1157640827.759790.308140@d34g2000cwd.googlegroups.com>
Hi,
I have poked around but haven't been able to figure this out:
I need to create a script that will go out and check a few TLS FTP
sites we have at my work. It needs to check every day to see if any new
file(s) have been placed since the previous day. If there is a new
file, the script should send me an email notification.
I have something similar working for "normal" FTP sites that uses
Net::FTP. It connects to each FTP site, finds the most recent file
modification time with $ftp->mdtm($file), then compares this time to
the value held in a simple text file i've set up.
I don't think Net::Lite::FTP would allow me to determine modification
time on a TLS FTP site. I have only used it to push/pull files to a TLS
site.
A few folks I have asked this question to have said "write a shell
script" but I don't know how to start. "Use wget or curl", etc, they
say.
Any ideas for me?
Thanks,
Chris
------------------------------
Date: Thu, 7 Sep 2006 11:06:31 -0500
From: Tad McClellan <tadmc@augustmail.com>
Subject: Re: TLS script
Message-Id: <slrneg0gs7.e9n.tadmc@magna.augustmail.com>
christophergraber@gmail.com <christophergraber@gmail.com> wrote:
> Subject: TLS script
What is "TLS" ?
--
Tad McClellan SGML consulting
tadmc@augustmail.com Perl programming
Fort Worth, Texas
------------------------------
Date: Thu, 7 Sep 2006 08:17:15 -0500
From: Tad McClellan <tadmc@augustmail.com>
Subject: Re: WHY NOT FINDING HASH KEY????
Message-Id: <slrneg06ur.dnh.tadmc@magna.augustmail.com>
jason@cyberpine.com <jason@cyberpine.com> wrote:
> Subject: WHY NOT FINDING HASH KEY????
Why is your thread invisible?
Because it hits *two* scorerules that indicate it can be ignored
without missing anything of consequence:
% foolish subjects (no lower case letters)
Score: -9000
~Subject: \c[a-z]
% foolish subjects
Score:: -9000
Subject: ^perl$
Subject: ^help!?$
Subject: ^question!?$
Subject: ^perl question!?$
Subject: (none)
Subject: no subject
Subject: ^$
Subject: !!!
Subject: ###
Subject: ~~~
Subject: \?\?\?
Subject: \$\$\$
--
Tad McClellan SGML consulting
tadmc@augustmail.com Perl programming
Fort Worth, Texas
------------------------------
Date: Thu, 7 Sep 2006 14:57:32 +0000 (UTC)
From: "A. Sinan Unur" <1usa@llenroc.ude.invalid>
Subject: Re: WHY NOT FINDING HASH KEY????
Message-Id: <Xns98376F7A31B97asu1cornelledu@132.236.56.8>
"Paul Lalli" <mritty@gmail.com> wrote in news:1157634187.902985.101410
@m79g2000cwm.googlegroups.com:
> jason@cyberpine.com wrote:
>> My bad.
>> Clearly I'm desperate.
>
You are about to get a little more desperate.
Sinan
------------------------------
Date: 6 Apr 2001 21:33:47 GMT (Last modified)
From: Perl-Users-Request@ruby.oce.orst.edu (Perl-Users-Digest Admin)
Subject: Digest Administrivia (Last modified: 6 Apr 01)
Message-Id: <null>
Administrivia:
#The Perl-Users Digest is a retransmission of the USENET newsgroup
#comp.lang.perl.misc. For subscription or unsubscription requests, send
#the single line:
#
# subscribe perl-users
#or:
# unsubscribe perl-users
#
#to almanac@ruby.oce.orst.edu.
NOTE: due to the current flood of worm email banging on ruby, the smtp
server on ruby has been shut off until further notice.
To submit articles to comp.lang.perl.announce, send your article to
clpa@perl.com.
#To request back copies (available for a week or so), send your request
#to almanac@ruby.oce.orst.edu with the command "send perl-users x.y",
#where x is the volume number and y is the issue number.
#For other requests pertaining to the digest, send mail to
#perl-users-request@ruby.oce.orst.edu. Do not waste your time or mine
#sending perl questions to the -request address, I don't have time to
#answer them even if I did know the answer.
------------------------------
End of Perl-Users Digest V10 Issue 9699
***************************************
