[26221] in Perl-Users-Digest
Perl-Users Digest, Issue: 8406 Volume: 10
daemon@ATHENA.MIT.EDU (Perl-Users Digest)
Mon Sep 12 03:06:25 2005
Date: Mon, 12 Sep 2005 00:05:05 -0700 (PDT)
From: Perl-Users Digest <Perl-Users-Request@ruby.OCE.ORST.EDU>
To: Perl-Users@ruby.OCE.ORST.EDU (Perl-Users Digest)
Perl-Users Digest Mon, 12 Sep 2005 Volume: 10 Number: 8406
Today's topics:
Re: Can a Perl Programmer Pick up PHP quickly? <gogala@sbcglobal.net>
Re: IPC::Shareable Problem with multidimentional hash <scottalorda@libello.com>
Re: IPC::Shareable Problem with multidimentional hash xhoster@gmail.com
Re: not or ! ? <someone@example.com>
Re: not or ! ? <sbryce@scottbryce.com>
Re: script attack <snowhare@nihongo.org>
Re: sorting data - hash vs. list <1usa@llenroc.ude.invalid>
Re: sorting data - hash vs. list <matthew.garrish@sympatico.ca>
Re: Space (\s) count problem <someone@example.com>
Digest Administrivia (Last modified: 6 Apr 01) (Perl-Users-Digest Admin)
----------------------------------------------------------------------
Date: Mon, 12 Sep 2005 00:42:31 GMT
From: Mladen Gogala <gogala@sbcglobal.net>
Subject: Re: Can a Perl Programmer Pick up PHP quickly?
Message-Id: <pan.2005.09.12.00.42.30.867022@sbcglobal.net>
On Sun, 11 Sep 2005 10:35:16 +0000, John Bokma wrote:
> Nonsense. Another Perl hater.
Actually, I love Perl and I use it quite frequently.
I also happen to love PHP.
--
http://www.mgogala.com
------------------------------
Date: Mon, 12 Sep 2005 07:56:39 +0200
From: =?ISO-8859-1?Q?S=E9bastien_Cottalorda?= <scottalorda@libello.com>
Subject: Re: IPC::Shareable Problem with multidimentional hash
Message-Id: <43251897$0$12675$626a54ce@news.free.fr>
xhoster@gmail.com wrote:
> Sébastien_Cottalorda <scottalordaNOSPAM@libello.com> wrote:
>
>>Hi all,
>>
>>I need to use that kind of shared hash:
>>%shared_hash = (
>> '123456' => {
>> 'bla' => 123,
>> 'bli' => 465,
>> ect ...
>> },
>> ect ...
>>);
>>
>>with about 500 primary keys.
>>When I "feed" the %shared_hash, I note that the system create as many
>>shared segment and semaphore than there are primary keys.
>
>
> Yes. This fact is documented.
Where ... ?
I understand limits but not the manner it stores datas like hashes.
>
>>As you can imagine, my process died with "Could not create semaphore
>>set: No space left on device at line ..."
>>
>>If I create small complex structure (10 primary keys), the program works
>>perfectly.
>>
>>If I well understand the perldoc IPC::Shareable documentation, I can
>>create as complex shared structure I want,
>
>
> As you said yourself, a small "complex" structure works perfectly. It is
> not the complexity, but the cardinality, that is the problem.
>
>
>>I don't understand why it
>>crashes.
>
>
> What is there to not understand?
No, I know why it crashes.
In fact, I just want to find a way to share a complex hash without that
limit.
> ...
> [2] IPC::Shareable provides no pre-set limits, but the system does. Namely,
> there are limits on the number of shared memory segments that can be
> allocated and the total amount of memory usable by shared memory. ...
>
> You could try using Storable or Data::Dumper to collapse the inner
> structures into flat strings for storage.
>
> Xho
I've read, re-read, re-re-read the perldoc IPC::Shareable : It seems to
already use Storable to store hash structure.
Concerning Data::Dumper, I don't understand anything (maybe it's me ;-) )
I don't understand how I can use that module to store and manipulate
hash of hash in a share structure.
If you have an exemple, I'll apreciate a lot.
Sebastien
------------------------------
Date: 12 Sep 2005 06:47:53 GMT
From: xhoster@gmail.com
Subject: Re: IPC::Shareable Problem with multidimentional hash
Message-Id: <20050912024753.901$Xs@newsreader.com>
Sébastien_Cottalorda <scottalorda@libello.com> wrote:
> xhoster@gmail.com wrote:
> > Sébastien_Cottalorda <scottalordaNOSPAM@libello.com> wrote:
> >
> >>Hi all,
> >>
> >>I need to use that kind of shared hash:
> >>%shared_hash = (
> >> '123456' => {
> >> 'bla' => 123,
> >> 'bli' => 465,
> >> ect ...
> >> },
> >> ect ...
> >>);
> >>
> >>with about 500 primary keys.
> >>When I "feed" the %shared_hash, I note that the system create as many
> >>shared segment and semaphore than there are primary keys.
> >
> >
> > Yes. This fact is documented.
>
> Where ... ?
> I understand limits but not the manner it stores datas like hashes.
It is not the hashes, it is the references to the hashes:
REFERENCES
When a reference to a non-tied scalar, hash, or array is assigned to a
tie()d variable, IPC::Shareable will attempt to tie() the thingy being
referenced[4] ....
Secondly, since a new shared memory segment is created for each thingy
being referenced, the liberal use of references could cause the system to
approach its limit for the total number of shared memory segments allowed.
> >
> > You could try using Storable or Data::Dumper to collapse the inner
> > structures into flat strings for storage.
> >
> > Xho
>
> I've read, re-read, re-re-read the perldoc IPC::Shareable : It seems to
> already use Storable to store hash structure.
It uses Storable to store the top level hash structure. It also uses
Storable to store each sub-hash too, but it does each of those in a
different shmem segment. If you preemptively use Storable on the inner
hashes, so that they are just strings and not hashrefs by the time they get
stuck in the main hash, then IPC::Shareable would just store them as
strings in the main shmem segment.
>
> Concerning Data::Dumper, I don't understand anything (maybe it's me ;-) )
> I don't understand how I can use that module to store and manipulate
> hash of hash in a share structure.
You use Data::Dumper the same way you use Storable, but the thawing is
more awkward.
> If you have an exemple, I'll apreciate a lot.
It ain't pretty, but I don't think using shared memory ever is pretty.
my ($i,$j);
for $i (1 .. 300){
my %x;
for $j (1..3){
$x{$j} = "fall";
}
$colours{$i} = freeze \%x;
}
print "Charged - Press a key to continue -\n";
my $e=<STDIN>;
foreach $i ( keys %colours){
print "$i => \n";
my $x=thaw $colurs{$i};
foreach $j (keys %$x){
print " $x{$j}\n";
}
}
exit;
Note that this won't be very fast, because it is freezing and thawing the
entire data structure just to access one part of it. I suspect that that
is why Shareable was implemented the way it was (using seperate segments
for each piece) in the first place.
Shared memory is not one of the hard things which Perl has made easy.
Xho
--
-------------------- http://NewsReader.Com/ --------------------
Usenet Newsgroup Service $9.95/Month 30GB
------------------------------
Date: Mon, 12 Sep 2005 02:21:27 GMT
From: "John W. Krahn" <someone@example.com>
Subject: Re: not or ! ?
Message-Id: <HC5Ve.269177$on1.12689@clgrps13>
Scott Bryce wrote:
> Huub wrote:
>> Hi,
>>
>> I'm trying to use 'not' but it gives an error. So, can I do this or
>> is it too simple thought?
>>
>> while !(/m\s/g) # While the character being read is no white space {
>> ... }
>
> What happened when you tried it?
What happened when you read "but it gives an error"?
John
--
use Perl;
program
fulfillment
------------------------------
Date: Sun, 11 Sep 2005 21:05:19 -0600
From: Scott Bryce <sbryce@scottbryce.com>
Subject: Re: not or ! ?
Message-Id: <d_WdnQLGP4_2bbneRVn-jA@comcast.com>
John W. Krahn wrote:
> Scott Bryce wrote:
>
>>Huub wrote:
>>
>>>Hi,
>>>
>>>I'm trying to use 'not' but it gives an error. So, can I do this or
>>>is it too simple thought?
>>>
>>>while !(/m\s/g) # While the character being read is no white space {
>>>... }
>>
>>What happened when you tried it?
>
>
> What happened when you read "but it gives an error"?
I understood that to mean that when he tries 'not' it gives an error. He
didn't tell us what happened when he tried while !(/m\s/g). He
asked us if he can do it. Well, he can try. Did I read that wrong?
------------------------------
Date: Sun, 11 Sep 2005 23:34:24 GMT
From: Benjamin Franz <snowhare@nihongo.org>
Subject: Re: script attack
Message-Id: <pan.2005.09.11.23.34.22.217067@nihongo.org>
On Fri, 09 Sep 2005 13:09:53 -0600, Eric Schwartz wrote:
> Brian Wakem <no@email.com> writes:
>> 'local our' is common use in mod_perl.
>>
>> http://perl.apache.org/docs/1.0/guide/porting.html#The_First_Mystery
>
> Okay, fair enough; all the mod_perl I've done has been with Mason,
> which hides all that with the MasonAllowGlobals Apache directive.
> Even so, I look at those examples, and think, "You know, if they just
> passed variables around, instead of using globals, they wouldn't need
> 'local our' at all."
>
> But thanks for the correction; if I find myself using mod_perl with
> other templating engines, I will remember that.
I'd have to agree. I've been using mod_perl for several years and
never needed 'local our' because I'm adverse to using globals
for any case that I don't have to use them. And there just
aren't a lot of times you _have_ to use them.
I was also startled by the porting guide's comment about seeing so
many warnings that they couldn't tell which ones were important and
which ones weren't: Warnings are nearly always important. If your
code _normally_ outputs tons of warnings, you have a serious
problem and very likely a number of outright bugs.
If you _must_ to do something that generates warnings, turn them
off in a scoped block so you don't have warnings about something
that is intended and understood behavior. I've found that fixing
warnings has a very close relationship to fixing bugs in general.
--
Benjamin Franz
------------------------------
Date: Sun, 11 Sep 2005 22:30:59 GMT
From: "A. Sinan Unur" <1usa@llenroc.ude.invalid>
Subject: Re: sorting data - hash vs. list
Message-Id: <Xns96CEBC57DFFE4asu1cornelledu@127.0.0.1>
"Fred@fred.net" <Fred@fred.net> wrote in
news:qf79i19v7kqklccnr86pqr56ck5rc6loio@4ax.com:
>|C|170|901|2|0000000000008|4|000008|5|09/05/05|6|BART'S HVYDTY
>|PROPHYLACTIC|14|3.99|15|00006|
...
> I need to sort, you guessed it, the date field |5|.
If the date field were formatted YYYYMMDD, you would not have to do any
extra work for sorting.
At this point, the interpretation of 09/05/05 is very ambiguous. Which
of the following formats is it in?
YYMMDD
YYDDMM
DDYYMM
DDMMYY
MMYYDD
MMDDYY
> Regarding "recipe" 3.2 & 3.3 where you use timelocal() to convert back
> and forth between epoch seconds and what a human thinks of time as and
> that's no problem.
That would probably not be necessary, but we would like to know the
format of your date field.
> wrote localtime().) My Problem: I have split each record into an array
> only to realize I have a varying length record due to product
> description, so a substr won't help. And at that rate I'd have to
> "swap" all the subscripts at once to keep the dates with the records
> they go with.
I am really not sure what you are talking about here.
> Back to the hash approach. I'd have to read the entire 18 Meg file
> into memory to sort it. (Unless this concern exposes the limits of my
> technical expertise in perl and programming at large in any case. And
> that is a troubling prospect but one for which I can find no remedy.
> So assuming I did that, could I then sort the filled hash and get the
> results I am looking for from the Unix command `sort -d"|" -f5` ?
If reading the whole file into memory bothers you, why not just use the
sort utility (good old DOS and XP have a sort utility as well)?
> I don't have any code to post
I see a whole buncha code below.
> But just so you'll know I am not a complete rookie I'll post some
> recent work,
Which seems to be completely irrelevant to the issue at hand.
I am a little baffled.
<irrelevant code snipped>
> I have posted here before and been flamed to death, so I guess I'm
> ready if it happens again. OTH -- I'd be interested in any helpful
> critisims on my file delete sub. Plus the main question on how to sort
> that dang file. Thanks,
Well, the following is not a flame.
Have you read the posting guidelines for this group? They contain
invaluable information on how to help yourself and help others help you.
One of the tips in the guidelines is to post code. But, please, not just
any old code that is lying around: Post code that is relevant to your
problem so we can see what you need help with.
Otherwise, this becomes a "write my code for me" service, and we are not
very interested in that.
On the other hand, searching the Google archives for this group may have
uncovered some useful threads.
You might find
http://search.cpan.org/~uri/Sort-Maker-0.04/Sort/Maker.pm
http://search.cpan.org/~creamyg/Sort-External-0.10/lib/Sort/External.pm
useful.
Anyway, here is a naive sort sub which will require you to slurp the
file into an array of lines first. The code is completely tested for
lack of data to test it with (see the posting guidelines on this also).
#!/usr/bin/perl
print sort by_date <STDIN>;
# Assumes that the date is formatted MM/DD/YY
# No years before 2000
sub by_date {
my($a_date, $b_date);
if($a =~ m!\|(\d\d)/(\d\d)/(\d\d)\|!) {
$a_date = "$3$1$2";
}
if($b =~ m!\|(\d\d)/(\d\d)/(\d\d)\|!) {
$b_date = "$3$1$2";
}
return $a_date cmp $b_date;
}
__END__
Sinan
--
A. Sinan Unur <1usa@llenroc.ude.invalid>
(reverse each component and remove .invalid for email address)
comp.lang.perl.misc guidelines on the WWW:
http://mail.augustmail.com/~tadmc/clpmisc/clpmisc_guidelines.html
------------------------------
Date: Sun, 11 Sep 2005 18:38:06 -0400
From: "Matt Garrish" <matthew.garrish@sympatico.ca>
Subject: Re: sorting data - hash vs. list
Message-Id: <dl2Ve.887$1G4.78762@news20.bellglobal.com>
<Fred@fred.net> wrote in message
news:qf79i19v7kqklccnr86pqr56ck5rc6loio@4ax.com...
>
> I have posted here before and been flamed to death, so I guess I'm ready
> if it happens again. OTH -- I'd be
> interested in any helpful critisims on my file delete sub. Plus the main
> question on how to sort that dang file.
> Thanks,
>
I can imagine you get flamed, considering your post (adding the isbn of the
book is called overkill). Anyway, I'm not one to read through reams of other
people's code to nitpick, so I'll only suggest that you google on the term
"Schwartzian Transform". The results should explain how trivial the task of
sorting any data structure with perl can be. For example:
#### Code
use strict;
use warnings;
use Time::Local;
my @data;
while (my $line = <DATA>) {
my @parts = split(/\|/, $line);
my $date = timelocal(1, 1, 1, split('/', $parts[9]));
unshift @parts, $date;
push @data, \@parts;
}
foreach my $aref (sort { $a->[0] <=> $b->[0] } @data) {
print join('|', @{ $aref }[1..$#{$aref}]);
}
__DATA__
|C|170|901|2|0000000000008|4|000008|5|09/05/05|6|
|C|170|901|2|0000000000003|4|000008|5|09/06/06|6|
|C|170|901|2|0000000000001|4|000008|5|09/05/06|6|
|C|170|901|2|0000000000028|4|000008|5|09/06/05|6|
#### Outputs
|C|170|901|2|0000000000008|4|000008|5|09/05/05|6|
|C|170|901|2|0000000000028|4|000008|5|09/06/05|6|
|C|170|901|2|0000000000001|4|000008|5|09/05/06|6|
|C|170|901|2|0000000000003|4|000008|5|09/06/06|6|
------------------------------
Date: Mon, 12 Sep 2005 02:16:02 GMT
From: "John W. Krahn" <someone@example.com>
Subject: Re: Space (\s) count problem
Message-Id: <Cx5Ve.269174$on1.27405@clgrps13>
A. Sinan Unur wrote:
>
> A rudimentary benchmark I ran with the corrected code showed that index
> is about 30% slower than the substitution based solution you posted.
>
> So much for early Sunday morning inspiration.
>
> #!/usr/bin/perl
>
> use strict;
> use warnings;
>
> use Benchmark 'cmpthese';
>
> my $s = join('', '012345678901234567890123456789012' x 100);
> my $k = '0123456789';
>
> cmpthese -1, {
> use_regex => \&use_regex,
> use_index => \&use_index,
> };
>
> sub use_index {
> my ($m, $i) = (0, -1);
> while(-1 != ($i = index($s, $k, $i + 1))) {
> $m += 1;
> }
> return $m;
> }
That will match overlapping patterns. A more acurate comparison would be:
sub use_index {
my ( $m, $i ) = ( 0, -length $k );
while ( -1 != ( $i = index $s, $k, $i += length $k ) ) {
$m++;
}
return $m;
}
An even faster solution is to call length() only one time:
sub use_index {
my $len = length $k;
my ( $m, $i ) = ( 0, -$len );
while ( -1 != ( $i = index $s, $k, $i += $len ) ) {
$m++;
}
return $m;
}
> sub use_regex {
> my $m = ($s =~ s!\Q$k!$k!g);
> }
John
--
use Perl;
program
fulfillment
------------------------------
Date: 6 Apr 2001 21:33:47 GMT (Last modified)
From: Perl-Users-Request@ruby.oce.orst.edu (Perl-Users-Digest Admin)
Subject: Digest Administrivia (Last modified: 6 Apr 01)
Message-Id: <null>
Administrivia:
#The Perl-Users Digest is a retransmission of the USENET newsgroup
#comp.lang.perl.misc. For subscription or unsubscription requests, send
#the single line:
#
# subscribe perl-users
#or:
# unsubscribe perl-users
#
#to almanac@ruby.oce.orst.edu.
NOTE: due to the current flood of worm email banging on ruby, the smtp
server on ruby has been shut off until further notice.
To submit articles to comp.lang.perl.announce, send your article to
clpa@perl.com.
#To request back copies (available for a week or so), send your request
#to almanac@ruby.oce.orst.edu with the command "send perl-users x.y",
#where x is the volume number and y is the issue number.
#For other requests pertaining to the digest, send mail to
#perl-users-request@ruby.oce.orst.edu. Do not waste your time or mine
#sending perl questions to the -request address, I don't have time to
#answer them even if I did know the answer.
------------------------------
End of Perl-Users Digest V10 Issue 8406
***************************************
