[22872] in Perl-Users-Digest
Perl-Users Digest, Issue: 5093 Volume: 10
daemon@ATHENA.MIT.EDU (Perl-Users Digest)
Sun Jun 8 00:10:30 2003
Date: Sat, 7 Jun 2003 21:10:08 -0700 (PDT)
From: Perl-Users Digest <Perl-Users-Request@ruby.OCE.ORST.EDU>
To: Perl-Users@ruby.OCE.ORST.EDU (Perl-Users Digest)
Perl-Users Digest Sat, 7 Jun 2003 Volume: 10 Number: 5093
Today's topics:
Re: Strange errors with huge hash (Tad McClellan)
Re: Strange errors with huge hash <me@verizon.invalid>
Re: Strange errors with huge hash <res1uzbe@verizon.net>
Re: Strange errors with huge hash <uri@stemsystems.com>
Re: Strange errors with huge hash <res1uzbe@verizon.net>
Re: Strange errors with huge hash <res1uzbe@verizon.net>
Re: Strange errors with huge hash <uri@stemsystems.com>
Re: Strange errors with huge hash <flavell@mail.cern.ch>
Re: Strange errors with huge hash <me@verizon.invalid>
Re: Strange errors with huge hash <uri@stemsystems.com>
Re: Strange errors with huge hash <res1uzbe@verizon.net>
Re: Strange errors with huge hash <res1uzbe@verizon.net>
Re: Unusual Can't load fail (web only) <kalinabears@hdc.com.au>
wait() behaviour for threads and DBD driver thread safe gordan@stopspam-bobich.net
Digest Administrivia (Last modified: 6 Apr 01) (Perl-Users-Digest Admin)
----------------------------------------------------------------------
Date: Sat, 7 Jun 2003 17:02:06 -0500
From: tadmc@augustmail.com (Tad McClellan)
Subject: Re: Strange errors with huge hash
Message-Id: <slrnbe4o6u.65u.tadmc@magna.augustmail.com>
emcee <res1uzbe@verizon.net> wrote:
> mkdir($path."data") unless -e($path."data");
You have a race condition there.
I think you said this is a CGI program, so you need to take
multitasking into account.
You should check the return value from mkdir() to see if you
actually got what you requested.
> open(HASH, "$_[0]") or return 0;
^ ^
^ ^
A useless use of double quotes.
open(HASH, $_[0]) or return 0;
> s/'/\'/g;
That does not change anything, so why bother writing it?
That code might be useful in the winter, since the cycles
may have the effect of keeping your feet warm, but it is summer(ish).
--
Tad McClellan SGML consulting
tadmc@augustmail.com Perl programming
Fort Worth, Texas
------------------------------
Date: Sat, 07 Jun 2003 23:07:43 GMT
From: "dw" <me@verizon.invalid>
Subject: Re: Strange errors with huge hash
Message-Id: <3fuEa.17195$b8.3331@nwrdny03.gnilink.net>
"emcee" <res1uzbe@verizon.net> wrote in message
news:b4rEa.62180$Pb.53874@nwrddc01.gnilink.net...
> Uri Guttman wrote:
> sub walk
> {
> my $hash=shift;
> foreach(@_)
> {
> s/'/\'/g;
> s/\\/\\\\/g;
This does (assuming you another \ to the first substitution):
blah 'bl\ah' blah => blah \'bl\ah\' blah => blah \\'bl\\ah\\' blah
I assume you really want:
blah 'bl\ah' blah => blah \'bl\\ah\' blah
wouldn't you want to rewrite the above to happen in the reverse order, or
even:
s/(['\\])/\\\1/g;
> if(ref $hash->{$_})
If your hash looks like:
blah 'bl\ah' blah => some value
the previous substitutions now have you looking for:
blah \'bl\\ah\' blah
which doesn't exist -- a temp var would help
> {
> $hasht.="'$_'=>{";
> &walk($hash->{$_}, keys %{$hash->{$_}});
> chop $hasht;
> $hasht.="},";
> }
> else
> {
> $hash->{$_}=~s/'/\'/g;
> $hash->{$_}=~s/\\/\\\\/g;
same thing here about the order of your substitutions....
> $hasht.="'$_'=>'$hash->{$_}',";
> }
> }
> }
What happens if
$hash->{$_} = [ some, array, reference ]
or some referenced object?
These problems might be why you want to use Data::Dumper.
------------------------------
Date: Sun, 08 Jun 2003 01:48:35 GMT
From: emcee <res1uzbe@verizon.net>
Subject: Re: Strange errors with huge hash
Message-Id: <TBwEa.36648$iT.14845@nwrddc04.gnilink.net>
dw wrote:
> "emcee" <res1uzbe@verizon.net> wrote in message
> news:b4rEa.62180$Pb.53874@nwrddc01.gnilink.net...
>
>>Uri Guttman wrote:
>>sub walk
>>{
>>my $hash=shift;
>>foreach(@_)
>>{
>>s/'/\'/g;
>>s/\\/\\\\/g;
>
>
> This does (assuming you another \ to the first substitution):
> blah 'bl\ah' blah => blah \'bl\ah\' blah => blah \\'bl\\ah\\' blah
> I assume you really want:
> blah 'bl\ah' blah => blah \'bl\\ah\' blah
> wouldn't you want to rewrite the above to happen in the reverse order, or
> even:
> s/(['\\])/\\\1/g;
>
>
>>if(ref $hash->{$_})
>
> If your hash looks like:
> blah 'bl\ah' blah => some value
> the previous substitutions now have you looking for:
> blah \'bl\\ah\' blah
> which doesn't exist -- a temp var would help
>
>
>>{
>>$hasht.="'$_'=>{";
>>&walk($hash->{$_}, keys %{$hash->{$_}});
>>chop $hasht;
>>$hasht.="},";
>>}
>>else
>>{
>>$hash->{$_}=~s/'/\'/g;
>>$hash->{$_}=~s/\\/\\\\/g;
>
> same thing here about the order of your substitutions....
>
>
>>$hasht.="'$_'=>'$hash->{$_}',";
>>}
>>}
>>}
>
>
Ah, thank you, that was it, I hadn't thought that part through to well I
guess, plus I simply forgot the other \ when escaping '. It's working
fine now.
> What happens if
> $hash->{$_} = [ some, array, reference ]
> or some referenced object?
>
It won't, I know because I'm the only one who uses it. If the need does
come up for array references however, I will add that functionality, it
shouldn't take more than a few minutes.
------------------------------
Date: Sun, 08 Jun 2003 01:52:55 GMT
From: Uri Guttman <uri@stemsystems.com>
Subject: Re: Strange errors with huge hash
Message-Id: <x7y90duyh4.fsf@mail.sysarch.com>
>>>>> "d" == dw <me@verizon.invalid> writes:
d> "emcee" <res1uzbe@verizon.net> wrote in message
d> news:b4rEa.62180$Pb.53874@nwrddc01.gnilink.net...
>> Uri Guttman wrote:
i didn't write this ugly code. please check your attrubutions. :)
>> sub walk
>> {
>> my $hash=shift;
>> foreach(@_)
>> {
>> s/'/\'/g;
>> s/\\/\\\\/g;
d> s/(['\\])/\\\1/g;
don't use \1 in the replacement string. read perldoc perlre on why you
should use $1.
d> These problems might be why you want to use Data::Dumper.
well, duh, we know data::dumper gets it right. we have all said that. but
emcee refuses to understand why dumping a hash tree accurately and in a
way that can be evaled is NOT TRIVIAL and not for beginners.
but who are we experts to say such a thing?
bah.
uri
--
Uri Guttman ------ uri@stemsystems.com -------- http://www.stemsystems.com
--Perl Consulting, Stem Development, Systems Architecture, Design and Coding-
Search or Offer Perl Jobs ---------------------------- http://jobs.perl.org
------------------------------
Date: Sun, 08 Jun 2003 01:59:46 GMT
From: emcee <res1uzbe@verizon.net>
Subject: Re: Strange errors with huge hash
Message-Id: <mMwEa.36649$iT.19254@nwrddc04.gnilink.net>
Tad McClellan wrote:
> emcee <res1uzbe@verizon.net> wrote:
>
>
>
>>mkdir($path."data") unless -e($path."data");
>
>
>
> You have a race condition there.
>
> I think you said this is a CGI program, so you need to take
> multitasking into account.
>
>
> You should check the return value from mkdir() to see if you
> actually got what you requested.
>
Yeah, I know, I just added that part last night, before it just assumed
the directory was there. I'm planning on cleaning it up before I run it
on server.
>
>
>> open(HASH, "$_[0]") or return 0;
>
> ^ ^
> ^ ^
>
> A useless use of double quotes.
>
> open(HASH, $_[0]) or return 0;
>
>
Yeah, I had more within those quotes but I took it out and forgot to
remove the quotes, there gone now.
>
>> s/'/\'/g;
>
>
>
> That does not change anything, so why bother writing it?
>
It was supposed to be s/'/\\'/, its fixed now.
> That code might be useful in the winter, since the cycles
> may have the effect of keeping your feet warm, but it is summer(ish).
What part exactly is so inefficent? I've looked through the Data::Dumper
source, it seems no more efficent than my code.
------------------------------
Date: Sun, 08 Jun 2003 02:25:46 GMT
From: emcee <res1uzbe@verizon.net>
Subject: Re: Strange errors with huge hash
Message-Id: <K8xEa.36650$iT.20884@nwrddc04.gnilink.net>
Uri Guttman wrote:
>>>>>>"d" == dw <me@verizon.invalid> writes:
>
>
> d> "emcee" <res1uzbe@verizon.net> wrote in message
> d> news:b4rEa.62180$Pb.53874@nwrddc01.gnilink.net...
> >> Uri Guttman wrote:
>
> i didn't write this ugly code. please check your attrubutions. :)
>
> >> sub walk
> >> {
> >> my $hash=shift;
> >> foreach(@_)
> >> {
> >> s/'/\'/g;
> >> s/\\/\\\\/g;
>
> d> s/(['\\])/\\\1/g;
>
> don't use \1 in the replacement string. read perldoc perlre on why you
> should use $1.
>
> d> These problems might be why you want to use Data::Dumper.
>
> well, duh, we know data::dumper gets it right. we have all said that. but
> emcee refuses to understand why dumping a hash tree accurately and in a
> way that can be evaled is NOT TRIVIAL and not for beginners.
>
> but who are we experts to say such a thing?
>
> bah.
>
> uri
>
Who said I was a beginner? If it was so incredibly hard, and I was such
a novice, I wouldn't have gotten anything that almost worked. But I
wrote it and it worked fine right from the first run, with the exception
of this one situation, which wasn't an issue of me not grasping some
concept or something, I just typed a couple of commands in the wrong order.
I have now just used the module to store a hash with over 12,000 entries
each with 12 key/value pairs, and it worked fine.
------------------------------
Date: Sun, 08 Jun 2003 02:40:36 GMT
From: Uri Guttman <uri@stemsystems.com>
Subject: Re: Strange errors with huge hash
Message-Id: <x7k7bxuw9n.fsf@mail.sysarch.com>
>>>>> "e" == emcee <res1uzbe@verizon.net> writes:
e> Who said I was a beginner? If it was so incredibly hard, and I was
e> such a novice, I wouldn't have gotten anything that almost worked. But
e> I wrote it and it worked fine right from the first run, with the
e> exception of this one situation, which wasn't an issue of me not
e> grasping some concept or something, I just typed a couple of commands
e> in the wrong order.
e> I have now just used the module to store a hash with over 12,000
e> entries each with 12 key/value pairs, and it worked fine.
i saw many general errors in your code but since you know better i won't
correct them.
and here is my version which is guaranteed to work with array refs,
circular data, all forms of scalar data, etc.
use Data::Dumper ;
print Dumper $ref_to_dumbass_large_hash ;
exit ;
uri
--
Uri Guttman ------ uri@stemsystems.com -------- http://www.stemsystems.com
--Perl Consulting, Stem Development, Systems Architecture, Design and Coding-
Search or Offer Perl Jobs ---------------------------- http://jobs.perl.org
------------------------------
Date: Sun, 8 Jun 2003 04:22:59 +0200
From: "Alan J. Flavell" <flavell@mail.cern.ch>
Subject: Re: Strange errors with huge hash
Message-Id: <Pine.LNX.4.53.0306080406460.5788@lxplus086.cern.ch>
On Sun, Jun 8, emcee inscribed on the eternal scroll:
> > may have the effect of keeping your feet warm, but it is summer(ish).
>
> What part exactly is so inefficent? I've looked through the Data::Dumper
> source, it seems no more efficent than my code.
Rule number 1 of programming is "don't optimise yet".
Getting the right answer is important. Doing it fast is _usually_
of secondary importance. Very few computer applications _need_ to be
efficient at runtime. All of them need to be efficient in terms of
developer time.
I recall an application (this was back when a decent computer cost
three quarters of a million pounds, and occupied a large computer
room) which took several days to compute. Code efficiency specialists
worked it over and reduced the time by some factor (tens of percent).
Then a real expert pointed out that the algorithm they were using was
inappropriate to the task, and coded-up a completely different
algorithm. Which got the right answer in 20 minutes.
Nowadays, with CPU speeds a hundred times faster (and nevertheless a
tiny fraction of the cost, and on sale in your local corner store),
I've no doubt that even the sub-optimal algorithm could be computed in
minutes.
By the way, rule 2 of programming is "don't optimise yet". Some would
say that rule 3 is the same.
------------------------------
Date: Sun, 08 Jun 2003 02:58:26 GMT
From: "dw" <me@verizon.invalid>
Subject: Re: Strange errors with huge hash
Message-Id: <mDxEa.17283$b8.12751@nwrdny03.gnilink.net>
"Uri Guttman" <uri@stemsystems.com> wrote in message
news:x7y90duyh4.fsf@mail.sysarch.com...
> i didn't write this ugly code. please check your attrubutions. :)
sorry... I wasn't paying attention to that.... please accept my humble
apologies...
> d> s/(['\\])/\\\1/g;
>
> don't use \1 in the replacement string. read perldoc perlre on why you
> should use $1.
good point... I like the $1 better .... it is just easier to read... I've
been stuck in the TCL world lately....
> well, duh, we know data::dumper gets it right. we have all said that. but
> emcee refuses to understand why dumping a hash tree accurately and in a
> way that can be evaled is NOT TRIVIAL and not for beginners.
I was just trying to re-iterate that thought....
------------------------------
Date: Sun, 08 Jun 2003 03:26:22 GMT
From: Uri Guttman <uri@stemsystems.com>
Subject: Re: Strange errors with huge hash
Message-Id: <x7el25uu5d.fsf@mail.sysarch.com>
>>>>> "d" == dw <me@verizon.invalid> writes:
>>
>> don't use \1 in the replacement string. read perldoc perlre on why you
>> should use $1.
d> good point... I like the $1 better .... it is just easier to read... I've
d> been stuck in the TCL world lately....
it has nothing to do with readability. \1 is wrong. it is supported for
backward compatibility. \1 is only to be used in the regex part of s///
and $1 is to be used in the replacement part. nothing more to it than
that.
uri
--
Uri Guttman ------ uri@stemsystems.com -------- http://www.stemsystems.com
--Perl Consulting, Stem Development, Systems Architecture, Design and Coding-
Search or Offer Perl Jobs ---------------------------- http://jobs.perl.org
------------------------------
Date: Sun, 08 Jun 2003 03:53:21 GMT
From: emcee <res1uzbe@verizon.net>
Subject: Re: Strange errors with huge hash
Message-Id: <RqyEa.30488$JW6.17682@nwrddc02.gnilink.net>
Alan J. Flavell wrote:
> On Sun, Jun 8, emcee inscribed on the eternal scroll:
>
>
>>>may have the effect of keeping your feet warm, but it is summer(ish).
>>
>>What part exactly is so inefficent? I've looked through the Data::Dumper
>>source, it seems no more efficent than my code.
>
>
> Rule number 1 of programming is "don't optimise yet".
>
> Getting the right answer is important. Doing it fast is _usually_
> of secondary importance. Very few computer applications _need_ to be
> efficient at runtime. All of them need to be efficient in terms of
> developer time.
>
> I recall an application (this was back when a decent computer cost
> three quarters of a million pounds, and occupied a large computer
> room) which took several days to compute. Code efficiency specialists
> worked it over and reduced the time by some factor (tens of percent).
> Then a real expert pointed out that the algorithm they were using was
> inappropriate to the task, and coded-up a completely different
> algorithm. Which got the right answer in 20 minutes.
>
> Nowadays, with CPU speeds a hundred times faster (and nevertheless a
> tiny fraction of the cost, and on sale in your local corner store),
> I've no doubt that even the sub-optimal algorithm could be computed in
> minutes.
>
> By the way, rule 2 of programming is "don't optimise yet". Some would
> say that rule 3 is the same.
I agree I shouldn't optimise first, but the nonoptimised version would
just use Data::Dumper, and since that obviously would work, I guess I'm
ready to move on to optimisation.
------------------------------
Date: Sun, 08 Jun 2003 03:59:23 GMT
From: emcee <res1uzbe@verizon.net>
Subject: Re: Strange errors with huge hash
Message-Id: <vwyEa.30499$JW6.30072@nwrddc02.gnilink.net>
Uri Guttman wrote:
>>>>>>"e" == emcee <res1uzbe@verizon.net> writes:
>
>
>
> e> Who said I was a beginner? If it was so incredibly hard, and I was
> e> such a novice, I wouldn't have gotten anything that almost worked. But
> e> I wrote it and it worked fine right from the first run, with the
> e> exception of this one situation, which wasn't an issue of me not
> e> grasping some concept or something, I just typed a couple of commands
> e> in the wrong order.
>
> e> I have now just used the module to store a hash with over 12,000
> e> entries each with 12 key/value pairs, and it worked fine.
>
> i saw many general errors in your code but since you know better i won't
> correct them.
>
> and here is my version which is guaranteed to work with array refs,
> circular data, all forms of scalar data, etc.
>
> use Data::Dumper ;
>
> print Dumper $ref_to_dumbass_large_hash ;
>
> exit ;
>
> uri
>
Alright fine, next time you need to save a hash, you can go ahead and
use Data::Dumper, I will use my module because it meets the criteria:
A: It does what I need it to do.
B: It's more efficent than the alternative.
------------------------------
Date: Sun, 8 Jun 2003 10:55:46 +1000
From: "Sisyphus" <kalinabears@hdc.com.au>
Subject: Re: Unusual Can't load fail (web only)
Message-Id: <3ee28abb$0$1218@echo-01.iinet.net.au>
"Randy Kobes" <randy@theoryx5.uwinnipeg.ca> wrote
> > Does that sound right, or am I on the wrong track?
>
> It might help if it's a problem in finding and loading shared
> libraries
I don't think that there's a problem in *finding* the shared libs. If there
was such a problem then the error message should start with "Can't locate
loadable object..." , not "Can't load...".
Cheers,
Rob
------------------------------
Date: Sun, 08 Jun 2003 00:57:06 +0100
From: gordan@stopspam-bobich.net
Subject: wait() behaviour for threads and DBD driver thread safety
Message-Id: <HZuEa.64$GU5.137093@newsfep2-win.server.ntli.net>
Hi.
Does anybody have an elegant solution for emulating for threads the
equivalent of wait() for processes? I know that I can do $Thread->join(),
which is sort of equivalent to waitpid(), but is there a solution that
provides for waiting for whichever thread returns first? The best I have
been able to do at the moment is keeping an shared list that keeps track of
threads that have finished and can therefore be immediately join()-ed, but
that is rather messy, IMHO. It also means that I have to constantly poll
(with a sleep() and threads->yield() in the loop), which is less than
ideal. Does anyone know of a better way? Because I have multiple worker
threads which can run for unpredictable amounts of time, I cannot rely on
the order in which the threads were created to join() them back, and I
don't want to block waiting to join() a long running thread, when I could
be handling/reaping/re-starting threads that have finished before.
Additionally, does anybody know whether PostgreSQL DBD driver is thread
safe? I have my program working fine with MySQL when using fork() and
processes, but since switching to threads it seems to crash intermittently.
Does anybody know if the PostgreSQL driver is better behaved with threads
instead of processes?
I don't specifically have to use threads in my application, I would merely
like to take the advantage of threads being lighter on the resources.
TIA.
Gordan
------------------------------
Date: 6 Apr 2001 21:33:47 GMT (Last modified)
From: Perl-Users-Request@ruby.oce.orst.edu (Perl-Users-Digest Admin)
Subject: Digest Administrivia (Last modified: 6 Apr 01)
Message-Id: <null>
Administrivia:
The Perl-Users Digest is a retransmission of the USENET newsgroup
comp.lang.perl.misc. For subscription or unsubscription requests, send
the single line:
subscribe perl-users
or:
unsubscribe perl-users
to almanac@ruby.oce.orst.edu.
To submit articles to comp.lang.perl.announce, send your article to
clpa@perl.com.
To request back copies (available for a week or so), send your request
to almanac@ruby.oce.orst.edu with the command "send perl-users x.y",
where x is the volume number and y is the issue number.
For other requests pertaining to the digest, send mail to
perl-users-request@ruby.oce.orst.edu. Do not waste your time or mine
sending perl questions to the -request address, I don't have time to
answer them even if I did know the answer.
------------------------------
End of Perl-Users Digest V10 Issue 5093
***************************************
