[31809] in Perl-Users-Digest
Perl-Users Digest, Issue: 3072 Volume: 11
daemon@ATHENA.MIT.EDU (Perl-Users Digest)
Wed Aug 11 16:09:57 2010
Date: Wed, 11 Aug 2010 13:09:07 -0700 (PDT)
From: Perl-Users Digest <Perl-Users-Request@ruby.OCE.ORST.EDU>
To: Perl-Users@ruby.OCE.ORST.EDU (Perl-Users Digest)
Perl-Users Digest Wed, 11 Aug 2010 Volume: 11 Number: 3072
Today's topics:
Re: FAQ 5.23 I still don't get locking. I just want to <justin.1007@purestblue.com>
Re: FAQ 5.23 I still don't get locking. I just want to <ben@morrow.me.uk>
Re: Plot Module Question <sbryce@scottbryce.com>
Re: Plot Module Question <edgrsprj@ix.netcom.com>
s/// returns the empty string, but not "" jidanni@jidanni.org
Re: s/// returns the empty string, but not "" <uri@StemSystems.com>
Re: simple indexing in Perl? (Jens Thoms Toerring)
Re: simple indexing in Perl? <ela@yantai.org>
Re: simple indexing in Perl? <xhoster@gmail.com>
Re: simple indexing in Perl? <tadmc@seesig.invalid>
Re: simple indexing in Perl? sln@netherlands.com
Re: sysopen failures <m@rtij.nl.invlalid>
Digest Administrivia (Last modified: 6 Apr 01) (Perl-Users-Digest Admin)
----------------------------------------------------------------------
Date: Wed, 11 Aug 2010 09:17:54 +0100
From: Justin C <justin.1007@purestblue.com>
Subject: Re: FAQ 5.23 I still don't get locking. I just want to increment the number in the file. How can I do this?
Message-Id: <iu2aj7-05t.ln1@zem.masonsmusic.co.uk>
On 2010-08-10, PerlFAQ Server <brian@theperlreview.com> wrote:
[snip]
The code example in the original is over 80 columns wide and could be
easier to follow - especially in ye olde slrn in an 80 column terminal.
I've reformatted it, perhaps $FAQ_maintainer may consider the change?
> Anyway, this is what you can do if you can't help yourself.
>
> use Fcntl qw(:DEFAULT :flock);
> sysopen my $fh, "numfile", O_RDWR|O_CREAT
> or die "can't open numfile: $!";
> flock $fh, LOCK_EX
> or die "can't flock numfile: $!";
> my $num = <$fh> || 0;
> seek $fh, 0, 0
> or die "can't rewind numfile: $!";
> truncate $fh, 0
> or die "can't truncate numfile: $!";
> (print $fh $num+1, "\n")
> or die "can't write numfile: $!";
> close $fh
> or die "can't close numfile: $!";
Justin.
--
Justin C, by the sea.
------------------------------
Date: Wed, 11 Aug 2010 11:41:12 +0100
From: Ben Morrow <ben@morrow.me.uk>
Subject: Re: FAQ 5.23 I still don't get locking. I just want to increment the number in the file. How can I do this?
Message-Id: <8bbaj7-89u1.ln1@osiris.mauzo.dyndns.org>
Quoth Justin C <justin.1007@purestblue.com>:
> On 2010-08-10, PerlFAQ Server <brian@theperlreview.com> wrote:
>
> The code example in the original is over 80 columns wide and could be
> easier to follow - especially in ye olde slrn in an 80 column terminal.
Ooh, yes, that's nasty. I rather suspect someone's gone trawling through
the docs adding 'or die' everywhere without thinking too hard about the
formatting.
> I've reformatted it, perhaps $FAQ_maintainer may consider the change?
>
> > Anyway, this is what you can do if you can't help yourself.
> >
> > use Fcntl qw(:DEFAULT :flock);
> > sysopen my $fh, "numfile", O_RDWR|O_CREAT
> > or die "can't open numfile: $!";
> > flock $fh, LOCK_EX
> > or die "can't flock numfile: $!";
> > my $num = <$fh> || 0;
> > seek $fh, 0, 0
> > or die "can't rewind numfile: $!";
> > truncate $fh, 0
> > or die "can't truncate numfile: $!";
> > (print $fh $num+1, "\n")
> > or die "can't write numfile: $!";
> > close $fh
> > or die "can't close numfile: $!";
Since autodie has been core since 5.10, this would seem like a perfect
place to use it. I don't know what the FAQ's policy is on things like
that (myself, I'm starting to seriously consider adding 'autodie' to
'warnings' and 'strict' as something I always have on).
Ben
------------------------------
Date: Tue, 10 Aug 2010 22:52:16 -0600
From: Scott Bryce <sbryce@scottbryce.com>
Subject: Re: Plot Module Question
Message-Id: <i3taap$4gt$1@news.eternal-september.org>
E.D.G. wrote:
> Why use an ocean liner to do what should be possible with a rowboat?
Because at some point someone is going to want to row across the ocean.
------------------------------
Date: Wed, 11 Aug 2010 02:43:59 -0500
From: "E.D.G." <edgrsprj@ix.netcom.com>
Subject: Re: Plot Module Question
Message-Id: <v9KdnVKIvPMiyf_RnZ2dnUVZ_hSdnZ2d@earthlink.com>
"Scott Bryce" <sbryce@scottbryce.com> wrote in message
news:i3taap$4gt$1@news.eternal-september.org...
> E.D.G. wrote:
>> Why use an ocean liner to do what should be possible with a rowboat?
>
> Because at some point someone is going to want to row across the ocean.
One possible approach is to select one of the available Perl language
modules for doing simple plotting for faster applications such as
interactive plots where the user wants something to move back and forth on
the computer screen under keyboard control. And, for more complex plotting
applications such as 3D charts Perl could link with the Gnuplot plotting
program.
For quite a while, on my own PC I have been using Perl and Gnuplot
for interactive plotting though it is my understanding that it is not really
designed for that.
It will probably take a while to check the different options.
------------------------------
Date: Thu, 12 Aug 2010 00:54:17 +0800
From: jidanni@jidanni.org
Subject: s/// returns the empty string, but not ""
Message-Id: <i3uocn$bcf$1@news.datemas.de>
I don't think this is returning the empty string ("") actually:
$ man perlop
s/PATTERN/REPLACEMENT/msixpogce
Searches a string for a pattern, and if found, replaces that pattern with the replacement
text and returns the number of substitutions made. Otherwise it returns false (specifically,
the empty string).
$ echo iiii|perl -nwle 'print 1 + s/i//g'
5
$ echo iiii|perl -nwle 'print 1 + s/j//g'
1
$ echo iiii|perl -nwle 'print 1 + ""' #if it really were the empty string, this would happen:
Argument "" isn't numeric in addition (+) at -e line 1, <> line 1.
1
So maybe the man page should be reworded to say just what in fact is
going on.
------------------------------
Date: Wed, 11 Aug 2010 14:11:28 -0400
From: "Uri Guttman" <uri@StemSystems.com>
Subject: Re: s/// returns the empty string, but not ""
Message-Id: <878w4d6mun.fsf@quad.sysarch.com>
>>>>> "j" == jidanni <jidanni@jidanni.org> writes:
j> I don't think this is returning the empty string ("") actually:
j> $ man perlop
j> s/PATTERN/REPLACEMENT/msixpogce
j> Searches a string for a pattern, and if found, replaces that pattern with the replacement
j> text and returns the number of substitutions made. Otherwise it returns false (specifically,
j> the empty string).
j> $ echo iiii|perl -nwle 'print 1 + s/i//g'
why all the work with echo and -n?
perl -wle 'print 1 + "iii" =~ /i/'
2
shows the same issues without all the extra cruft
it is returning the double valued false that other booleans also
return. it is both the empty string AND 0 and you can see that with
various inspection subs like ones in Scalar::Util. so you can use a
boolean result in numeric or string expressions and never get a
warning. it probably says null string because explaining that it really
is both is very wordy and would have to be explained everywhere a
boolean is returned. not enough of a big deal to worry about.
uri
--
Uri Guttman ------ uri@stemsystems.com -------- http://www.sysarch.com --
----- Perl Code Review , Architecture, Development, Training, Support ------
--------- Gourmet Hot Cocoa Mix ---- http://bestfriendscocoa.com ---------
------------------------------
Date: 10 Aug 2010 22:40:11 GMT
From: jt@toerring.de (Jens Thoms Toerring)
Subject: Re: simple indexing in Perl?
Message-Id: <8ce2qbFp9gU1@mid.uni-berlin.de>
Tad McClellan <tadmc@seesig.invalid> wrote:
> Jens Thoms Toerring <jt@toerring.de> wrote:
> > ela <ela@yantai.org> wrote:
> >> print '($listfile, $accfile, $infofile)'; <STDIN>;
> >
> > What's that at end of the line good for?
> Pausing the program until something is typed on STDIN.
Oh, I see. I was a bit confused why to wait for some input
in that situation when one is complaining that the program
is taking so long;-)
Regards, Jens
--
\ Jens Thoms Toerring ___ jt@toerring.de
\__________________________ http://toerring.de
------------------------------
Date: Wed, 11 Aug 2010 10:51:40 +0800
From: "ela" <ela@yantai.org>
Subject: Re: simple indexing in Perl?
Message-Id: <i3t37r$n6u$1@ijustice.itsc.cuhk.edu.hk>
After testing different approaches, Jens Thoms Toerring's works better and
therefore I modified the codes accordingly. Now I just don't know why the
array content cannot be retrieved but only a number "1" is returned. Can
anyone tell me the reason? In fact I can simply pass $line instead of @cells
but what I finally want to achieve is to only print out several cells
instead of all.
my %ahash;
while ( my $line = <$afp> ) {
my @cells = split /\t/, $line;
$ahash{ $cells[ 5 ] } = $cells[ 1 ];
}
close $afp;
open my $ifp, '<', $infofile or die "Can't open $infofile for reading\n";
my %ihash;
while ( my $line = <$ifp> ) {
my @cells = split /\t/, $line;
$ihash{ $cells[ 1 ] } = @cells;
}
close $ifp;
while ( my $line = <$fp> ) {
if ( $line eq "\n" ) {
print $ofp "\n";
next;
}
chomp $line;
if ( $format eq "" ) {
@cells = split /:/, $line;
$tag = $cells[ 0 ];
} else {
@cells = split /\t/, $line;
$tag = $cells[ $acci ];
}
$gid = $ahash{ $tag } if exists $ahash{ $tag };
@gene_info = $ihash{$gid};
print $ofp "$line\t@gene_info";
}
close $fp;
------------------------------
Date: Tue, 10 Aug 2010 19:06:36 -0700
From: Xho Jingleheimerschmidt <xhoster@gmail.com>
Subject: Re: simple indexing in Perl?
Message-Id: <4c6213f1$0$10914$ed362ca5@nr5-q3a.newsreader.com>
ela wrote:
> I'm new to database programming and just previously learnt to use loops to
> look up and enrich information using the following codes. However, when the
> tables are large,
How large?
> I find this process is very slow. Then, somebody told me I
> can build a database for one of the file real time and so no need to read
> the file from the beginning till the end again and again.
Not sure what you mean by "real time" here.
> However, perl DBI
> has a lot of sophisticated functions there and in fact my tables are only
> large but nothing special, linked by an ID.
Data is data. It doesn't need to "something special" in order to put
into a database. Databases themselves are nothing special, just
specialized tools to do a specialized job.
> Is there any simple way to
> achieve the same purpose? I just wish the ID can be indexed and then
> everytime I access the record through memory and not through I/O...
You can read the data into a hash, depending on just how large it is,
and exactly how it needs to be matched.
> open (OFP, ">$outname");
>
> open(FP, $listfile);
You should check that your open commands succeed.
>
> print OFP "$line\tgene info\n";
>
> $nl = '\n';
This is never used, and I don't see what one would use it for.
>
> while (<FP>) {
...
>
> open(AFP, $accfile);
Again, you should check that the open succeeds.
>
> while (<AFP>) {
> @cells = split (/\t/, $_);
> if ($cells[5] =~ /$tag/) {
> $des = $cells[1];
> last;
> }
> }
> close AFP;
This would actually be quite hard to optimize if the match really needs
to be as written, $cells[5] =~ /$tag/. Are you sure it wouldn't still
be correct (or even be more correct) to test $cells[5] eq $tag, or at
least $cells[5] =~ /^\Q$tag/ ?
>
> if ($found == 0) {
> print OFP "$line\tNo gene info available\n";
> }
> }
In your code, $found never gets set to anything, or changed.
Xho
------------------------------
Date: Tue, 10 Aug 2010 22:17:26 -0500
From: Tad McClellan <tadmc@seesig.invalid>
Subject: Re: simple indexing in Perl?
Message-Id: <slrni645c9.vmv.tadmc@tadbox.sbcglobal.net>
ela <ela@yantai.org> wrote:
> Now I just don't know why the
> array content cannot be retrieved but only a number "1" is returned. Can
> anyone tell me the reason?
> open my $ifp, '<', $infofile or die "Can't open $infofile for reading\n";
You should include the reason (in $!)for open's failure in your diag message:
open my $ifp, '<', $infofile or die "Can't open $infofile for reading: $!\n";
^^
^^
> my %ihash;
> while ( my $line = <$ifp> ) {
> my @cells = split /\t/, $line;
> $ihash{ $cells[ 1 ] } = @cells;
@cells is in scalar context there, so it returns the number of
elements in the array. See the "Context" section in:
perldoc perldata
You cannot store an array in a hash value, you need to instead
store a *reference* to an array.
You should read all of:
perldoc perlreftut
Apply "Make Rule 1" from perlreftut:
$ihash{ $cells[ 1 ] } = \@cells;
> if ( $format eq "" ) {
> @cells = split /:/, $line;
> $tag = $cells[ 0 ];
You can do away with the @cells temporary variable by making
use of a List Assignment, replacing the 2 lines above with this line:
($tag) = split /:/, $line;
> $gid = $ahash{ $tag } if exists $ahash{ $tag };
> @gene_info = $ihash{$gid};
To retrieve the cells stored earlier, apply "Use Rule 1"
from perlreftut to dereference the stored reference:
@gene_info = @{ $ihash{$gid} };
--
Tad McClellan
email: perl -le "print scalar reverse qq/moc.liamg\100cm.j.dat/"
The above message is a Usenet post.
I don't recall having given anyone permission to use it on a Web site.
------------------------------
Date: Wed, 11 Aug 2010 11:28:25 -0700
From: sln@netherlands.com
Subject: Re: simple indexing in Perl?
Message-Id: <6gp566pvf2lh5fib7t3nt3p7br3i3n0hhb@4ax.com>
On Wed, 11 Aug 2010 10:51:40 +0800, "ela" <ela@yantai.org> wrote:
>After testing different approaches, Jens Thoms Toerring's works better and
>therefore I modified the codes accordingly. Now I just don't know why the
>array content cannot be retrieved but only a number "1" is returned. Can
>anyone tell me the reason? In fact I can simply pass $line instead of @cells
>but what I finally want to achieve is to only print out several cells
>instead of all.
>
>
>my %ahash;
>while ( my $line = <$afp> ) {
> my @cells = split /\t/, $line;
> $ahash{ $cells[ 5 ] } = $cells[ 1 ];
>}
>close $afp;
>
>open my $ifp, '<', $infofile or die "Can't open $infofile for reading\n";
>
>my %ihash;
>while ( my $line = <$ifp> ) {
> my @cells = split /\t/, $line;
> $ihash{ $cells[ 1 ] } = @cells;
>}
>close $ifp;
>
>while ( my $line = <$fp> ) {
> if ( $line eq "\n" ) {
> print $ofp "\n";
> next;
> }
> chomp $line;
>
> if ( $format eq "" ) {
> @cells = split /:/, $line;
> $tag = $cells[ 0 ];
> } else {
> @cells = split /\t/, $line;
> $tag = $cells[ $acci ];
> }
>
> $gid = $ahash{ $tag } if exists $ahash{ $tag };
> @gene_info = $ihash{$gid};
> print $ofp "$line\t@gene_info";
>}
>
>close $fp;
>
I'm puzzled why you should tackle this in Perl when
I'm guessing this would be a hard SLQ statement for you
to do.
Realizing its a simple sql from 3 tables on a key field
then trying to do it in Perl, etc ..
Your looking for speed, but you can't normalize the task.
You make the big mistake of gathering everything into memory
thereby hogging memory with useless information, then
compounding that error with one time use. Although, I'm not
sure about the one time use, unless its interactive, but
I didn't look to hard for that in the code.
It doesen't appear you have multiple lines per key
gene data, however, that data could be massive.
There is no need to keep all the data in memory.
You could in effect, keep a key => file position
hash via tell(), then retrieve the data later with a
seek.
Applying a pseudo analysis on your content-less code,
it is storing data beyond its use. Its like formal
symbolic logic. Write the equation, then solve it,
its called reverse-engineering.
This is the bottom line equation of your work:
------------------
@Gene-Info Array = @{ I-Hash{ A-Hash{ fp0 } } } if A-Hash{ fp0 } exists
------------------
From inner to outer, when constructing the A-Hash, there is no
need to add a key to the I-Hash if it does not exist in the A-Hash.
If you wrote the sql for this you would have picked this up.
And since the I-Hash contains all the mega gene data, you just
ruptured your memory's brain.
Start over, write pseudo-code, re-check your work via logic analysis
from the inner to outer context. This will save you countless hours
of headache.
-sln
------------------------------
Date: Wed, 11 Aug 2010 00:16:30 +0200
From: Martijn Lievaart <m@rtij.nl.invlalid>
Subject: Re: sysopen failures
Message-Id: <umv8j7-slo.ln1@news.rtij.nl>
On Tue, 10 Aug 2010 00:20:34 -0700, Marc Girod wrote:
> On Aug 9, 6:32 pm, Martijn Lievaart <m...@rtij.nl.invlalid> wrote:
>
>> Stupid suggestion and does not match exactly what you wrote above, but:
>> Are there maybe two instances of your program running and interfering
>> with each other?
>
> Thanks... This script is invoked via a ~/.forward file, i.e. for every
> incoming mail. I guess it is very possible that the same mail is
> presented twice, e.g. if it contains the address twice (?) Again
> something I should log.
> But each instance generates its own filenames, and sure, the first would
> easily clash (protected only by timestamp), but it should impact only
> the first of the two instances...
In that case I suggest you make sure the filenames are really unique
(hint, use the pid, available in $$), or implement some sort of locking
mechanism.
M4
------------------------------
Date: 6 Apr 2001 21:33:47 GMT (Last modified)
From: Perl-Users-Request@ruby.oce.orst.edu (Perl-Users-Digest Admin)
Subject: Digest Administrivia (Last modified: 6 Apr 01)
Message-Id: <null>
Administrivia:
To submit articles to comp.lang.perl.announce, send your article to
clpa@perl.com.
Back issues are available via anonymous ftp from
ftp://cil-www.oce.orst.edu/pub/perl/old-digests.
#For other requests pertaining to the digest, send mail to
#perl-users-request@ruby.oce.orst.edu. Do not waste your time or mine
#sending perl questions to the -request address, I don't have time to
#answer them even if I did know the answer.
------------------------------
End of Perl-Users Digest V11 Issue 3072
***************************************
