[27102] in Perl-Users-Digest
Perl-Users Digest, Issue: 8983 Volume: 10
daemon@ATHENA.MIT.EDU (Perl-Users Digest)
Wed Feb 22 14:05:37 2006
Date: Wed, 22 Feb 2006 11:05:04 -0800 (PST)
From: Perl-Users Digest <Perl-Users-Request@ruby.OCE.ORST.EDU>
To: Perl-Users@ruby.OCE.ORST.EDU (Perl-Users Digest)
Perl-Users Digest Wed, 22 Feb 2006 Volume: 10 Number: 8983
Today's topics:
Questions about globs (was: XS variable creation) <bol@adv.magwien.gv.at>
Re: Reading Mac / Unix / DOS text files <tadmc@augustmail.com>
Re: Reading Mac / Unix / DOS text files <january.weiner@gmail.com>
Re: Reading Mac / Unix / DOS text files <1usa@llenroc.ude.invalid>
Digest Administrivia (Last modified: 6 Apr 01) (Perl-Users-Digest Admin)
----------------------------------------------------------------------
Date: Wed, 22 Feb 2006 17:57:43 +0100
From: "Ferry Bolhar" <bol@adv.magwien.gv.at>
Subject: Questions about globs (was: XS variable creation)
Message-Id: <1140627466.946633@proxy.dienste.wien.at>
Hi,
Ilya Zakharevich wrote:
> sub Perl_wrapper (args) {
> *MyModule::Values::This = XS_Workhorse(args);
> }
>
> Make XS_Workhorse() return a reference to the created value.
>
> RETVAL = newRV_noinc(RETVAL); /* sp? */
This brings some questions about globs in my mind, so I changed
the title of the thread. And before someone ask: Yes, I have read
perlsub, perlref, perlxs, perlxstut, perlguts and perlapi, among others.
They describe more or less clearfully what someone can do with a
glob, but they do not describe what globs actually _are_ and how
they are represented interally.
I know about globs that they are data structures used in conjuction
with symbol table hashes ('stashes'). So, when executing
'print $a';
to obtain the SV of 'a', perl looks in the '%main::' stash for the value
of key 'a', which is a gv. Then it looks into the gv at offset xgv_gp
to find the gp. Finally, it looks into the gp at offset gp_sv to locate
the address of 'a's sv. Right?
So I thought that a glob is used to access all values (sv, av, hv, cv,..)
for a given stash entry. And I know that globs are used by the
Exporter module for aliasing.
But it seems that there is more behind the scenes, since globs
can be used as lvalues and rvalues in normal Perl statements as well
using the '*' character:
*a = \$b; Assign a reference of 'b' to glob 'a'
*b = 4; Assign the value 4 to the glob
*c = sub {...}; Assign an (anonymous) subroutine to the glob
$d = *a; Assign glob 'a' (whatever this is) to scalar 'd'
What exactly happens here? Which data structures are set up here
in which ways?
Is '*b = 4' the same like '$b = 4'? It doesn't seem so. But what else?
Is a glob just a special value type which can be assigned to a scalar,
just like a reference?
And what exactly is the difference between
*XX::aa = *YY::bb; and
$XX::{aa} = $YY::{bb};
BTW: while playing a little bit with Devel::Peek, I saw that the
gp_sv entry in a gp is always used, even when no scalar is assigned
to the glob. For example, with
@a = qw (1 2 3);
Dump *a;
I can see that an address is shown in the GV as well in the AV
entry of the gp. What is stored in the SV entry in this case?
When dumping an array or a hash with Dump, I have it to preceede
with a '\' because Dump accepts scalars only. However, Dump still
accepts glob values without a '\' (see above). Does this mean that *a
is a sv? And if so, what it can be used for?
And finally: is a glob always magic? What are a glob's magics?
(In other words, what do the 'get' and 'set' entries in PL_vtbl_glob)?
Hope I do not attempt to look too deeply...
Many thanks for all answers, and kind greetings from Vienna,
Ferry
--
Ing. Ferry Bolhar
Municipality of Vienna, Department 14
A-1010 Vienna / AUSTRIA
E-mail: bol@adv.magwien.gv.at
------------------------------
Date: Wed, 22 Feb 2006 08:17:46 -0600
From: Tad McClellan <tadmc@augustmail.com>
Subject: Re: Reading Mac / Unix / DOS text files
Message-Id: <slrndvoska.7mi.tadmc@magna.augustmail.com>
January Weiner <january.weiner@gmail.com> wrote:
> I'd like my script to treat text files coming from various systems alike.
> More specifically, I'd like to recognize ends of line as one of: \r, \l,
> \r\l.
In Perl, \l lowercases the following character.
I think you must have meant \n instead?
Furthermore, sometimes \n means CR rather than LF.
We better get our terminology precise if we are to avoid
confusing ourselves.
I will use "carriage return" (CR) and "linefeed" (LF) to
avoid further confusion.
> Is there a more elegant way than doing the obvious?:
The obvious will not work, so I wouldn't characterize it as "obvious".
You need a "correct way" before exploring for a "more elegant way".
> while(<IF>) {
Too late.
At this point, you have *already done* an operation that depends
on the definition of line-ending.
If the file is Mac-style and the program is running on *nix,
then the loop executes 1 time, and the entire file will be
in $_ already...
> s/\r?\l?$// ; # is this correct anyway? will an end of line be
> # recognized with a Mac file?
... so this will delete the final CR but leave all the rest untouched.
> I would expect that there is some weird variable out there (like the $/)
> that changes the behaviour of chomp to be more promiscous.
You would be disappointed then. :-)
> The problem, of course, is, that this cannot be set platform- or
> scriptwide. One file might contain DOS eols, another one would come from
> Mac.
Then you should "normalize" the data before doing any line-oriented
processing.
In other words, you must treat these "text files" as if they
were "binary" files. That is, use read() or sysread()
rather than readline().
--
Tad McClellan SGML consulting
tadmc@augustmail.com Perl programming
Fort Worth, Texas
------------------------------
Date: Wed, 22 Feb 2006 17:41:13 +0100 (CET)
From: January Weiner <january.weiner@gmail.com>
Subject: Re: Reading Mac / Unix / DOS text files
Message-Id: <dti479$eti$1@sagnix.uni-muenster.de>
A. Sinan Unur <1usa@llenroc.ude.invalid> wrote:
> >> > s/\r?\l?$// ; # is this correct anyway? will an end of line be
> it is easy to get confused (from perldoc perlre):
> \l lowercase next char (think vi)
> That is, \l is not linefeed.
:-))) nice demonstration of the problem. But this
s/(?:\r\n?|\n)/
should work correctly? (except for the fact that one should use the codes)
> In any case, these escapes could potentially mean different things on
> different systems. Why not be very specific in what you really are
> looking for?
Hmmmm, I assumed that I should rather use what Perl thinks is a linefeed
than the ASCII code I think it is. But this is really a minor issue.
> > I think that I am rather that stupid, because I did go through both,
> > FAQ and a dozen hits Google returned
> http://www.google.com/search?q=perl+eol
> http://www.google.com/search?q=newline
OK. However, I was not looking for a solution with string substitution, as
you have seen (demonstrated on my faulty code snippet) I came up with that
one myself. I was rather thinking along the following lines: isn't there a
general way to tell Perl "Hey, treat all the text files alike, wherever
they come from: DOS, Mac or Unix".
The point is: (i) I have written a handfull of various scripts, some of them
quite large. All of them work on text files. Recently I have discovered
problems due to the fact that some of the files that I work on recently
come from the DOS world. Now, I'd rather insert _one_ command or variable
assignment somewhere at the beginning of the script that would change the
behaviour of chomp than to go through all that code and substitute each
chomp by a substitution. (ii) A substitution takes more time by orders of
magnitude:
:~ $ head -100000 /db/prodom/prodom.mul | (time perl -p -e 'chomp ;' > /dev/null ; )
real 0m0.157s
user 0m0.123s
sys 0m0.034s
:~ $ head -100000 /db/prodom/prodom.mul | (time perl -p -e 's{ \012 | (?: \015\012? ) }{\n}x ;' > /dev/null ; )
real 0m2.012s
user 0m1.990s
sys 0m0.024s
And, surprise, the files can be quite large:
:~ $ wc -l /db/prodom/prodom.mul
7900570 /db/prodom/prodom.mul
I simply thought there might be a better solution than to use
substitutions, like assigning $/ in a special way or using a module that
adds a layer to the file open() or redefines chomp. What do I know. I
thought that the problem was common enough to be addressed in a better way.
I think that I will find some way to determine the file type (possibly by
looking at the ending of the first line), redefine $/ and continue reading.
Some untested code follows:
#!/usr/bin/perl -w
use strict;
use warnings;
my $DFNTNTF = myopen("<test.mul") ;
die "Cannot open file: $!\n" unless($DFNTNTF) ;
while( <$DFNTNTF> ) {
chomp ;
print "line $.:$_\n" ;
}
close $DFNTNTF ;
exit 0 ;
# open a file and set the input record separator
sub myopen {
my $file_mode = shift ;
my $definitelynotif ;
open ( $definitelynotif, $file_mode ) or return ;
my $line = <$definitelynotif> ;
if($line =~ m/(\015\012|\012|\015)/) {
$/ = $1 ;
}
close $definitelynotif ;
open ( $definitelynotif, $file_mode ) or return ;
return $definitelynotif ;
}
> In any case, I should probably have put a smiley there, because I had not
> intended it to come across that harshly.
No offence taken.
Cheers,
January
--
------------------------------
Date: Wed, 22 Feb 2006 17:12:02 +0000 (UTC)
From: "A. Sinan Unur" <1usa@llenroc.ude.invalid>
Subject: Re: Reading Mac / Unix / DOS text files
Message-Id: <Xns97727C1D45B93asu1cornelledu@132.236.56.8>
January Weiner <january.weiner@gmail.com> wrote in
news:dti479$eti$1@sagnix.uni-muenster.de:
> A. Sinan Unur <1usa@llenroc.ude.invalid> wrote:
>> >> > s/\r?\l?$// ; # is this correct anyway? will an end of line
>> >> > be
>
>> it is easy to get confused (from perldoc perlre):
>
>> \l lowercase next char (think vi)
>
>> That is, \l is not linefeed.
>
>:-))) nice demonstration of the problem. But this
>
> s/(?:\r\n?|\n)/
>
> should work correctly?
Have you tried that on a DOS file on Unix? Take a look at it in a hex
editor.
> OK. However, I was not looking for a solution with string
> substitution, as you have seen (demonstrated on my faulty code
> snippet) I came up with that one myself. I was rather thinking along
> the following lines: isn't there a general way to tell Perl "Hey,
> treat all the text files alike, wherever they come from: DOS, Mac or
> Unix".
Open in binmode, don't use \n to match eol.
Sinan
------------------------------
Date: 6 Apr 2001 21:33:47 GMT (Last modified)
From: Perl-Users-Request@ruby.oce.orst.edu (Perl-Users-Digest Admin)
Subject: Digest Administrivia (Last modified: 6 Apr 01)
Message-Id: <null>
Administrivia:
#The Perl-Users Digest is a retransmission of the USENET newsgroup
#comp.lang.perl.misc. For subscription or unsubscription requests, send
#the single line:
#
# subscribe perl-users
#or:
# unsubscribe perl-users
#
#to almanac@ruby.oce.orst.edu.
NOTE: due to the current flood of worm email banging on ruby, the smtp
server on ruby has been shut off until further notice.
To submit articles to comp.lang.perl.announce, send your article to
clpa@perl.com.
#To request back copies (available for a week or so), send your request
#to almanac@ruby.oce.orst.edu with the command "send perl-users x.y",
#where x is the volume number and y is the issue number.
#For other requests pertaining to the digest, send mail to
#perl-users-request@ruby.oce.orst.edu. Do not waste your time or mine
#sending perl questions to the -request address, I don't have time to
#answer them even if I did know the answer.
------------------------------
End of Perl-Users Digest V10 Issue 8983
***************************************
