[26270] in Perl-Users-Digest
Perl-Users Digest, Issue: 8453 Volume: 10
daemon@ATHENA.MIT.EDU (Perl-Users Digest)
Sun Sep 25 00:05:47 2005
Date: Sat, 24 Sep 2005 21:05:04 -0700 (PDT)
From: Perl-Users Digest <Perl-Users-Request@ruby.OCE.ORST.EDU>
To: Perl-Users@ruby.OCE.ORST.EDU (Perl-Users Digest)
Perl-Users Digest Sat, 24 Sep 2005 Volume: 10 Number: 8453
Today's topics:
Packing/Unpacking bit fields from a byte. <skhanv1@uic.edu>
Re: Packing/Unpacking bit fields from a byte. <see.sig@rochester.rr.com>
Re: Packing/Unpacking bit fields from a byte. <skhanv1@uic.edu>
Re: regex, number of matches <abigail@abigail.nl>
Digest Administrivia (Last modified: 6 Apr 01) (Perl-Users-Digest Admin)
----------------------------------------------------------------------
Date: Sat, 24 Sep 2005 18:46:56 -0500
From: Shashank R Khanvilkar <skhanv1@uic.edu>
Subject: Packing/Unpacking bit fields from a byte.
Message-Id: <Pine.GSO.4.58.0509241840300.11319@icarus.cc.uic.edu>
Hi,
All help appreciated.
I have to parse a file which has a header. This header is say 4 bytes long
and contains many fields which are not byte aligned. For example
struct header {
#fieldname: Size(bits)
fld1: 12
fld2: 1
fld3: 2
...
}
I have already written a lengthy routing to unpack these fileds from the
header (using shifts and masks). However i was wondering if the same could
have been achieved using the pack and unpack functions?
For example:
@hdr = unpack("B12B1B2....", $header);
I find that this does not work.
Does anyone know how this can be done.
To complicate the problem fursther, how can the same be achieved for any
aribitary length header (say 15 bytes).
Thanks
Shashank
------------------------------
Date: Sun, 25 Sep 2005 01:20:02 GMT
From: Bob Walton <see.sig@rochester.rr.com>
Subject: Re: Packing/Unpacking bit fields from a byte.
Message-Id: <6XmZe.2825$Xl2.92@twister.nyroc.rr.com>
Shashank R Khanvilkar wrote:
...
> I have to parse a file which has a header. This header is say 4 bytes long
> and contains many fields which are not byte aligned. For example
>
> struct header {
> #fieldname: Size(bits)
> fld1: 12
> fld2: 1
> fld3: 2
> ...
> }
>
> I have already written a lengthy routing to unpack these fileds from the
> header (using shifts and masks). However i was wondering if the same could
> have been achieved using the pack and unpack functions?
> For example:
>
> @hdr = unpack("B12B1B2....", $header);
>
> I find that this does not work.
Why do you think that doesn't work? It seems to work to me:
C:>perl -MData::Dumper -e "print Dumper unpack 'B12B1B2','abcd'"
$VAR1 = '011000010110';
$VAR2 = '0';
$VAR3 = '01';
C:\Temp>
...
> Shashank
--
Bob Walton
Email: http://bwalton.com/cgi-bin/emailbob.pl
------------------------------
Date: Sat, 24 Sep 2005 23:01:00 -0500
From: Shashank R Khanvilkar <skhanv1@uic.edu>
Subject: Re: Packing/Unpacking bit fields from a byte.
Message-Id: <Pine.GSO.4.58.0509242254540.1460@icarus.cc.uic.edu>
On Sun, 25 Sep 2005, Bob Walton wrote:
> Shashank R Khanvilkar wrote:
>
> ...
> > I have to parse a file which has a header. This header is say 4 bytes long
> > and contains many fields which are not byte aligned. For example
> >
> > struct header {
> > #fieldname: Size(bits)
> > fld1: 12
> > fld2: 1
> > fld3: 2
> > ...
> > }
> >
> > I have already written a lengthy routing to unpack these fileds from the
> > header (using shifts and masks). However i was wondering if the same could
> > have been achieved using the pack and unpack functions?
> > For example:
> >
> > @hdr = unpack("B12B1B2....", $header);
> >
> > I find that this does not work.
>
> Why do you think that doesn't work? It seems to work to me:
>
> C:>perl -MData::Dumper -e "print Dumper unpack 'B12B1B2','abcd'"
> $VAR1 = '011000010110';
> $VAR2 = '0';
> $VAR3 = '01';
>
Thanks.. I guess it is taking the ascii values of a(=61 = 0x01100001)..
Thanks for your help
Shank
> C:\Temp>
> ...
> > Shashank
> --
> Bob Walton
> Email: http://bwalton.com/cgi-bin/emailbob.pl
>
------------------------------
Date: 24 Sep 2005 23:37:45 GMT
From: Abigail <abigail@abigail.nl>
Subject: Re: regex, number of matches
Message-Id: <slrndjboq3.9lk.abigail@alexandra.abigail.nl>
Dr.Ruud (rvtol+news@isolution.nl) wrote on MMMMCDVII September MCMXCIII
in <URL:news:dh3dfb.194.1@news.isolution.nl>:
@@ Abigail schreef:
@@ > Dr.Ruud:
@@
@@ >> There is a problem with the benchmark, because [of what] perlre(1)
@@ says
@@
@@ > I'm fully aware of the penalty associated with $& and friends.
@@ > But why does that cause a problem with the benchmark?
@@
@@ My mistake again. I wrongly read the text about the "price for each
@@ pattern that contains capturing parentheses" as also penalizing other
@@ pattern matches.
@@
@@ I find it hard to think of a reason why the first use of $& should harm
@@ all other pattern matches. And then why ()/$1 doesn't. Because they can
@@ be handled in about the same way. I haven't looked into the Perl-source
@@ yet, this is as good a reason as any.
To be able to capture (part) of the matched string, Perl will have to
do extra work. Basically it needs to copy the string to be matched, and
remember offsets in the string. But what it does isn't important, the
fact that it needs to do extra work is important.
Now, if you use capturing parenthesis, Perl knows for which regexes it
needs to do extra work. Regexes that do not use capturing parenthesis
don't need the bookkeeping overhead.
However, if you use $&, $` or $', Perl doesn't know in advance to which
regex they will refer (you need to be able to solve the halting problem
to determine this). So, Perl will need to do bookkeeping on every regex.
Regardless whether it $& is going to refer to it or not.
@@ I like to see the benchmark without the ()/$1 test, to see if the
@@ remaining cases behave about the same. That might need absolute scores
@@ next to the percentages, so running on a system that doesn't do much
@@ else. I'll try to arrange that later today or tomorrow.
Here's a benchmark:
#!/usr/bin/perl
use strict;
use warnings;
no warnings 'syntax';
use Benchmark qw 'cmpthese';
our @data;
our ($list, $list2, $while, $sub);
BEGIN {@data = split /\n/ => <<'--';
"What happens next?" asked Twoflower.
Hrun screwed a finger in his ear and inspected it absently.
"Oh,", he said, "I expect in a minute the door will be
flung back and I'll be dragged off to some sort of temple
arena where I'll fight maybe a couple of giant spiders
and an eight-foot slave from the jungles of Klatch and then
I'll rescue some kind of a princess from the altar and then
I'll kill off a few guards or whatever and then this girl
will show me the secret passage out of the place and we'll
liberate a couple of horses and escape with the treasure."
Hrun leaned his head back on his hands and looked at the
ceiling, whistling tunelessly.
"All that?" said Twoflower.
"Usually."
--
}
BEGIN {
cmpthese -1 => {
list => '$list = 0; $list += () = /\S+/g for @data;',
list2 => '$list2 = 0; $list2 += (my @t = /\S+/g) for @data;',
while => '$while = 0; do {$while ++ while /\S+/g} for @data;',
sub => '$sub = 0; $sub += s/(\S+)/$1/g for @data;',
};
die "\$list = $list; \$list2 = $list2; ",
"\$while = $while; \$sub = $sub\n"
unless $list == $list2 && $list == $while && $list == $sub
}
my $foo = $&;
cmpthese -1 => {
list => '$list = 0; $list += () = /\S+/g for @data;',
list2 => '$list2 = 0; $list2 += (my @t = /\S+/g) for @data;',
while => '$while = 0; do {$while ++ while /\S+/g} for @data;',
sub => '$sub = 0; $sub += s/(\S+)/$1/g for @data;',
};
die "\$list = $list; \$list2 = $list2; ",
"\$while = $while; \$sub = $sub\n"
unless $list == $list2 && $list == $while && $list == $sub
__END__
Note that by doing the first 'cmpthese' in a BEGIN statement, perl will
run its regexes without doing the extra bookkeeping - it doesn't know about
$& yet. The second set however is run after seeing a $&. Here are the results:
Rate sub list2 list while
sub 2619/s -- -23% -26% -61%
list2 3398/s 30% -- -4% -49%
list 3523/s 34% 4% -- -47%
while 6698/s 156% 97% 90% --
Rate sub list2 list while
sub 2559/s -- -23% -28% -44%
list2 3339/s 30% -- -6% -28%
list 3556/s 39% 6% -- -23%
while 4610/s 80% 38% 30% --
Note the difference in iteration rate of the while - it dropped almost
by a third.
Note also that the 'sub', 'list' and 'list2' regexes run at almost the
same rate - this is because they use parenthesis (the list ones have
implicate parens because the /\S+/g is in list context, and doesn't have
parens itself). But the 'while /\S+/g' doesn't have parens, and is
penalized because of the $&.
Abigail
--
A perl rose: perl -e '@}>-`-,-`-%-'
------------------------------
Date: 6 Apr 2001 21:33:47 GMT (Last modified)
From: Perl-Users-Request@ruby.oce.orst.edu (Perl-Users-Digest Admin)
Subject: Digest Administrivia (Last modified: 6 Apr 01)
Message-Id: <null>
Administrivia:
#The Perl-Users Digest is a retransmission of the USENET newsgroup
#comp.lang.perl.misc. For subscription or unsubscription requests, send
#the single line:
#
# subscribe perl-users
#or:
# unsubscribe perl-users
#
#to almanac@ruby.oce.orst.edu.
NOTE: due to the current flood of worm email banging on ruby, the smtp
server on ruby has been shut off until further notice.
To submit articles to comp.lang.perl.announce, send your article to
clpa@perl.com.
#To request back copies (available for a week or so), send your request
#to almanac@ruby.oce.orst.edu with the command "send perl-users x.y",
#where x is the volume number and y is the issue number.
#For other requests pertaining to the digest, send mail to
#perl-users-request@ruby.oce.orst.edu. Do not waste your time or mine
#sending perl questions to the -request address, I don't have time to
#answer them even if I did know the answer.
------------------------------
End of Perl-Users Digest V10 Issue 8453
***************************************
