[31314] in Perl-Users-Digest
Perl-Users Digest, Issue: 2559 Volume: 11
daemon@ATHENA.MIT.EDU (Perl-Users Digest)
Tue Aug 18 18:09:45 2009
Date: Tue, 18 Aug 2009 15:09:10 -0700 (PDT)
From: Perl-Users Digest <Perl-Users-Request@ruby.OCE.ORST.EDU>
To: Perl-Users@ruby.OCE.ORST.EDU (Perl-Users Digest)
Perl-Users Digest Tue, 18 Aug 2009 Volume: 11 Number: 2559
Today's topics:
Re: end-of-line conventions <ben@morrow.me.uk>
Re: growing hashes of arrays <netnews@invalid.com>
Re: revisiting web development in Perl...where to start <cartercc@gmail.com>
Re: revisiting web development in Perl...where to start (Jamie)
Re: Rounding issue <lehmannmapson@cnm.de>
Re: Rounding issue (Jens Thoms Toerring)
Re: Rounding issue <lehmannmapson@cnm.de>
Re: Rounding issue sln@netherlands.com
Re: Rounding issue <willem@stack.nl>
Re: Rounding issue (Doug Miller)
Re: Rounding issue sln@netherlands.com
Re: Rounding issue <jurgenex@hotmail.com>
Re: Rounding issue <m@rtij.nl.invlalid>
Re: Rounding issue <willem@stack.nl>
Re: Rounding issue (Alan Curry)
search for hex characters in a binary file and remove t <venkateshwar1981@gmail.com>
Re: search for hex characters in a binary file and remo sln@netherlands.com
Re: search for hex characters in a binary file and remo sln@netherlands.com
Re: search for hex characters in a binary file and remo <venkateshwar1981@gmail.com>
Re: search for hex characters in a binary file and remo <bugbear@trim_papermule.co.uk_trim>
Re: search for hex characters in a binary file and remo sln@netherlands.com
Re: search for hex characters in a binary file and remo sln@netherlands.com
Re: search for hex characters in a binary file and remo sln@netherlands.com
Re: search for hex characters in a binary file and remo sln@netherlands.com
Digest Administrivia (Last modified: 6 Apr 01) (Perl-Users-Digest Admin)
----------------------------------------------------------------------
Date: Tue, 18 Aug 2009 21:48:35 +0100
From: Ben Morrow <ben@morrow.me.uk>
Subject: Re: end-of-line conventions
Message-Id: <3mgrl6-imp2.ln1@osiris.mauzo.dyndns.org>
Quoth Shmuel (Seymour J.) Metz <spamtrap@library.lspace.org.invalid>:
> In <6vrll6-112.ln1@osiris.mauzo.dyndns.org>, on 08/16/2009
> at 06:24 PM, Ben Morrow <ben@morrow.me.uk> said:
>
> >No it's not, at least not as far as Perl is concerned.
>
> Perldoc says otherwise. so does the code running on my machine.
Where in perldoc?
> >If you have a file containing
> > fooCRLF
> >and you read a line with
> > open my $FOO, "<", "foo";
> > my $foo = <$FOO>;
> >then $foo will be four bytes long, not five.
>
> Perhaps on youir machine. On mine the CR and LF are both there.
OK, now I'm really confused, since I didn't think perl worked like that
anywhere. What do you get from
perl -le"print for unpack "C*", qq{\n}"
What do you get if you open a file in the normal way and print
"a\012b\n" to it?
> >Perl is no longer maintained for Mac OS Classic or the EBCDIC platforms.
>
> There are still people working on the EBCDIC code.
Really? AFAIR last time it was brought up on p5p the conclusion was that
nobody was testing it or (apparently) using it, so while none of it
would be broken on purpose no particular effort would be made to keep it
working. I would be pleased to find out I'm wrong about that :).
Ben
------------------------------
Date: Tue, 18 Aug 2009 11:55:09 -0700
From: HASM <netnews@invalid.com>
Subject: Re: growing hashes of arrays
Message-Id: <m3tz05c5pu.fsf@127.0.0.1>
Martijn Lievaart <m@rtij.nl.invlalid> writes:
> Just "push (@{$hash{$key}}, $value);"
Thanks Christian and Martijn. Interestingly I would have bet that was the
code I initially had and it didn't seem to work. No point in replicating
it though ...
-- HASM
------------------------------
Date: Tue, 18 Aug 2009 07:37:01 -0700 (PDT)
From: ccc31807 <cartercc@gmail.com>
Subject: Re: revisiting web development in Perl...where to start?
Message-Id: <4b48baf8-36a7-49fd-b891-f7df5765cd09@k6g2000yqn.googlegroups.com>
On Aug 17, 12:33=A0pm, "Peter J. Holzer" <hjp-usen...@hjp.at> wrote:
> If you use mod_perl to execute your scripts, they are *not* standard CGI
> scripts. The CGI specification mandates that each invocation of a CGI
> script creates a new process which uses the environment, stdin and
> stdout to communicate with the server. mod_perl doesn't do this.
It's interesting to note that a number of books advocating different
technologies often have a section near the front as to why their
technology is superior to the others. Most of these sections feature
Perl as one of the 'big' rivals, and most slam Perl by accusing it of
using CGI and therefore being slow and creating a lot of unnecessary
overhead.
Several years ago, I had a conversation with a rather large employer
about web technologies. They were using mostly JSP, migrating to .NET.
Now, I did some JSP years ago, and build a couple of sites with .NET
(C# and VB, not ASP), and my complaint was that these were slow,
particularly on the first invocation. I asked the person I talked to
about using Perl, and he said (and this is almost an exact quote),
"Perl is too slow, JSP is much faster." When I asked for the reason
for Perl's slowness, he said 'CGI.' I asked him if he used mod_perl,
and he said, and I quote, "What's mod_perl?"
You are absolutely right about the original meaning of CGI, but these
days many people use CGI to refer to a style of coding, using Perl,
Python, Ruby, and maybe some other languages. I'm guilty of this
myself, since I describe what I do as Perl/CGI, even though the last
time I actually wrote a real CGI script was about 2001.
As I pointed out in a previous post, the web development field is very
crowded with no one technology standing out from the crowd. I continue
to believe that Perl is an exceptionally strong competitor, and over
the years I've developed a strong distaste for embedded technologies
(PHP, CF, JSP, ASP, and the like).
I've played some with Erlang and found it much to my liking. Wrox has
a book coming out this year entitled 'Erlang Web Application
Development' by Nick Gerakines, and I suspect that some Erlang CGI
scripts will be in my future. Erlang probably is the closest thing to
real CGI that we have had in a long time, but because the processes
live in the language layer rather than in the OS layer, it's extremely
quick. Maybe we can say in the future that we should use CGI because
it is much faster and has much less overhead ... who knows?
CC
------------------------------
Date: Tue, 18 Aug 2009 20:30:00 GMT
From: nospam@geniegate.com (Jamie)
Subject: Re: revisiting web development in Perl...where to start?
Message-Id: <Lc1250624329159370x183d7b0@saturn>
In <4b48baf8-36a7-49fd-b891-f7df5765cd09@k6g2000yqn.googlegroups.com>,
ccc31807 <cartercc@gmail.com> mentions:
>On Aug 17, 12:33=A0pm, "Peter J. Holzer" <hjp-usen...@hjp.at> wrote:
>> If you use mod_perl to execute your scripts, they are *not* standard CGI
>> scripts. The CGI specification mandates that each invocation of a CGI
>> script creates a new process which uses the environment, stdin and
>> stdout to communicate with the server. mod_perl doesn't do this.
>
>It's interesting to note that a number of books advocating different
>technologies often have a section near the front as to why their
>technology is superior to the others. Most of these sections feature
>Perl as one of the 'big' rivals, and most slam Perl by accusing it of
>using CGI and therefore being slow and creating a lot of unnecessary
>overhead.
I usually see it slammed because it's older and therefore, inferior.
If you're using mod_perl, I highly recommend checking into FastCGI, for
90% of the things a web app is going to do, FastCGI handles it better
than mod_perl (IMO), because it lives outside the web server as a separate
daemon process, you can keep apache lean & trim plus, it's much easier
to work with. (I found it to be more elegant anyway)
mod_perl is still better for writing apache-centric stuff though.
>Several years ago, I had a conversation with a rather large employer
>about web technologies. They were using mostly JSP, migrating to .NET.
>Now, I did some JSP years ago, and build a couple of sites with .NET
>(C# and VB, not ASP), and my complaint was that these were slow,
>particularly on the first invocation. I asked the person I talked to
JSP! I actually really liked jsp (actually, servlets), it's just too..
convoluted to be practical, it's like they created something great and then,
rather than stop there, kept going until it's a mess. (But then, thats java
for you)
Java is well worth messing around with, if you can find an excuse to use
it for something, because of the ideas it contains.
>You are absolutely right about the original meaning of CGI, but these
>days many people use CGI to refer to a style of coding, using Perl,
>Python, Ruby, and maybe some other languages. I'm guilty of this
>myself, since I describe what I do as Perl/CGI, even though the last
>time I actually wrote a real CGI script was about 2001.
I always thought of CGI as a kind of protocol, didn't have to be a new
process, just the way the web server communicated with another process
using stdio and environment variables. (never realized it /had/ to be a
new process)
>As I pointed out in a previous post, the web development field is very
>crowded with no one technology standing out from the crowd. I continue
>to believe that Perl is an exceptionally strong competitor, and over
>the years I've developed a strong distaste for embedded technologies
>(PHP, CF, JSP, ASP, and the like).
I deal with PHP quite a bit, it's an awesome template system, but a
horrible programming language.
The criteria for selecting a technology is strictly popularity,
the underlying technology is irrelevant.
Popularity and group mentality are strong motivators, if you look at a
mob for example, the individual people would NEVER toss a brick through
someones window, BUT.. get people in a group and suddenly they start
doing foolish things, it actually seems rational to smash windows.
It's a kind of snowball effect. A casual google for one of these PHP scripts (a
popular content manager) yields 284,000,000 hits, while the keyword "perl"
yields only 65,000,000
I suspect most of the php coders out there have PHP as their first "language" and
aren't programmers so much as web designers, they figure out how to do a couple
things with it and walla, they jump into writing full-fledged php applications
that aren't planned out.
Because these people are web designers first, programmers second, the scripts
they churn out look attractive to people.
I'd sure like to find a way to turn the tide, but 284 million hits can't
be wrong, can they? :-)
Jamie
--
http://www.geniegate.com Custom web programming
Perl * Java * UNIX User Management Solutions
------------------------------
Date: Tue, 18 Aug 2009 15:31:17 +0200
From: Marten Lehmann <lehmannmapson@cnm.de>
Subject: Re: Rounding issue
Message-Id: <7evop6F2i683nU1@mid.individual.net>
Hello,
maybe I should change the question to:
How can I force Perl to process certain calculations using decimals?
There are special datatypes in Python and databases. Is there any in Perl?
Ben proposed using the bignums pragma, but I'm afraid that this breaks
something else in the code.
Kind regards
Marten
------------------------------
Date: 18 Aug 2009 16:27:19 GMT
From: jt@toerring.de (Jens Thoms Toerring)
Subject: Re: Rounding issue
Message-Id: <7f0337F2hi0g8U1@mid.uni-berlin.de>
Marten Lehmann <lehmannmapson@cnm.de> wrote:
> maybe I should change the question to:
> How can I force Perl to process certain calculations using decimals?
> There are special datatypes in Python and databases. Is there any in Perl?
If you use just ints you should be fine since there aren't any
rounding errors. There's no built-in BCD type or similar. But
there seems to be rather new module
http://search.cpan.org/~zefram/Math-Decimal-0.001/lib/Math/Decimal.pm
for doing decimal arithmetic. Or have a look at
http://search.cpan.org/~tels/Math-BigRat-0.22/lib/Math/BigRat.pm
if you want as much precision as the amount of memory in your
machine allows (note: I didn't use any of these modules, so
I can't tell how well they work).
> Ben proposed using the bignums pragma, but I'm afraid that this breaks
> something else in the code.
That you will need if the numbers you use ints and end up with
are too large to be stored in a simple int.
But what will do for you (if it's possible at all) depends on
what exactly you want to do. Until now all we have seen is a
few one-liners, what you need it for is still unclear.
Regards, Jens
--
\ Jens Thoms Toerring ___ jt@toerring.de
\__________________________ http://toerring.de
------------------------------
Date: Tue, 18 Aug 2009 20:04:37 +0200
From: Marten Lehmann <lehmannmapson@cnm.de>
Subject: Re: Rounding issue
Message-Id: <7f08pmF2fjc6rU1@mid.individual.net>
Hello,
> But what will do for you (if it's possible at all) depends on
> what exactly you want to do.
I want to do financial calculations (nothing special, just simple things
like adding items of an invoice, adding taxes, removing disagio from
creditcard transactions etc.) and I want to be sure of correct results.
So if a total with tax is $0.015, then I need it to be rounded to $0.02
and not $0.01. Therefor, numbers need to be represented as they are and
not the typical internal representation. I don't want to use cents as
integers. Even cents will get odd numbers after the decimal point if you
have to add e.g. 19% value added taxes. And even worser: The whole code
would get bloated and the original intention of the code would be harder
to understand if the whole source is full of "* 100" and "/ 100". In
databases it is easy to define a decimal like (10,2): 10 digits before
the point, 2 after it. It would be great to have this directly in Perl
so that sprintf and other functions can still work with such values and
variables, not only specific modules like Math::Decimal.
Regards
Marten
------------------------------
Date: Tue, 18 Aug 2009 11:36:40 -0700
From: sln@netherlands.com
Subject: Re: Rounding issue
Message-Id: <ojsl85l2al6mqtectu3310kdosf9oes1ln@4ax.com>
On Tue, 18 Aug 2009 20:04:37 +0200, Marten Lehmann <lehmannmapson@cnm.de> wrote:
>Hello,
>
>> But what will do for you (if it's possible at all) depends on
>> what exactly you want to do.
>
>I want to do financial calculations (nothing special, just simple things
>like adding items of an invoice, adding taxes, removing disagio from
>creditcard transactions etc.) and I want to be sure of correct results.
>
>So if a total with tax is $0.015, then I need it to be rounded to $0.02
>and not $0.01. Therefor, numbers need to be represented as they are and
>not the typical internal representation. I don't want to use cents as
>integers. Even cents will get odd numbers after the decimal point if you
>have to add e.g. 19% value added taxes. And even worser: The whole code
>would get bloated and the original intention of the code would be harder
>to understand if the whole source is full of "* 100" and "/ 100". In
>databases it is easy to define a decimal like (10,2): 10 digits before
>the point, 2 after it. It would be great to have this directly in Perl
>so that sprintf and other functions can still work with such values and
>variables, not only specific modules like Math::Decimal.
>
>Regards
>Marten
I don't understand your big concern. Anytime you see numbers, decimal
points, or anything printed in a view from a computer is a rounded
representation of what the internal variable contains.
Its just a view, its a one way snapshot of the contents of the variable.
That doesen't affect the internal calculations. Indeed for percentage
calculations, there is absolutely nothing wrong with floating point
at all.
When you have to display (have a view) where you think its necessary
to show a correct image of the results of percentages as fractions of
dollars (cents) then use a rounding translation that everybody and thier
uncle uses.
Why is it so hard?
sub getFmtRound
{
my ($val,$width) = @_;
my $fmt = "%.$width"."f";
return sprintf $fmt,(($val * 10**($width+1))+.5)/(10**($width+1));
}
-sln
------------------------------
Date: Tue, 18 Aug 2009 19:30:33 +0000 (UTC)
From: Willem <willem@stack.nl>
Subject: Re: Rounding issue
Message-Id: <slrnh8m0ap.t1l.willem@turtle.stack.nl>
sln@netherlands.com wrote:
) I don't understand your big concern. Anytime you see numbers, decimal
) points, or anything printed in a view from a computer is a rounded
) representation of what the internal variable contains.
No it isn't. It can also be an exact representation. For example,
if the numbers are integers or some kind of decimal format.
) Its just a view, its a one way snapshot of the contents of the variable.
) That doesen't affect the internal calculations. Indeed for percentage
) calculations, there is absolutely nothing wrong with floating point
) at all.
Yes there is. Percentage calculations are fixed point, and to get exact
results, you need to be doing them in an exact representation such as
integers or rationals.
) When you have to display (have a view) where you think its necessary
) to show a correct image of the results of percentages as fractions of
) dollars (cents) then use a rounding translation that everybody and thier
) uncle uses.
If you use floating point internally, then the display result will not
always be exact.
) Why is it so hard?
Because people think too easy of it. Such as you.
SaSW, Willem
--
Disclaimer: I am in no way responsible for any of the statements
made in the above text. For all I know I might be
drugged or something..
No I'm not paranoid. You all think I'm paranoid, don't you !
#EOT
------------------------------
Date: Wed, 19 Aug 2009 07:37:49 GMT
From: spambait@milmac.com (Doug Miller)
Subject: Re: Rounding issue
Message-Id: <h6evpt$74h$1@news.eternal-september.org>
In article <7f08pmF2fjc6rU1@mid.individual.net>, Marten Lehmann <lehmannmapson@cnm.de> wrote:
>I don't want to use cents as integers.
Then you have to live with rounding errors.
------------------------------
Date: Tue, 18 Aug 2009 13:12:29 -0700
From: sln@netherlands.com
Subject: Re: Rounding issue
Message-Id: <de1m8593pt76i517s51n89b4ctno42v144@4ax.com>
On Tue, 18 Aug 2009 19:30:33 +0000 (UTC), Willem <willem@stack.nl> wrote:
>sln@netherlands.com wrote:
>) I don't understand your big concern. Anytime you see numbers, decimal
>) points, or anything printed in a view from a computer is a rounded
>) representation of what the internal variable contains.
>
>No it isn't. It can also be an exact representation. For example,
>if the numbers are integers or some kind of decimal format.
Makes sense. Its either exact or rounded for display only. Either way,
the internal representation produces the same result as exact, to the
viewer.
>
>) Its just a view, its a one way snapshot of the contents of the variable.
>) That doesen't affect the internal calculations. Indeed for percentage
>) calculations, there is absolutely nothing wrong with floating point
>) at all.
>
>Yes there is. Percentage calculations are fixed point, and to get exact
>results, you need to be doing them in an exact representation such as
>integers or rationals.
Wrong. Percentage calculations are linear, the percentage being the
slope. The result is NOT an integer in a calculator nor computer.
>
>) When you have to display (have a view) where you think its necessary
>) to show a correct image of the results of percentages as fractions of
>) dollars (cents) then use a rounding translation that everybody and thier
>) uncle uses.
>
>If you use floating point internally, then the display result will not
>always be exact.
Wrong. I gave you the formulae to display EXACT representation given the
column you want to round to.
>
>) Why is it so hard?
>
>Because people think too easy of it. Such as you.
>
>
>SaSW, Willem
Its hard to believe that say the OMB has any problems at all using floating
point numbers where formulae and fractions are everything.
Its also amazing that any computational accuracy takes place at all.
I can categorically tell you that .149999999999999000000000 equals .15
The difference can't be measured, tasted, felt or seen. It only exists
in an 80 bit register so far to the right, its meaningless and won't
affect any other calculation at all !!
my $f3 = .149999999999999000000000;
print getFmtRound($f3,8),"\n";
sub getFmtRound
{
my ($val,$width) = @_;
my $fmt = "%.$width"."f";
return sprintf $fmt,(($val * 10**($width+1))+.5)/(10**($width+1));
}
Output:
0.15000000
-sln
------------------------------
Date: Tue, 18 Aug 2009 13:24:27 -0700
From: Jürgen Exner <jurgenex@hotmail.com>
Subject: Re: Rounding issue
Message-Id: <j12m855c4rvjkbqtk39ls6psdgh8mu1l5o@4ax.com>
Marten Lehmann <lehmannmapson@cnm.de> wrote:
>> But what will do for you (if it's possible at all) depends on
>> what exactly you want to do.
>
>I want to do financial calculations (nothing special, just simple things
>like adding items of an invoice, adding taxes, removing disagio from
>creditcard transactions etc.) and I want to be sure of correct results.
Then I _strongly_ suggest to brush up on computer numerics. Writing a
financial application that is sound is anything but trivial. So far you
discovered merely the tip of the iceberg of potential issues.
>So if a total with tax is $0.015, then I need it to be rounded to $0.02
>and not $0.01.
Just one other tiny glimpse into the peculiarities of financial
applications: If you have 5 of those 0.015$ amounts, should their total
be 0.75 or should their total be 1.00?
And there are numerous other pitfalls....
>Therefor, numbers need to be represented as they are and
>not the typical internal representation.
With "as they are" I suppose you mean in the decimal system.
No, Perl does not provide BCD natively (Binary Coded Decimals; someone
else already pointed that out). The reason is simple: they not only take
up more space (probably a minor issue on modern hardware) but more
important most hardware does not support BCD arithmetic, therefore all
calculations would have to be handcoded in software, making them very
slow.
>I don't want to use cents as
>integers. Even cents will get odd numbers after the decimal point if you
>have to add e.g. 19% value added taxes. And even worser: The whole code
>would get bloated and the original intention of the code would be harder
>to understand if the whole source is full of "* 100" and "/ 100". In
I think you still don't recognize the basic issue: what you are asking
for cannot be done. Not because of a limitation in the binary system,
but because of a fundamantal mathematical limitation: if you have a tax
of 1/3 of one dollar, then how do you suggest to represent that value in
your favoured decimal system. You can not as we all know from 3rd grade
math. Therefore you have to deal with those rounding imperfections, no
matter which system (binary, decimal, ...) you are using. Just saying "I
want to use decimal instead of binary" doesn't absolve you from that
requirement.
How to deal with them is being tought in Computer Numerics, and that's
why I suggest several times already to brush up on that.
>databases it is easy to define a decimal like (10,2): 10 digits before
>the point, 2 after it.
And you will still run into the same fundamental limitations.
>It would be great to have this directly in Perl
>so that sprintf and other functions can still work with such values and
>variables, not only specific modules like Math::Decimal.
Well, you got a few pointers to maybe helpful modules. If they don't
suit you needs then you are always free to write your own BCD module.
But as I said, even BCD won't solve your rounding problem, they "bad"
numbers will just be different.
jue
------------------------------
Date: Tue, 18 Aug 2009 22:41:43 +0200
From: Martijn Lievaart <m@rtij.nl.invlalid>
Subject: Re: Rounding issue
Message-Id: <79grl6-7pb.ln1@news.rtij.nl>
On Tue, 18 Aug 2009 20:04:37 +0200, Marten Lehmann wrote:
> Hello,
>
>> But what will do for you (if it's possible at all) depends on what
>> exactly you want to do.
>
> I want to do financial calculations (nothing special, just simple things
> like adding items of an invoice, adding taxes, removing disagio from
> creditcard transactions etc.) and I want to be sure of correct results.
Then you need to define first what the correct result is, which is
actually harder then it looks.
I worked on programs that defined results up to 5 digits after the
decimal point and had exact rounding rules for every step in the
calculations. Yes, these were financial programs and no, you will
probably not encounter them unless you work at an insurance company.
The solution used by 99% of programs I know is to use a suitably large
integer type to represent cents, converting to $, E, whatever on output.
M4
------------------------------
Date: Tue, 18 Aug 2009 21:27:03 +0000 (UTC)
From: Willem <willem@stack.nl>
Subject: Re: Rounding issue
Message-Id: <slrnh8m757.ut8.willem@turtle.stack.nl>
sln@netherlands.com wrote:
) Makes sense. Its either exact or rounded for display only. Either way,
) the internal representation produces the same result as exact, to the
) viewer.
Floating point is rounded during almost any calculation you care
to do with it. That's the whole problem.
) Wrong. Percentage calculations are linear, the percentage being the
) slope. The result is NOT an integer in a calculator nor computer.
Yes they are, as long as you keep track of factors of 10 separately.
That is: <integer> times <n%> is <integer times n> divided by 100.
You just separately (implicitly) keep track of that 'divided by 100' bit.
If you want accuracy to 0.1%, just work with promilles instead, etc.
)>If you use floating point internally, then the display result will not
)>always be exact.
)
) Wrong. I gave you the formulae to display EXACT representation given the
) column you want to round to.
The floating point number *itself* will not be exact. It will have
internal rounding errors because it has limited precision.
) Its hard to believe that say the OMB has any problems at all using floating
) point numbers where formulae and fractions are everything.
I find it very easy to believe. Floating point inherently has rounding
errors. You need a lot of careful analysis to make sure that you always
get the correct result. If that is even possible.
Using fractions, you want exact results ? Use rationals. It's that simple.
) my $f3 = .149999999999999000000000;
) <snip>
) 0.15000000
So you're saying that the OP should round twice ?
Once to 8 digits and then to 2 digits ?
Why 8 ? Why not 5, or 10 ?
How do you know that this magic 8 will always give the correct result ?
Answer: you don't. Use an exact representation.
SaSW, Willem
--
Disclaimer: I am in no way responsible for any of the statements
made in the above text. For all I know I might be
drugged or something..
No I'm not paranoid. You all think I'm paranoid, don't you !
#EOT
------------------------------
Date: Tue, 18 Aug 2009 21:34:32 +0000 (UTC)
From: pacman@kosh.dhis.org (Alan Curry)
Subject: Re: Rounding issue
Message-Id: <h6f6p8$tn5$1@aioe.org>
In article <j12m855c4rvjkbqtk39ls6psdgh8mu1l5o@4ax.com>,
Jürgen Exner <jurgenex@hotmail.com> wrote:
>
>No, Perl does not provide BCD natively (Binary Coded Decimals; someone
>else already pointed that out). The reason is simple: they not only take
>up more space (probably a minor issue on modern hardware) but more
>important most hardware does not support BCD arithmetic, therefore all
>calculations would have to be handcoded in software, making them very
>slow.
Does anyone else find this argument ("We have to use the CPU's floating-point
because anything else is too slow") a bit odd? Did we not once have CPUs with
no floating point at all?[1] It was all "in software", and at about 1MHz too.
Still, somehow, things got done.
I wouldn't be surprised if arbitrary-precision GMP-based Math::BigRat on a
recent machine outperforms the limited-precision software floating-point that
was used in the old days. But still people say "Too slow. Gotta use the FPU
for speed." Really?
Will there ever be a time when our need for speed is satiated, and we can
start thinking about making it easier to get correct results, instead of
bad results quickly? Can a well-rounded, general-purpose language like perl
ever decide that computers are fast enough now, that accuracy-preserving
"bignum" should be the default mode, and FPU speed freaks should be the ones
importing a special module to meet their needs?
If not, then can we at least hope that in the future, Math::BigRat will
play well with sprintf?
[1] I even remember being surprised when I found out the 80x86 could do
integer multiplication in a single instruction. No shift-add-shift-loop
subroutine?! Then I saw "idiv" and was sure someone was joking.
--
Alan Curry
------------------------------
Date: Tue, 18 Aug 2009 07:14:34 -0700 (PDT)
From: venkateshwar D <venkateshwar1981@gmail.com>
Subject: search for hex characters in a binary file and remove them
Message-Id: <51897830-6265-45db-aee3-d0a7d3a7a23c@e27g2000yqm.googlegroups.com>
Hi All,
I need to look for a sequence of hex characters in a binary file and
remove them. the binary file has 00 00 02 02 01 00 sequence somewhere
in the file.
The script should open the file and look for this sequence 00 00 02 02
01 00 <18 variable bytes> and remove the 18 + 6 = 24 bytes from the
file.can someone please help. I can open the binary file and buffer
byte by byte but since the pattern can be anywhere in the file i dont
know how to proceed
regards
venkat
------------------------------
Date: Tue, 18 Aug 2009 07:36:32 -0700
From: sln@netherlands.com
Subject: Re: search for hex characters in a binary file and remove them
Message-Id: <mfel8557918o0661g1ihl6ta0o5unhm09r@4ax.com>
On Tue, 18 Aug 2009 07:14:34 -0700 (PDT), venkateshwar D <venkateshwar1981@gmail.com> wrote:
>Hi All,
>
>I need to look for a sequence of hex characters in a binary file and
>remove them. the binary file has 00 00 02 02 01 00 sequence somewhere
>in the file.
>The script should open the file and look for this sequence 00 00 02 02
>01 00 <18 variable bytes> and remove the 18 + 6 = 24 bytes from the
>file.can someone please help. I can open the binary file and buffer
>byte by byte but since the pattern can be anywhere in the file i dont
>know how to proceed
>
>regards
>venkat
Hex characters? Like [a-f0-9] ? Or integers?
$sequence = " 00 00 02 02 01 00";
open my $fin, '<:raw', 'filename.in' or die "can't open input file: $!";
open my $fout, '>:raw', 'filename.out' or die "can't open output file: $!";
{
local $/;
$buf = <$fin>;
$buf =~ s/$sequence//;
print $fout $buf;
}
close $fout;
close $fin;
-sln
------------------------------
Date: Tue, 18 Aug 2009 07:42:42 -0700
From: sln@netherlands.com
Subject: Re: search for hex characters in a binary file and remove them
Message-Id: <n7fl8512ff4ru206h1evcfp2knsqsu3vog@4ax.com>
On Tue, 18 Aug 2009 07:36:32 -0700, sln@netherlands.com wrote:
>On Tue, 18 Aug 2009 07:14:34 -0700 (PDT), venkateshwar D <venkateshwar1981@gmail.com> wrote:
>
>>Hi All,
>>
>>I need to look for a sequence of hex characters in a binary file and
>>remove them. the binary file has 00 00 02 02 01 00 sequence somewhere
>>in the file.
>>The script should open the file and look for this sequence 00 00 02 02
>>01 00 <18 variable bytes> and remove the 18 + 6 = 24 bytes from the
>>file.can someone please help. I can open the binary file and buffer
>>byte by byte but since the pattern can be anywhere in the file i dont
>>know how to proceed
>>
>>regards
>>venkat
>
>Hex characters? Like [a-f0-9] ? Or integers?
>
>$sequence = " 00 00 02 02 01 00";
>
>open my $fin, '<:raw', 'filename.in' or die "can't open input file: $!";
>open my $fout, '>:raw', 'filename.out' or die "can't open output file: $!";
>{
> local $/;
> $buf = <$fin>;
> $buf =~ s/$sequence//;
^^
$buf =~ s/$sequence.{6}//gs;
The 6 bytes after the sequence as well?
-sln
------------------------------
Date: Tue, 18 Aug 2009 08:34:49 -0700 (PDT)
From: venkateshwar D <venkateshwar1981@gmail.com>
Subject: Re: search for hex characters in a binary file and remove them
Message-Id: <d992f76c-cd60-4342-8769-a1437270790e@j21g2000yqe.googlegroups.com>
On Aug 18, 7:42=A0pm, s...@netherlands.com wrote:
> On Tue, 18 Aug 2009 07:36:32 -0700, s...@netherlands.com wrote:
> >On Tue, 18 Aug 2009 07:14:34 -0700 (PDT), venkateshwar D <venkateshwar1.=
..@gmail.com> wrote:
>
> >>Hi All,
>
> >>I need to look for a sequence of hex characters in a binary file and
> >>remove them. the binary file has 00 00 02 02 01 00 sequence somewhere
> >>in the file.
> >>The script should open the file and look for this sequence 00 00 02 02
> >>01 00 <18 variable bytes> and remove the 18 + 6 =3D 24 bytes from the
> >>file.can someone please help. I can open the binary file and buffer
> >>byte by byte but since the pattern can be anywhere in the file i dont
> >>know how to proceed
>
> >>regards
> >>venkat
>
> >Hex characters? Like [a-f0-9] ? Or integers?
>
> >$sequence =3D " 00 00 02 02 01 00";
>
> >open my $fin, '<:raw', 'filename.in' or die "can't open input file: $!";
> >open my $fout, '>:raw', 'filename.out' or die "can't open output file: $=
!";
> >{
> > =A0 local $/;
> > =A0 $buf =3D <$fin>;
> > =A0 $buf =3D~ s/$sequence//;
>
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 ^^
> =A0 =A0$buf =3D~ s/$sequence.{6}//gs;
>
> The 6 bytes after the sequence as well?
> -sln- Hide quoted text -
>
> - Show quoted text -
Hi
Thanks a lot. This does not seem to be working. it is doing a binary
file copy.
I want to search for that pattern in the binary file (000002020100)
(it is hex character file) and remove this pattern + the next 18 bytes
in the file.
thanks
venkat
------------------------------
Date: Tue, 18 Aug 2009 16:47:08 +0100
From: bugbear <bugbear@trim_papermule.co.uk_trim>
Subject: Re: search for hex characters in a binary file and remove them
Message-Id: <YK6dnTvXuK1hURfXnZ2dnUVZ8ili4p2d@brightview.co.uk>
sln@netherlands.com wrote:
> Hex characters? Like [a-f0-9] ? Or integers?
>
> $sequence = " 00 00 02 02 01 00";
>
> open my $fin, '<:raw', 'filename.in' or die "can't open input file: $!";
> open my $fout, '>:raw', 'filename.out' or die "can't open output file: $!";
> {
> local $/;
> $buf = <$fin>;
> $buf =~ s/$sequence//;
> print $fout $buf;
> }
> close $fout;
> close $fin;
For extra merit, make it work without reading
the whole file into ram at once ;-)
BugBear
------------------------------
Date: Tue, 18 Aug 2009 11:12:27 -0700
From: sln@netherlands.com
Subject: Re: search for hex characters in a binary file and remove them
Message-Id: <fqql8555or8aublagjtjq4uktlbv7ogjv6@4ax.com>
On Tue, 18 Aug 2009 08:34:49 -0700 (PDT), venkateshwar D <venkateshwar1981@gmail.com> wrote:
>On Aug 18, 7:42 pm, s...@netherlands.com wrote:
>> On Tue, 18 Aug 2009 07:36:32 -0700, s...@netherlands.com wrote:
>> >On Tue, 18 Aug 2009 07:14:34 -0700 (PDT), venkateshwar D <venkateshwar1...@gmail.com> wrote:
>>
>>
>> >Hex characters? Like [a-f0-9] ? Or integers?
>>
>> >$sequence = " 00 00 02 02 01 00";
>>
>> >open my $fin, '<:raw', 'filename.in' or die "can't open input file: $!";
>> >open my $fout, '>:raw', 'filename.out' or die "can't open output file: $!";
>>
>> - Show quoted text -
>
>Hi
>
>Thanks a lot. This does not seem to be working. it is doing a binary
>file copy.
>
>I want to search for that pattern in the binary file (000002020100)
>(it is hex character file) and remove this pattern + the next 18 bytes
>in the file.
>thanks
>venkat
I don't understand what you mean. Opening the file in ':raw' mode
takes away any CRLF translations and or possible encoding.
Your free to read it as bytes then.
Surely "000002020100" as text can be represented in a regular expression.
Regular expressions are all about 'text'.
Each character there has an ordinal value that you would consider binary.
If you are instead looking for binary value, 0 value would be \x{0}
character, 2 is \x{2}.
If its text, just look for the sequence + the next 18 bytes:
=~ s/000002020100.{18}//s
After the buffer is modified, write it out to a different ':raw' file,
where no translations will take place.
You can get the same affect in translated mode just make sure the buffer
isin't upgraded to utf8.
-sln
------------------------------
Date: Tue, 18 Aug 2009 12:31:48 -0700
From: sln@netherlands.com
Subject: Re: search for hex characters in a binary file and remove them
Message-Id: <pcvl85pleets74rtea67f1b8o8t7tdhc39@4ax.com>
On Tue, 18 Aug 2009 16:47:08 +0100, bugbear <bugbear@trim_papermule.co.uk_trim> wrote:
>sln@netherlands.com wrote:
>
>> Hex characters? Like [a-f0-9] ? Or integers?
>>
>> $sequence = " 00 00 02 02 01 00";
>>
>> open my $fin, '<:raw', 'filename.in' or die "can't open input file: $!";
>> open my $fout, '>:raw', 'filename.out' or die "can't open output file: $!";
>> {
>> local $/;
>> $buf = <$fin>;
>> $buf =~ s/$sequence//;
>> print $fout $buf;
>> }
>> close $fout;
>> close $fin;
>
>For extra merit, make it work without reading
>the whole file into ram at once ;-)
>
> BugBear
Double buffer, something like this then:
open my $fin, '<:raw', 'filename.in' or die "can't open input file: $!";
open my $fout, '>:raw', 'filename.out' or die "can't open output file: $!";
$keep = 50;
{
local $/ = \4092;
($buf,$block) = ('','');
while (defined ($block = <$fin>))
{
$buf .= $block;
$keep = 0 if ($keep and $buf =~ s/000002020100.{18}//s);
print $fout substr( $buf, 0, length($buf)-$keep, "");
}
}
close $fout;
close $fin;
======================
Or, a little more efficient, but this may actually be slower:
$keep = 50;
{
local $/ = \4092;
($buf,$block) = ('','');
$bref = \$block;
while (defined ($$bref = <$fin>))
{
if ($keep)
{
$buf .= $block;
if ($buf =~ s/000002020100.{18}//s) {
$keep = 0;
$bref = \$buf;
}
print $fout substr( $buf, 0, length($buf)-$keep, "");
next;
}
print $fout $buf;
}
}
close $fout;
close $fin;
======================
-sln
------------------------------
Date: Tue, 18 Aug 2009 12:45:50 -0700
From: sln@netherlands.com
Subject: Re: search for hex characters in a binary file and remove them
Message-Id: <301m85l6hqiduk7mr8vlj446ag17bdia36@4ax.com>
On Tue, 18 Aug 2009 12:31:48 -0700, sln@netherlands.com wrote:
>On Tue, 18 Aug 2009 16:47:08 +0100, bugbear <bugbear@trim_papermule.co.uk_trim> wrote:
>
>>sln@netherlands.com wrote:
>>
>>For extra merit, make it work without reading
>>the whole file into ram at once ;-)
>>
>> BugBear
>
>Double buffer, something like this then:
>
>
> $keep = 50;
> {
> }
Of course you have to check $keep or $buf here
incase nothing was found, but if it wasn't found, the
output will match the input file, so invalid results:
print $fout $buf if $keep;
> close $fout;
> close $fin;
-sln
------------------------------
Date: Tue, 18 Aug 2009 13:21:04 -0700
From: sln@netherlands.com
Subject: Re: search for hex characters in a binary file and remove them
Message-Id: <laql859p3abug4au6fro0ad23kvj18gidj@4ax.com>
On Tue, 18 Aug 2009 16:47:08 +0100, bugbear <bugbear@trim_papermule.co.uk_trim> wrote:
>sln@netherlands.com wrote:
>
>> Hex characters? Like [a-f0-9] ? Or integers?
>>
>> $sequence = " 00 00 02 02 01 00";
>>
>> open my $fin, '<:raw', 'filename.in' or die "can't open input file: $!";
>> open my $fout, '>:raw', 'filename.out' or die "can't open output file: $!";
>> {
>> local $/;
>> $buf = <$fin>;
>> $buf =~ s/$sequence//;
>> print $fout $buf;
>> }
>> close $fout;
>> close $fin;
>
>For extra merit, make it work without reading
>the whole file into ram at once ;-)
>
> BugBear
haha, I can do than. special buffering, and algo.
-sln
------------------------------
Date: 6 Apr 2001 21:33:47 GMT (Last modified)
From: Perl-Users-Request@ruby.oce.orst.edu (Perl-Users-Digest Admin)
Subject: Digest Administrivia (Last modified: 6 Apr 01)
Message-Id: <null>
Administrivia:
#The Perl-Users Digest is a retransmission of the USENET newsgroup
#comp.lang.perl.misc. For subscription or unsubscription requests, send
#the single line:
#
# subscribe perl-users
#or:
# unsubscribe perl-users
#
#to almanac@ruby.oce.orst.edu.
NOTE: due to the current flood of worm email banging on ruby, the smtp
server on ruby has been shut off until further notice.
To submit articles to comp.lang.perl.announce, send your article to
clpa@perl.com.
#To request back copies (available for a week or so), send your request
#to almanac@ruby.oce.orst.edu with the command "send perl-users x.y",
#where x is the volume number and y is the issue number.
#For other requests pertaining to the digest, send mail to
#perl-users-request@ruby.oce.orst.edu. Do not waste your time or mine
#sending perl questions to the -request address, I don't have time to
#answer them even if I did know the answer.
------------------------------
End of Perl-Users Digest V11 Issue 2559
***************************************
