[30787] in Perl-Users-Digest
Perl-Users Digest, Issue: 2032 Volume: 11
daemon@ATHENA.MIT.EDU (Perl-Users Digest)
Thu Dec 4 21:09:46 2008
Date: Thu, 4 Dec 2008 18:09:12 -0800 (PST)
From: Perl-Users Digest <Perl-Users-Request@ruby.OCE.ORST.EDU>
To: Perl-Users@ruby.OCE.ORST.EDU (Perl-Users Digest)
Perl-Users Digest Thu, 4 Dec 2008 Volume: 11 Number: 2032
Today's topics:
a homework need help <jiao_he@sbcglobal.net>
Re: a homework need help <jurgenex@hotmail.com>
Re: Mathematica 7 compares to other languages sln@netherlands.com
Re: Mathematica 7 compares to other languages <xahlee@gmail.com>
Re: perl segfault - how to troubleshoot sln@netherlands.com
Re: perl segfault - how to troubleshoot sln@netherlands.com
Re: perl segfault - how to troubleshoot sln@netherlands.com
Re: perl segfault - how to troubleshoot sln@netherlands.com
Re: sysread and syswrite analogy xhoster@gmail.com
Re: Web programming: issues with large amounts og data sln@netherlands.com
Re: Web programming: issues with large amounts og data <rkb@i.frys.com>
Digest Administrivia (Last modified: 6 Apr 01) (Perl-Users-Digest Admin)
----------------------------------------------------------------------
Date: Thu, 4 Dec 2008 16:38:52 -0800
From: "Camel" <jiao_he@sbcglobal.net>
Subject: a homework need help
Message-Id: <VK_Zk.7526$as4.4004@nlpi069.nbdc.sbc.com>
I have no clue with my homework, can anyone help me?
Three digits "1,2,3", how many 3-digit numbers can be made? print all the
numbers
I know how to calculate the total numbers and using loop to print all the
numbers. The problem is how to write a generic program that takes any number
of digits. For example, if given "1,2,3,4,5" digits, you still can use your
program to print all the combinations of 5-digit numbers without modifing
it.
Thanks in advance.
------------------------------
Date: Thu, 04 Dec 2008 17:25:44 -0800
From: Jürgen Exner <jurgenex@hotmail.com>
Subject: Re: a homework need help
Message-Id: <a9vgj49rj45c9bb20u5e22gncbgld2rml5@4ax.com>
"Camel" <jiao_he@sbcglobal.net> wrote:
>I have no clue with my homework, can anyone help me?
>
>Three digits "1,2,3", how many 3-digit numbers can be made? print all the
>numbers
>
>I know how to calculate the total numbers and using loop to print all the
>numbers. The problem is how to write a generic program that takes any number
>of digits.
Quite simple. If you really know how to do the loop for a constant
number like e.g. 3, then just use a variable instead of that fixed
number 3 everywhere where you make any assumptions about 3.
E.g. instead of
for $i (1..3) {
use
for $i(1..$n){
and preset $n to the desired length.
jue
------------------------------
Date: Fri, 05 Dec 2008 00:09:53 GMT
From: sln@netherlands.com
Subject: Re: Mathematica 7 compares to other languages
Message-Id: <u7sgj4tqsgmr9ab401gqnvvgprnol9hn7p@4ax.com>
On Wed, 3 Dec 2008 16:32:57 -0800 (PST), Xah Lee <xahlee@gmail.com> wrote:
>On Dec 3, 4:22 pm, "Thomas M. Hermann" <tmh.pub...@gmail.com> wrote:
>> On Dec 3, 5:26 pm, Xah Lee <xah...@gmail.com> wrote:
>>
>>
>>
>> > Agreed. My paypal address is “xah @@@ xahlee.org”. (replace the triple
>> > @ to single one.) Once you paid thru paypal, you can post receit here
>> > if you want to, or i'll surely acknowledge it here.
>>
>> > Here's what i will do:
>>
>> > I will give a version of Mathematica code that has the same behavior
>> > as his. And i will give timing result. The code will run in
>> > Mathematica version 4. (sorry, but that's what i have) As i
>> > understand, Jon is running Mathematica 6. However, i don't see
>> > anything that'd require Mathematica 6. If my code is not faster or in
>> > other ways not satisfactory (by your judgement), or it turns out
>> > Mathematica 6 is necessary, or any problem that might occure, i offer
>> > money back guarantee.
>>
>> > Xah
>> > ?http://xahlee.org/
>>
>> > ?
>>
>> Alright, I've sent $20. The only reason I would request a refund is if
>> you don't do anything. As long as you improve the code as you've
>> described and post the results, I'll be satisfied. If the improvements
>> you've described don't result in better performance, that's OK.
>>
>> Good luck,
>>
>> Tom
>
>Got the payment. Thanks.
>
>I'll reply back with code tonight or tomorrow. Wee!
>
> Xah
>? http://xahlee.org/
>
>?
Well, its past 'tonight' and 6 hours to go till past 'tomorrow'.
Where the hell is it Zah Zah?
------------------------------
Date: Thu, 4 Dec 2008 17:02:59 -0800 (PST)
From: Xah Lee <xahlee@gmail.com>
Subject: Re: Mathematica 7 compares to other languages
Message-Id: <d2653d93-07bb-4e4c-9279-00cd0ef0549f@w1g2000prk.googlegroups.com>
alright, here's my improved code, pasted near the bottom.
let me say a few things about Jon's code.
If we rate that piece of mathematica code on the level of: Beginner
Mathematica programer, Intermediate, Advanced, where Beginner is
someone who just learned tried to program Mathematica no more than 6
months, then that piece of code is Beginner level.
Here's some basic analysis and explanation.
The program has these main functions:
=E2=80=A2 RaySphere
=E2=80=A2 Intersect
=E2=80=A2 RayTrace
=E2=80=A2 Create
=E2=80=A2 Main
The Main calls Create then feed it to RayTrace.
Create calls itself recursively, and basically returns a long list of
a repeating element, each of the element differ in their parameter.
RayTrace calls Intersect 2 times. Intersect has 2 forms, one of them
calls itself recursively. Both forms calls RaySphere once.
So, the core loop is with the Intersect function and RaySphere. Some
99.99% of time are spent there.
------------------
I didn't realize until after a hour, that if Jon simply give numerical
arguments to Main and Create, the result timing by a factor of 0.3 of
original. What a incredible sloppiness! and he intended this to show
Mathematica speed with this code?
The Main[] function calls Create. The create has 3 parameters: level,
c, and r. The level is a integer for the recursive level of
raytracing . The c is a vector for sphere center i presume. The r is
radius of the sphere. His input has c and r as integers, and this in
Mathematica means computation with exact arithmetics (and automatic
kicks into infinite precision if necessary). Changing c and r to float
immediately reduced the timing to 0.3 of original.
------------------
now, back to the core loop.
The RaySphere function contain codes that does symbolic computation by
calling Im, which is the imaginary part of a complex number!! and if
so, it returns the symbol Infinity! The possible result of Infinity is
significant because it is used in Intersect to do a numerical
comparison in a If statement. So, here in these deep loops,
Mathematica's symbolic computation is used for numerical purposes!
So, first optimization at the superficial code form level is to get
rid of this symbolic computation.
Instead of checking whethere his =E2=80=9Cdisc =3D Sqrt[b^2 - v.v + r^2]=E2=
=80=9D has
imaginary part, one simply check whether the argument to sqrt is
negative.
after getting rid of the symbolic computation, i made the RaySphere
function to be a Compiled function.
I stopped my optimization at this step.
The above are some _fundamental_ things any dummy who claims to code
Mathematica for speed should know. Jon has written a time series
Mathematica package that he's selling commercially. So, either he got
very sloppy with this Mathematica code, or he intentionally made it
look bad, or that his Mathematica skill is truely beginner level. Yet
he dares to talk bullshit in this thread.
Besides the above basic things, there are several aspects that his
code can improve in speed. For example, he used pattern matching to do
core loops.
e.g. Intersect[o_, d_][{lambda_, n_}, Bound[c_, r_, s_]]
any Mathematica expert knows that this is something you don't want to
do if it is used in a core loop. Instead of pattern matching, one can
change the form to Function and it'll speed up.
Also, he used =E2=80=9CBlock=E2=80=9D, which is designed for local variable=
s and the
scope is dynamic scope. However the local vars used in this are local
constants. A proper code would use =E2=80=9CWith=E2=80=9D instead. (in lisp=
, this is
various let, let*. Lispers here can imagine how lousy the code is
now.)
Here's a improved code. The timing of this code is about 0.2 of the
original. Also, optimization is purely based on code doodling. That
is, i do not know what his code is doing, i do not have experience in
writing a ray tracer. All i did is eyeballing his code flow, and
improved the form.
norm=3DFunction[#/Sqrt@(Plus@@(#^2))];
delta=3DSqrt[$MachineEpsilon];
myInfinity=3D10000.;
Clear[RaySphere];
RaySphere =3D Compile[{o1, o2, o3, d1, d2, d3, c1, c2, c3, r},
Block[{v =3D {c1 - o1, c2 - o2, c3 - o3},
b =3D d1*(c1 - o1) + d2*(c2 - o2) + d3*(c3 - o3),
discriminant =3D -(c1 - o1)^2 - (c2 - o2)^2 +
(d1*(c1 - o1) + d2*(c2 - o2) + d3*(c3 - o3))^2 -
(c3 - o3)^2 + r^2, disc, t1, t2},
If[discriminant < 0., myInfinity,
disc =3D Sqrt[discriminant]; If[(t1 =3D b - disc) > 0.,
t1, If[(t2 =3D b + disc) <=3D 0., myInfinity, t2]]]]];
Remove[Intersect];
Intersect[{o1_,o2_,o3_},{d1_,d2_,d3_}][{lambda_,n_},Sphere
[{c1_,c2_,c3_},r_]]:=3D
Block[{lambda2=3DRaySphere[o1,o2,o3,d1,d2,d3,c1,c2,c3,r]},
If[lambda2=E2=89=A5lambda,{lambda,n},{lambda2,
norm[{o1,o2,o3}+lambda2 *{d1,d2,d3}-{c1,c2,c3}]}]]
Intersect[{o1_,o2_,o3_},{d1_,d2_,d3_}][{lambda_,n_},
Bound[{c1_,c2_,c3_},r_,s_]]:=3D
Block[{lambda2=3DRaySphere[o1,o2,o3,d1,d2,d3,c1,c2,c3,r]},
If[lambda2=E2=89=A5lambda,{lambda,n},
Fold[Intersect[{o1,o2,o3},{d1,d2,d3}],{lambda,n},s]]]
Clear[neglight,nohit]
neglight=3DN@norm[{1,3,-2}];
nohit=3D{myInfinity,{0.,0.,0.}};
Clear[RayTrace];
RayTrace[o_,d_,scene_]:=3D
Block[{lambda,n,g,p},{lambda,n}=3DIntersect[o,d][nohit,scene];
If[lambda\[Equal]myInfinity,0,g=3Dn.neglight;
If[g=E2=89=A40,
0,{lambda,n}=3DIntersect[o+lambda d+delta n,neglight]
[nohit,scene];
If[lambda<myInfinity,0,g]]]]
Clear[Create];
Create[level_,c_,r_]:=3D
Block[{obj=3DSphere[c,r]},
If[level\[Equal]1,obj,
Block[{a=3D3*r/Sqrt[12],Aux},
Aux[x1_,z1_]:=3DCreate[level-1,c+{x1,a,z1},0.5 r];
Bound[c,3 r,{obj,Aux[-a,-a],Aux[a,-a],Aux[-a,a],Aux[a,a]}]
]
]]
Main[level_,n_,ss_]:=3D
With[{scene=3DCreate[level,{0.,-1.,4.},1.]},
Table[Sum[
RayTrace[{0,0,0},
N@norm[{(x+s/ss/ss)/n-1/2,(y+Mod[s,ss]/ss)/
n-1/2,1}],scene],{s,0,
ss^2-1}]/ss^2,{y,0,n-1},{x,0,n-1}]]
Timing[Export["image.pgm",Graphics@Raster@Main[2,100,4.]]]
Note to those who have Mathematica.
Mathematica 6 has Normalize, but that's not in Mathematica 4, so i
cooked up above.
Also, Mathematica 6 has AbsoluteTiming, which is intended to be
equivalent if you use stop watch to measure timing. Mathematica 4 has
only Timing, which measures CPU time. My speed improvement is based on
Timing. But the same factor will shown when using Mathematica 6 too.
I'm pretty sure further speed up by 0.5 factor of above's timing is
possible. Within 2 more hours of coding.
Jon wrote:
=C2=ABThe Mathematica code is 700,000x slower so a 50% improvement will be
uninteresting. Can you make my Mathematica code five orders of
magnitude faster or not?=C2=BB
If anyone pay me $300, i can try to make it whatever the level of F#
or OCaml's speed is as cited in Jon's website. (
http://www.ffconsultancy.com/languages/ray_tracer/index.html ).
Please write out or write to me the detail exactly what speed is
required in some precise terms. If i agree to do it, spec satisfaction
is guaranteed or your money back.
PS Thanks Thomas M Hermann. It was fun.
Xah
=E2=88=91 http://xahlee.org/
=E2=98=84
------------------------------
Date: Fri, 05 Dec 2008 01:07:51 GMT
From: sln@netherlands.com
Subject: Re: perl segfault - how to troubleshoot
Message-Id: <rnugj496i89ie7u1p6hva0thammijjrga1@4ax.com>
On Thu, 04 Dec 2008 10:24:33 -0800, Tim Greer <tim@burlyhost.com> wrote:
>James Harris wrote:
>
>> I was thinking more of indications while Linux is running and the
>> disks are mounted, not of taking them down to scan them.
>
>You can run the checks while the drives/partitions are mounted. fsck
>doesn't have to be told to actually try and fix the issues it finds,
>but you can simply check with smartctl or similar tools, as well as
>running badblocks on a live environment (again, you needn't tell it to
>fix the bad blocks or issues it finds). What tools you use, depends on
>your drives, drivers, kernel options and what you have installed. I'd
>recommend asking in a Linux group or searching google for specifics.
I'm not sure about anything Linux. Usually segment faults originates from
the cpu, filters up through the OS, then to the app. Or, it originates from
the OS, then up to the app, this usually a pointer that is not in the virtual
address space. Like de-referencing address 0.
It depends on the fault level, since, all the app see's is the OS generated
exceptions, it may be too serious for the app to continue.
In fact, sometimes, the app can do all the exception handling it wants but
it cannot trap the exception. The OS takes a dive and terminates without
prejudice.
When the OS tanks like this, usually the app see's nothing, or if anything
just spurts out a message like 'segfault', when its notified its about to be
terminated by the OS and nothing you can do about it, and nothing I'm gonna
tell you about it.
In Windows, a pretty system dialog comes up telling you Perl is gonna die,
do you want to Debug? Do you feel lucky sucker, well do ya? Of course, there
is not debug build, so click ok and watch it terminate.
Its a hardware problem, fix it. It will happen again and again and again.
OR, you can try to DEBUG the OS, because thats what it comes down to.
If you think the OS is faulty, you should contact Linux. But, they will
laugh to death before they hang up on you.
sln
------------------------------
Date: Fri, 05 Dec 2008 01:30:00 GMT
From: sln@netherlands.com
Subject: Re: perl segfault - how to troubleshoot
Message-Id: <520hj4174r4pqkv6bvm20p01a5khea3qgl@4ax.com>
On Fri, 05 Dec 2008 01:07:51 GMT, sln@netherlands.com wrote:
>On Thu, 04 Dec 2008 10:24:33 -0800, Tim Greer <tim@burlyhost.com> wrote:
>
>>James Harris wrote:
>>
>>> I was thinking more of indications while Linux is running and the
>>> disks are mounted, not of taking them down to scan them.
>>
>>You can run the checks while the drives/partitions are mounted. fsck
>>doesn't have to be told to actually try and fix the issues it finds,
>>but you can simply check with smartctl or similar tools, as well as
>>running badblocks on a live environment (again, you needn't tell it to
>>fix the bad blocks or issues it finds). What tools you use, depends on
>>your drives, drivers, kernel options and what you have installed. I'd
>>recommend asking in a Linux group or searching google for specifics.
>
>I'm not sure about anything Linux. Usually segment faults originates from
>the cpu, filters up through the OS, then to the app. Or, it originates from
>the OS, then up to the app, this usually a pointer that is not in the virtual
>address space. Like de-referencing address 0.
>
>It depends on the fault level, since, all the app see's is the OS generated
>exceptions, it may be too serious for the app to continue.
>
>In fact, sometimes, the app can do all the exception handling it wants but
>it cannot trap the exception. The OS takes a dive and terminates without
>prejudice.
>
>When the OS tanks like this, usually the app see's nothing, or if anything
>just spurts out a message like 'segfault', when its notified its about to be
>terminated by the OS and nothing you can do about it, and nothing I'm gonna
>tell you about it.
>
>In Windows, a pretty system dialog comes up telling you Perl is gonna die,
>do you want to Debug? Do you feel lucky sucker, well do ya? Of course, there
>is not debug build, so click ok and watch it terminate.
>
>Its a hardware problem, fix it. It will happen again and again and again.
>OR, you can try to DEBUG the OS, because thats what it comes down to.
>If you think the OS is faulty, you should contact Linux. But, they will
>laugh to death before they hang up on you.
>
>
But remember, in your case, the OS wasn't faulty, neither was Perl.
The OS was fed faulty IDE electronic data from its file drivers.
BIG indication of hardware problems.
Here's a lesson and test for you all at the same time. Try to fix a
faulty drive, recover data, fix tables, try to get it right.
Reboot, then do it again. Watch 10,000 more errors come up.
Stick in some borderline memory, with known good drives, boot up, run some
stuff, reboot, then check the disk. Fix the disk, then run some more stuf.
Reboot, check the disk. Guess what? Your fixing the disk as much as when
the known bad disk was in there.
Kinda makes ya think doesen't it? Chicken/Egg thing.
In reality, you don't know if its bad disk or bad memory.
Hey, its not over, you just spent $800 bucks on new memory and disk
hardware... but man, its still happening... the cpu is bad.
But wait, it gets better, the bios is trashed giving false info to
the major components.. oh man, the DMA controller is fried.
Now you have to replace the motherboard.
But wait, how much/many OS re-installs does this all take.
Oh, maybe 10-20 before you get smart. By this time you have respect
for your hardware capturs' and have given up one man band
System Administrator job forever, and are praying at the feet of
the Hardware God, so help me Jesus, and chanting "I believe, I believe.."
when they find you catatonic in your house, frothing at the mouth with
a gun to your head !!!!!!!!!!!!!!!!!!
sln
------------------------------
Date: Fri, 05 Dec 2008 01:40:28 GMT
From: sln@netherlands.com
Subject: Re: perl segfault - how to troubleshoot
Message-Id: <af1hj4de9kb30iqgtubal6b7n73giaekri@4ax.com>
On Fri, 05 Dec 2008 01:30:00 GMT, sln@netherlands.com wrote:
>On Fri, 05 Dec 2008 01:07:51 GMT, sln@netherlands.com wrote:
>
>>On Thu, 04 Dec 2008 10:24:33 -0800, Tim Greer <tim@burlyhost.com> wrote:
>>
>>>James Harris wrote:
>>>
>>>> I was thinking more of indications while Linux is running and the
>>>> disks are mounted, not of taking them down to scan them.
>>>
>>>You can run the checks while the drives/partitions are mounted. fsck
>>>doesn't have to be told to actually try and fix the issues it finds,
>>>but you can simply check with smartctl or similar tools, as well as
>>>running badblocks on a live environment (again, you needn't tell it to
>>>fix the bad blocks or issues it finds). What tools you use, depends on
>>>your drives, drivers, kernel options and what you have installed. I'd
>>>recommend asking in a Linux group or searching google for specifics.
>>
>>I'm not sure about anything Linux. Usually segment faults originates from
>>the cpu, filters up through the OS, then to the app. Or, it originates from
>>the OS, then up to the app, this usually a pointer that is not in the virtual
>>address space. Like de-referencing address 0.
>>
>>It depends on the fault level, since, all the app see's is the OS generated
>>exceptions, it may be too serious for the app to continue.
>>
>>In fact, sometimes, the app can do all the exception handling it wants but
>>it cannot trap the exception. The OS takes a dive and terminates without
>>prejudice.
>>
>>When the OS tanks like this, usually the app see's nothing, or if anything
>>just spurts out a message like 'segfault', when its notified its about to be
>>terminated by the OS and nothing you can do about it, and nothing I'm gonna
>>tell you about it.
>>
>>In Windows, a pretty system dialog comes up telling you Perl is gonna die,
>>do you want to Debug? Do you feel lucky sucker, well do ya? Of course, there
>>is not debug build, so click ok and watch it terminate.
>>
>>Its a hardware problem, fix it. It will happen again and again and again.
>>OR, you can try to DEBUG the OS, because thats what it comes down to.
>>If you think the OS is faulty, you should contact Linux. But, they will
>>laugh to death before they hang up on you.
>>
>>
>But remember, in your case, the OS wasn't faulty, neither was Perl.
>The OS was fed faulty IDE electronic data from its file drivers.
>BIG indication of hardware problems.
>
>Here's a lesson and test for you all at the same time. Try to fix a
>faulty drive, recover data, fix tables, try to get it right.
>Reboot, then do it again. Watch 10,000 more errors come up.
>Stick in some borderline memory, with known good drives, boot up, run some
>stuff, reboot, then check the disk. Fix the disk, then run some more stuf.
>Reboot, check the disk. Guess what? Your fixing the disk as much as when
>the known bad disk was in there.
>
>Kinda makes ya think doesen't it? Chicken/Egg thing.
>In reality, you don't know if its bad disk or bad memory.
>Hey, its not over, you just spent $800 bucks on new memory and disk
>hardware... but man, its still happening... the cpu is bad.
>But wait, it gets better, the bios is trashed giving false info to
>the major components.. oh man, the DMA controller is fried.
>Now you have to replace the motherboard.
>
>But wait, how much/many OS re-installs does this all take.
>Oh, maybe 10-20 before you get smart. By this time you have respect
>for your hardware capturs' and have given up one man band
>System Administrator job forever, and are praying at the feet of
>the Hardware God, so help me Jesus, and chanting "I believe, I believe.."
>when they find you catatonic in your house, frothing at the mouth with
>a gun to your head !!!!!!!!!!!!!!!!!!
>
>
Oh, but it gets better. What if right now, the operating system files
are corrupt. Which ones? Where when, how to find out? When did this happen?
Seriously, are you telling me the OS could be corrupt? Why yes, yes I am.
I'm telling you right now the OS could be corrupt and you don't even know
about it.
Get a load of dem apples !
sln
------------------------------
Date: Fri, 05 Dec 2008 01:55:42 GMT
From: sln@netherlands.com
Subject: Re: perl segfault - how to troubleshoot
Message-Id: <os1hj4t5m9svo6hl3bs4uo4itspk246kmi@4ax.com>
On Fri, 05 Dec 2008 01:40:28 GMT, sln@netherlands.com wrote:
>On Fri, 05 Dec 2008 01:30:00 GMT, sln@netherlands.com wrote:
>
>>On Fri, 05 Dec 2008 01:07:51 GMT, sln@netherlands.com wrote:
>>
>>>On Thu, 04 Dec 2008 10:24:33 -0800, Tim Greer <tim@burlyhost.com> wrote:
>>>
>>>>James Harris wrote:
>>>>
>>>>> I was thinking more of indications while Linux is running and the
>>>>> disks are mounted, not of taking them down to scan them.
>>>>
>>>>You can run the checks while the drives/partitions are mounted. fsck
>>>>doesn't have to be told to actually try and fix the issues it finds,
>>>>but you can simply check with smartctl or similar tools, as well as
>>>>running badblocks on a live environment (again, you needn't tell it to
>>>>fix the bad blocks or issues it finds). What tools you use, depends on
>>>>your drives, drivers, kernel options and what you have installed. I'd
>>>>recommend asking in a Linux group or searching google for specifics.
>>>
>>>I'm not sure about anything Linux. Usually segment faults originates from
>>>the cpu, filters up through the OS, then to the app. Or, it originates from
>>>the OS, then up to the app, this usually a pointer that is not in the virtual
>>>address space. Like de-referencing address 0.
>>>
>>>It depends on the fault level, since, all the app see's is the OS generated
>>>exceptions, it may be too serious for the app to continue.
>>>
>>>In fact, sometimes, the app can do all the exception handling it wants but
>>>it cannot trap the exception. The OS takes a dive and terminates without
>>>prejudice.
>>>
>>>When the OS tanks like this, usually the app see's nothing, or if anything
>>>just spurts out a message like 'segfault', when its notified its about to be
>>>terminated by the OS and nothing you can do about it, and nothing I'm gonna
>>>tell you about it.
>>>
>>>In Windows, a pretty system dialog comes up telling you Perl is gonna die,
>>>do you want to Debug? Do you feel lucky sucker, well do ya? Of course, there
>>>is not debug build, so click ok and watch it terminate.
>>>
>>>Its a hardware problem, fix it. It will happen again and again and again.
>>>OR, you can try to DEBUG the OS, because thats what it comes down to.
>>>If you think the OS is faulty, you should contact Linux. But, they will
>>>laugh to death before they hang up on you.
>>>
>>>
>>But remember, in your case, the OS wasn't faulty, neither was Perl.
>>The OS was fed faulty IDE electronic data from its file drivers.
>>BIG indication of hardware problems.
>>
>>Here's a lesson and test for you all at the same time. Try to fix a
>>faulty drive, recover data, fix tables, try to get it right.
>>Reboot, then do it again. Watch 10,000 more errors come up.
>>Stick in some borderline memory, with known good drives, boot up, run some
>>stuff, reboot, then check the disk. Fix the disk, then run some more stuf.
>>Reboot, check the disk. Guess what? Your fixing the disk as much as when
>>the known bad disk was in there.
>>
>>Kinda makes ya think doesen't it? Chicken/Egg thing.
>>In reality, you don't know if its bad disk or bad memory.
>>Hey, its not over, you just spent $800 bucks on new memory and disk
>>hardware... but man, its still happening... the cpu is bad.
>>But wait, it gets better, the bios is trashed giving false info to
>>the major components.. oh man, the DMA controller is fried.
>>Now you have to replace the motherboard.
>>
>>But wait, how much/many OS re-installs does this all take.
>>Oh, maybe 10-20 before you get smart. By this time you have respect
>>for your hardware capturs' and have given up one man band
>>System Administrator job forever, and are praying at the feet of
>>the Hardware God, so help me Jesus, and chanting "I believe, I believe.."
>>when they find you catatonic in your house, frothing at the mouth with
>>a gun to your head !!!!!!!!!!!!!!!!!!
>>
>>
>Oh, but it gets better. What if right now, the operating system files
>are corrupt. Which ones? Where when, how to find out? When did this happen?
>
>Seriously, are you telling me the OS could be corrupt? Why yes, yes I am.
>I'm telling you right now the OS could be corrupt and you don't even know
>about it.
>
>Get a load of dem apples !
>
My home windows xp system uses a DFI Lanparty SLI with a 4x250 GB
Fujitsu Sata II raid array, running an overclocked Opteron dual core at 3 Ghz,
with 4G dual-core (the FSB is actually o/c'd to 270, and with the multiplyer
give the cpu its 3Ghz) No cooling problems because I have a massive sink on the cpu,
on air. All I can tell you is its a beast, and very reliable.
These Fujitsu's are/were the fastest in the world a year or so ago. Dunno now.
All I can tell you is. Whatever problems you 'think' can be solved on a computer
language usenet group, you are sadly mistaken.
sln
------------------------------
Date: 05 Dec 2008 00:03:11 GMT
From: xhoster@gmail.com
Subject: Re: sysread and syswrite analogy
Message-Id: <20081204190210.083$8t@newsreader.com>
Jim Gibson <jimsgibson@gmail.com> wrote:
> In article <20081204145716.083$RZ@newsreader.com>, <xhoster@gmail.com>
> wrote:
>
> > For file handles that don't have O_NONBLOCK turned on, I see this
> > behavior:
> >
> > sysread will return partial reads (fewer bytes than were requested),
> > rather than blocking, provided that the partial read is at least one
> > byte.
>
> Check "Advanced Programming in the Unix Environment", R. Stevens., p.
> 54. The Unix read system call operating in blocked mode will return
> fewer than the number of bytes requested if:
>
> 1. File I/O: only the last record.
> 2. Terminal I/O: returns on end-of-line
> 3. Network I/O: returns if network can't supply bytes in reasonable
> time.
> 4. Device I/O: device is record-oriented.
>
> >
> > syswrite will not perform partial writes, but rather it will block
> > waiting for the entire write to be accepted.
>
> Well you are operating in blocked mode, so it should block. So the
> question should be "why does read not always block", and the exceptions
> as listed above seem reasonable.
Sure, but (the analogy to) exception 3 seems reasonable for syswrite, as
well.
> > Based on sysread's behavior, I would have expected syswrite to return
> > immediately with a partial write, provided that at least one character
> > was written. Why the lack of analogy between the two?
>
> I think that one basic difference is that for a read, you do not know
> how many bytes are being made available.
If the protocol decrees "You will send me 31,517 bytes" then I do know how
many will be available. Just not exactly when.
> For a write, you know exactly
> how many bytes are available for writing.
>
> If the system were to only accept some of your bytes, you would have to
> save the unwritten bytes and try to write them again, which is what
> happens for non-blocking writes.
If I get a partial read, I need to save the partial read someplace,
come back later to get the rest, then concatenate them together. That
seems completely analogous to what I have to do with partial writes.
> For input, it makes sense to return fewer bytes in those cases where
> blocking would be unreasonable (stalled network I/O or record-oriented
> data.)
>
> For output, it doesn't make much sense to only accept a partial write
> and return, because the program will usually just try to write the
> unwritten bytes again, so the system call may as well block. It makes
> more sense to use the select function to make sure that the output
> stream is ready to accept output, and try to use a buffer size
> sufficient to accept most output records.
But because syswrite will not do partial writes, the size of buffer which
it is safe to use is 1 byte. Which isn't much of a buffer.
The way sysread works, it makes sense to use select and sysread on a handle
that is in standard (blocking) mode. The way syswrite works, it doesn't
make much sense to use it with select in standard (blocking) mode, you
would have to syswrite one byte at a time. It seems like an odd
arrangement. But apparently it is an odd arrangement that Perl inherits
from the underlying system, and neglected to shield me from.
Does O_NONBLOCK (or $fh->blocking(0))work on all the systems which select
works on?
Xho
--
-------------------- http://NewsReader.Com/ --------------------
The costs of publication of this article were defrayed in part by the
payment of page charges. This article must therefore be hereby marked
advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate
this fact.
------------------------------
Date: Thu, 04 Dec 2008 23:37:43 GMT
From: sln@netherlands.com
Subject: Re: Web programming: issues with large amounts og data
Message-Id: <08qgj4d2jf00mg9nbnaslie0rf6cd71fr8@4ax.com>
On Thu, 04 Dec 2008 20:14:14 GMT, sln@netherlands.com wrote:
[snip]
>> system("..\\generic_download_child.pl $uid \"$req_url\" 1>
>>$uid.stdout 2>$uid.stderr");
>
>You should get out of the habit of passing unquoted "$variables" to
>the command shell, even if you know they don't need quotes.
>
> system("..\\generic_download_child.pl \"$uid\" \"$req_url\" 1>\"$uid.stdout\" 2>\"$uid.stderr\"");
> ^^ ^^ ^^ ^^ ^^ ^^ ^^
>
>sln
I'm not saying this is your problem, but some other guy was trying to do
the same thing, not quoting $variables passed to the command shell.
Below is his problem and the fix. Just something to watch out for.
sln
________________________________________________________
Here is the simple loop with the problem:
print "\n\nProcessing merchant data: \n";
foreach $merchant(sort keys %merchants) {
print "\tmerchant name: $merchant\n\tmid: $merchants{$merchant}\n
\tAPI user name: $merchants_usernames{$merchant}\n\n";
system("activity.report.1.pl \"$merchant\" $merchants_usernames
{$merchant} $merchants{$merchant} $date_string 1>$dir\\activity.Report.
$merchant.stdout 2>$dir\\activity.Report.$merchant.stderr") == 0
or warn "Problem creating activity report for $merchant\n";
}
I am using "use strict" and $| = 1, from the top of the script.
The hash is fully populated and there are no null values in any
variable in the code shown.
The problem is that for 90% of the merchants, the script
activity.report.1.pl executes fine, producing the expected files
(those with the contents of stdout and stderr, as well as the desired
PDF file). For the rest, we get the merchant name, and the values
from the two hashes (that contain values to be provided as arguments
for the activity report script), followed by:
The process cannot access the file because it is being used by another
process.
Problem creating activity report for xxxxxxxxxxx xxxxxxx
The second line shown here is obviously from the die clause, but where
is the message "The process cannot access the file because it is being
used by another process." coming from? None of the information that
is written to stdout by the activity report script appears (e.g. the
first statement is to write to standard out the values of the
arguments), and nothing appears to be written to stderr either. The
files that ought to contain what is supposed to be written to stdout
and stderr don't even get created, hence my guess that the problem is
with the call to system.
Having previously printed out the contents of the hashes, I know the
correct arguments are being passed to "activity.report.1.pl" (from
comparing what was printed to the contents of the database table from
which the values are obtained, and what the activity report script
prints to standard out for the arguments it has received), and
activity.report.1.pl always runs to completion successfully when I
invoke it manually with all the same arguments that this script uses,
so I am at a loss. Why is my call to system failing (or how can I
find out), and what can be done to fix it? And why would it fail for
only 10% of the merchants?
Thanks
Ted
==========================================================================
I posted an arglist with each one quoted. That is the correct fix with one
small change. I didn't see the redirect parameters.
Below does both show the real problem(s) in 'Consideration #1 and #2' and
the fix in 'Consideration #3 and #4'.
There is only one problem when the OP gets 'used by another process' message.
sln
============================================================================
Command line:
____________________________
c:\temp>perl ttt.pl
Consideration #1:
---------------------------
The process cannot access the file because it is being used by another process.
problem with jjj.pl: 256
Consideration #2:
---------------------------
Consideration #3:
---------------------------
Consideration #4:
---------------------------
c:\temp>
____________________________________________________________
-----------------------------------------
ttt.pl:
-----------------------------------------
use strict;
use warnings;
my @args;
my $dir = "c:\\temp";
my $merchant = "Home Depot";
my %merchants = ($merchant => 'home depot');
my %merchants_usernames = ($merchant => 'H DEPOT');
my $date_string = "Jan 1, 2009";
print "Consideration #1:\n---------------------------\n";
system("perl jjj.pl \"$merchant\" $merchants_usernames{$merchant} $merchants{$merchant} $date_string 1>$dir\\activity.Report.$merchant.stdout1 2>$dir\\activity.Report.$merchant.stderr1") == 0
or warn "problem with jjj.pl: $?\n";
print "\nConsideration #2:\n---------------------------\n";
@args = (
'jjj.pl',
"\"$merchant\"",
$merchants_usernames{$merchant},
$merchants{$merchant},
$date_string,
"1>\"$dir\\activity.Report.$merchant.stdout2\"",
"2>\"$dir\\activity.Report.$merchant.stderr2\"",
);
system("perl @args") == 0 or warn "problem with jjj.pl: $?";
print "\nConsideration #3:\n---------------------------\n";
@args = (
'jjj.pl',
"\"$merchant\"",
"\"$merchants_usernames{$merchant}\"",
"\"$merchants{$merchant}\"",
"\"$date_string\"",
"1>\"$dir\\activity.Report.$merchant.stdout3\"",
"2>\"$dir\\activity.Report.$merchant.stderr3\"",
);
system("perl @args") == 0 or warn "problem with jjj.pl: $?";
print "Consideration #4:\n---------------------------\n";
system("perl jjj.pl \"$merchant\" \"$merchants_usernames{$merchant}\" \"$merchants{$merchant}\" \"$date_string\" 1>\"$dir\\activity.Report.$merchant.stdout4\"
2>\"$dir\\activity.Report.$merchant.stderr4\"") == 0
or warn "problem with jjj.pl: $?\n";
----------------------------------------
jjj.pl:
----------------------------------------
use strict;
use warnings;
for (@ARGV) {
print "$_\n";
}
activity.Report.Home:
-------------------------------------
activity.Report.Home Depot.stdout2:
--------------------------------------
Home Depot
H
DEPOT
home
depot
Jan
1,
2009
activity.Report.Home Depot.stdout3:
activity.Report.Home Depot.stdout4:
---------------------------------------
Home Depot
H DEPOT
home depot
Jan 1, 2009
------------------------------
Date: Thu, 4 Dec 2008 16:11:16 -0800 (PST)
From: Ron Bergin <rkb@i.frys.com>
Subject: Re: Web programming: issues with large amounts og data
Message-Id: <1353575f-a2e8-4b22-8b4b-eef7e30238ae@f40g2000pri.googlegroups.com>
On Dec 4, 12:14 pm, s...@netherlands.com wrote:
>
> You should get out of the habit of passing unquoted "$variables" to
> the command shell, even if you know they don't need quotes.
>
> system("..\\generic_download_child.pl \"$uid\" \"$req_url\" 1>\"$uid.stdout\" 2>\"$uid.stderr\"");
> ^^ ^^ ^^ ^^ ^^ ^^ ^^
>
> sln
Personally, I think this is cleaner.
system(qq/..\\generic_download_child.pl "$uid" "$req_url"
1>"$uid.stdout" 2>"$uid.stderr"/);
------------------------------
Date: 6 Apr 2001 21:33:47 GMT (Last modified)
From: Perl-Users-Request@ruby.oce.orst.edu (Perl-Users-Digest Admin)
Subject: Digest Administrivia (Last modified: 6 Apr 01)
Message-Id: <null>
Administrivia:
#The Perl-Users Digest is a retransmission of the USENET newsgroup
#comp.lang.perl.misc. For subscription or unsubscription requests, send
#the single line:
#
# subscribe perl-users
#or:
# unsubscribe perl-users
#
#to almanac@ruby.oce.orst.edu.
NOTE: due to the current flood of worm email banging on ruby, the smtp
server on ruby has been shut off until further notice.
To submit articles to comp.lang.perl.announce, send your article to
clpa@perl.com.
#To request back copies (available for a week or so), send your request
#to almanac@ruby.oce.orst.edu with the command "send perl-users x.y",
#where x is the volume number and y is the issue number.
#For other requests pertaining to the digest, send mail to
#perl-users-request@ruby.oce.orst.edu. Do not waste your time or mine
#sending perl questions to the -request address, I don't have time to
#answer them even if I did know the answer.
------------------------------
End of Perl-Users Digest V11 Issue 2032
***************************************
