[26771] in Perl-Users-Digest
Perl-Users Digest, Issue: 8841 Volume: 10
daemon@ATHENA.MIT.EDU (Perl-Users Digest)
Mon Jan 9 06:05:18 2006
Date: Mon, 9 Jan 2006 03:05:06 -0800 (PST)
From: Perl-Users Digest <Perl-Users-Request@ruby.OCE.ORST.EDU>
To: Perl-Users@ruby.OCE.ORST.EDU (Perl-Users Digest)
Perl-Users Digest Mon, 9 Jan 2006 Volume: 10 Number: 8841
Today's topics:
Re: IO::Pipe and loss of data <ina.scherf@sap.com>
Digest Administrivia (Last modified: 6 Apr 01) (Perl-Users-Digest Admin)
----------------------------------------------------------------------
Date: Mon, 9 Jan 2006 10:13:54 +0100
From: "Ina Scherf" <ina.scherf@sap.com>
Subject: Re: IO::Pipe and loss of data
Message-Id: <dpt9gk$l1i$1@news.sap-ag.de>
Hi again and sorry for the delay,
<xhoster@gmail.com> wrote in message
news:20060105165122.247$Oo@newsreader.com...
> In your sample code, the message is always short. Is it always about that
> size in the real code?
No. Messages vary in their length but can reach up to 2000 characters. The
shortest might be about 20 characters I think. I once had the problem that
everything over 2500 gets cut so I tried to keep this down to 2000. But
anyway the error I got there was a different.
> There is a fixed amount of buffer (probably 4K
> on your system) for the pipe. If the reader can't keep up with the
> writer(s), then that buffer gets full, and writers will block until the
> reader has a chance to catch up, making more room in the buffer.
Ok, thanks for this information, I did not knew about that. I was just
thinking if this might cause the error. But your explanation gets me to know
this is not the case.
>> Wed Jan 4 19:55:25 2006 Server 3025: Missing message between 70 and 74
> I would make it log the just-received line in it's entirety, not only the
> serial number of that line. In fact, the main program should probably
> store not just the previous message-id, but the entire previous message,
> for each "server". Then, when there seems to be a gap, you can print out
> the entire message before, and the entire message after, the gap. That
> could help a lot in debugging.
After now switching to a named pipe I started with printing more info to my
log. The line before is always complete and does not show any smithereens.
>
>> sub logProcessing{
>> my $line = '';
>> my ($msg, $server);
>> eval {
>> local $SIG{ALRM} = sub { die "lesedauer" };
>> alarm(1);
>> $line = <$pipe_log>;
>> alarm(0);
>> };
>
> I don't understand the point of this time out. If the timeout occurs,
> you never complain about it or even detect it. You merely start another
> call of the same thing that just timed out. If all you do upon a timeout
> is restart the same thing, why bother timing out?
This seems to be obsolete because I cut away code. Normally the father
process shall do three things parallel in its routine. One of them has to do
with user input to another script and sending data. So I wanted to reduce
wating time at this point (although I believe this pipe is _never_ empty for
over one second).
> Anyway, if the alarm goes off after the next line is read from $pipe_log,
> but before the "alarm(0)" gets executed, then you die out of the eval{}
> and
> the line you just read from $pipe_log gets ignored. That could account
> for
> your missing data!
But should not be at least half a line in $line then? I changed my log to
have the old line and the new line. No half lines so far. Either there is
the whole line or nothing in it. And it is very easy to check if the line is
not complete as the last thing to be transfered is a dumped object and this
gives an error on every missing character at the end.
>> return if ($line !~ /::/);
> Wouldn't it be better to die upon malformed data?
Yes - you're completely right :)
>> ($msg, $server, $line) = split(/::/, $line);
>>
>> if(!exists $mshash{$server}){
>> $mshash{$server} = $msg;
>> }else{
>> if($msg != $mshash{$server} +1){
>> writeLog("Server $server: Missing message between $mshash{$server} and
>> $msg");
> I would add $line, if not $mshash2{$server}, to the logged info.
I do both right now.
Here is just a short update on what I found out right now:
I checked the length of the message I send from the child (first parameter
is now the length of message followed by a delimiter, father reads this out
first and checks if this is correct). All messages send match this length.
So it should not be a line cut through by anything.
The error occured (on thursday) when I only had _one_ child writing to the
pipe. To have a number it happens between message nr 327059 and 327061 so it
took very long to occur. Well, sadly the error did not appear while I am
writing this (because I made some additional change on the log).
I give out either the last line that was send sucessfully and the new line
after the missing one. Both seem to be just normal - nothing is missing.
Thanks for your help!
------------------------------
Date: 6 Apr 2001 21:33:47 GMT (Last modified)
From: Perl-Users-Request@ruby.oce.orst.edu (Perl-Users-Digest Admin)
Subject: Digest Administrivia (Last modified: 6 Apr 01)
Message-Id: <null>
Administrivia:
#The Perl-Users Digest is a retransmission of the USENET newsgroup
#comp.lang.perl.misc. For subscription or unsubscription requests, send
#the single line:
#
# subscribe perl-users
#or:
# unsubscribe perl-users
#
#to almanac@ruby.oce.orst.edu.
NOTE: due to the current flood of worm email banging on ruby, the smtp
server on ruby has been shut off until further notice.
To submit articles to comp.lang.perl.announce, send your article to
clpa@perl.com.
#To request back copies (available for a week or so), send your request
#to almanac@ruby.oce.orst.edu with the command "send perl-users x.y",
#where x is the volume number and y is the issue number.
#For other requests pertaining to the digest, send mail to
#perl-users-request@ruby.oce.orst.edu. Do not waste your time or mine
#sending perl questions to the -request address, I don't have time to
#answer them even if I did know the answer.
------------------------------
End of Perl-Users Digest V10 Issue 8841
***************************************
