[23849] in Perl-Users-Digest
Perl-Users Digest, Issue: 6052 Volume: 10
daemon@ATHENA.MIT.EDU (Perl-Users Digest)
Thu Jan 29 23:06:55 2004
Date: Thu, 29 Jan 2004 20:01:25 -0800 (PST)
From: Perl-Users Digest <Perl-Users-Request@ruby.OCE.ORST.EDU>
To: Perl-Users@ruby.OCE.ORST.EDU (Perl-Users Digest)
Perl-Users Digest Thu, 29 Jan 2004 Volume: 10 Number: 6052
Today's topics:
Yet Another Autoflush problem -- What's wrong with this <jmchambers@rcn.com>
Re: Yet Another Autoflush problem -- What's wrong with <usenet@morrow.me.uk>
Re: Yet Another Autoflush problem -- What's wrong with <jmchambers@rcn.com>
Re: Yet Another Autoflush problem -- What's wrong with (Walter Roberson)
{ipc - windows} -| no such command? <ufo.removethisspamnote@quicknet.nl>
Re: {ipc - windows} -| no such command? <usenet@morrow.me.uk>
Re: {ipc - windows} -| no such command? <uri@stemsystems.com>
Re: {ipc - windows} -| no such command? <ufo.removethisspamnote@quicknet.nl>
Re: {ipc - windows} -| no such command? <usenet@morrow.me.uk>
Re: {ipc - windows} -| no such command? <uri@stemsystems.com>
Digest Administrivia (Last modified: 6 Apr 01) (Perl-Users-Digest Admin)
----------------------------------------------------------------------
Date: Wed, 21 Jan 2004 17:48:50 -0500
From: John Chambers <jmchambers@rcn.com>
Subject: Yet Another Autoflush problem -- What's wrong with this code?
Message-Id: <400f01d2$0$2444$61fed72c@news.rcn.com>
I've grabbed a number of perl TCP server/client pairs, and experimented with
getting them to do some simple request-response sequences. A bizarre flushing
failure has popped up in all of them, and no amount of futzing with $| and
autoflush seems to make them work.
Here's the code for one of the simplest pairs.
=====================================
The TCPserver.pl program is:
#!/usr/local/bin/perl -w
use IO::Socket;
use Net::hostent;
$port = 4217; # pick something not in use
select STDOUT; $| = 1;
($P = $0) =~ s".*/"";
$V = $ENV{"V_$P"} || 2; # Verbose level
$prompt = "Command? ";
$EOL = "\015\012"; # Paranoia
$server = IO::Socket::INET->new( Proto => 'tcp',
LocalPort => $port,
Listen => SOMAXCONN,
Reuse => 1);
die "can't setup server ($!)" unless $server;
print "[Server $0 accepting clients on port $port]$EOL";
while ($client = $server->accept()) {
$client->autoflush(1);
select $client; $| = 1; select STDOUT;
print $client "Welcome to $0; type help for command list.$EOL";
$hostinfo = gethostbyaddr($client->peeraddr);
printf "[Connect from %s]$EOL", $hostinfo->name || $client->peerhost;
select STDOUT;
print "SEND \"$prompt\"$EOL";
print $client $prompt;
print "SENT \"$prompt\"$EOL";
while ($line = <$client>) {
print "RCVD \"$line\"$EOL" if $V>1;
$line =~ s/[\r\n]+$//;
next unless $line; # blank line
# autoflush $client 1; # Does this help? Nope
if ($line =~ /quit|exit/i) { last; }
elsif ($line =~ /date|time/i) { printf $client "%s$EOL", scalar localtime; }
elsif ($line =~ /who/i ) { print $client `who 2>&1`; }
elsif ($line =~ /cookie/i ) { print $client `/usr/games/fortune 2>&1`; }
elsif ($line =~ /motd/i ) { print $client `cat /etc/motd 2>&1`; }
else {
print $client "Commands: quit date who cookie motd$EOL";
}
} continue {
select STDOUT;
print "SEND \"$prompt\"$EOL";
print $client $prompt;
print "SENT \"$prompt\"$EOL";
}
close $client;
}
==============================
And here's the TCPclient.pl program:
#!/usr/local/bin/perl -w
use strict;
use IO::Socket;
my ($host, $port, $kidpid, $server, $line);
my $EOL = "\015\012"; # Paranoia
select STDOUT; $| = 1;
my $P = $0; $P =~ s".*/"";
my $V = $ENV{"V_$P"} || 2; # Verbose level
if (@ARGV <1) {push @ARGV, 'localhost'} # Default to local server
if (@ARGV <2) {push @ARGV, '4217'} # Default port for TCPserver.pl
($host, $port) = @ARGV;
# create a tcp connection to the specified host and port
$server = IO::Socket::INET->new(Proto => "tcp",
PeerAddr => $host,
PeerPort => $port)
or die "can't connect to port $port on $host: $!";
$server->autoflush(1); # so output gets there right away
#autoflush $server 1;
select $server; $| = 1; select STDOUT;
print "[Connected to $host:$port]$EOL";
# split the program into two processes, identical twins
die "can't fork: $!" unless defined($kidpid = fork());
# the if{} block runs only in the parent process
if ($kidpid) { # copy the socket to standard output
print "READ ...$EOL" if $V>1;
while (defined ($line = <$server>)) {
print "RCVD \"$line\"$EOL" if $V>1;
print STDOUT $line;
}
kill("TERM", $kidpid); # send SIGTERM to child
} else { # the else{} block runs only in the child process
# copy standard input to the socket
while (defined ($line = <STDIN>)) {
print "SEND \"$line\"$EOL" if $V>1;
print $server $line;
print "SENT \"$line\"$EOL" if $V>1;
}
}
=====================================
Some readers may recognize these from online sources. Anyway, the Server's
"Command? " prompt is sent to the client, but the client doesn't receive it
at all until the server sends something that ends with a newline (which was
coded "\n" in an earlier version, and "\015" here as a variant). This despite
the setting of $| to 1 for every file in sight, and the use of autoflush(1)
for the sockets also. I tried the autoflush function, too, though it's
commented out here. None of these attempts to subvert the buffering works;
the server's prompt requires a newline for it to be read by the client.
For example, I started the two programs up in two windows, they both printed
their startup messages, the server produced a "SEND ..." and "SENT ..." message
for the prompt, and both were hung. In the client window, I hit Enter, and then
types a "date" command plus an Enter. On the server side, the output was:
=====================================
: ./TCPserver.pl
[Server ./TCPserver.pl accepting clients on port 4217]
[Connect from localhost]
SEND "Command? "
SENT "Command? "
RCVD "
"
SEND "Command? "
SENT "Command? "
RCVD "date
"
SEND "Command? "
SENT "Command? "
=====================================
That looks like exactly what you'd expect.
Meanwhile, over on the client side, what's on the screen is:
=====================================
: ./TCPclient.pl
[Connected to localhost:4217]
READ ...
RCVD "Welcome to ./TCPserver.pl; type help for command list.
"
Welcome to ./TCPserver.pl; type help for command list.
[Here I hit the Enter key]
SEND "
"
SENT "
"
date [Here I sent an actual command]
SEND "date
"
SENT "date
"
RCVD "Command? Command? Wed Jan 21 17:17:03 2004 <=== The prompts finally appear!!
"
Command? Command? Wed Jan 21 17:17:03 2004
=====================================
As you can see here, the client received no input at all until I sent the
"date\n" command. The server ran the "date" command, and sent the results
back to the client. The client recenved the date and time, preceded by
the two "Command? " prompts that it hadn't gotten earlier.
As you can see, I'm familiar with $| and the uses of autoflush. According
to the FAQs, any one of these should suffice to unblock the buffering. But
the data going from TCPserver to TCPclient is bufferred until a newline is
sent.
Is there any way to make the messaging work here?
------------------------------
Date: Wed, 21 Jan 2004 23:00:12 +0000 (UTC)
From: Ben Morrow <usenet@morrow.me.uk>
Subject: Re: Yet Another Autoflush problem -- What's wrong with this code?
Message-Id: <bun09s$i44$2@wisteria.csv.warwick.ac.uk>
John Chambers <jmchambers@rcn.com> wrote:
> I've grabbed a number of perl TCP server/client pairs, and
> experimented with getting them to do some simple request-response
> sequences. A bizarre flushing failure has popped up in all of them,
> and no amount of futzing with $| and autoflush seems to make them
> work.
<snip>
> ==============================
>
> And here's the TCPclient.pl program:
>
<snip>
> # the if{} block runs only in the parent process
> if ($kidpid) { # copy the socket to standard output
> print "READ ...$EOL" if $V>1;
> while (defined ($line = <$server>)) {
Here is your problem. <$server> will not return until it reads a
newline. You either want to set $/ to \1 (which will read a byte at a
tyme: not very efficient) or set non-blobking mode and use
while (read $server, $line, 1024) {
; or maybe sysread instead.
> print "RCVD \"$line\"$EOL" if $V>1;
> print STDOUT $line;
> }
Ben
--
EAT
KIDS (...er, whoops...)
FOR ben@morrow.me.uk
99p
------------------------------
Date: Fri, 23 Jan 2004 15:54:54 -0500
From: John Chambers <jmchambers@rcn.com>
Subject: Re: Yet Another Autoflush problem -- What's wrong with this code?
Message-Id: <40118a1e$0$12721$61fed72c@news.rcn.com>
Ben Morrow wrote:
>># the if{} block runs only in the parent process
>>if ($kidpid) { # copy the socket to standard output
>> print "READ ...$EOL" if $V>1;
>> while (defined ($line = <$server>)) {
>
>
> Here is your problem. <$server> will not return until it reads a
> newline. You either want to set $/ to \1 (which will read a byte at a
> tyme: not very efficient) or set non-blobking mode and use
>
> while (read $server, $line, 1024) {
>
> ; or maybe sysread instead.
Well, I was wondering about that. I grepped and googled for
everything I could find on the topic, and found lots and lots
of advice that !| or one of the autoflush() calls would solve
all my problems. I kept thinking that those undo the buffering
on the sending end, but I don't see any evidence that it can't
also be a problem on the receiving end.
So I guess all those FAQs and RTFMs are just red herrings, and
I was guessing right all along. I wonder why I never ran across
any comments about this? Others have had to stumbled across the
same problem. There's gotta be a lot of people trying to send
data across TCP links in perl, right? And data isn't always in
the form of ASCII text with newlines at the end of every data
object, right?
Anyway, thanks for the advice. I think I'll try setting nonblocking
and use sysread(). Maybe I can copy some of my C code, and add a
few $'s, to get the corresponding perl code. Or maybe I won't figure
out how to set nonblocking, and I'll be back with another dumb
question soon. ;-)
------------------------------
Date: 24 Jan 2004 05:32:58 GMT
From: roberson@ibd.nrc-cnrc.gc.ca (Walter Roberson)
Subject: Re: Yet Another Autoflush problem -- What's wrong with this code?
Message-Id: <but02a$k16$1@canopus.cc.umanitoba.ca>
In article <40118a1e$0$12721$61fed72c@news.rcn.com>,
John Chambers <jmchambers@rcn.com> wrote:
:Ben Morrow wrote:
:> Here is your problem. <$server> will not return until it reads a
:> newline. You either want to set $/ to \1 (which will read a byte at a
:> tyme: not very efficient) or set non-blobking mode and use
:> while (read $server, $line, 1024) {
:> ; or maybe sysread instead.
:So I guess all those FAQs and RTFMs are just red herrings, and
:I was guessing right all along. I wonder why I never ran across
:any comments about this? Others have had to stumbled across the
:same problem. There's gotta be a lot of people trying to send
:data across TCP links in perl, right? And data isn't always in
:the form of ASCII text with newlines at the end of every data
:object, right?
The behaviour is deducible, by reading in perlops that <handle>
is equivilent to readline(*handle) and then reading the perlfunc
documentation of readline as returning the next line delimited
by the current record separator.
More explicit discussion appears in the perlipc documentation in
the section "Interactive Client with IO::Socket"
If the remote server sends data a byte at time, and you need
that data immediately without waiting for a newline (which
might not happen), you may wish to replace the "while" loop
in the parent with the following:
my $byte;
while (sysread($handle, $byte, 1) == 1) {
print STDOUT $byte;
}
Making a system call for each byte you want to read is not
very efficient (to put it mildly) but is the simplest to
explain and works reasonably well.
They are, as they hint, hiding complications in that reading of a
single byte. The sysread() call can potentially return anywhere -up-
to the number of bytes you ask for, dependant on internal buffering.
The maximum number of bytes of pipe data that are guaranteed to be handled
atomically is 512 for any POSIX compliant system; the SGI IRIX systems
I use most provide up to 10240 bytes atomically.
(For more information about these limits on a POSIX system, see
<limits.h> and the sysconf() and pathconf() calls, and the
constants _POSIX_PIPE_BUF and PIPE_BUF .)
When you are reading from a device (e.g., a tty) instead of a pipe, then
all guarantees about minimum I/O sizes are off: essentially you get
whatever became available between interrupts. When data is flowing
at a steady rate and you are in a tight loop, the read sizes tend to
be consistant, but not necessarily very big, and the first read
isn't necessarily going to be the same size. In one of the tests I
did, I tended to get 3 bytes on the first read and then groups of 10
bytes when running at 38400 bps across a serial port... but
that was only a tendancy, and it varied with load and was
pom-dependent.
The net result of these uncertainties is that if you are using
sysread() with a buffer size greater than 1 and there is any
possibility that the data might not arrive atomically, then you need
to explicitly examine the number of bytes that arrived and cycle back
until you get the full number of bytes that you want to read in
this trip. As sysread() does not pay attention to line boundaries,
you will probably need to decrement your buffer size by the number
of bytes that arrived, and probably need to munge the offset within
the buffer to be stored into. Once you've seen the resulting code
a couple of times, it's easy to recognize.
--
How does Usenet function without a fixed point?
------------------------------
Date: Sat, 24 Jan 2004 11:23:01 +0100
From: Steven Mocking <ufo.removethisspamnote@quicknet.nl>
Subject: {ipc - windows} -| no such command?
Message-Id: <1014hs8ahqjnt17@corp.supernews.com>
According to perlipc you can open filehandles to the "filenames" -| and |-
for forking a child and connecting the filehandle to either the child's
STDOUT or STDIN and that works perfectly under unices.
$pid = open(KID_TO_READ, "-|");
I'm writing a script which needs to work on Windows machines as well. Now
when i try to use that construct it dies with the dreaded
no-such-internal-or-external-command-error. Is there any way to do this in
a portable fashion? Or at least in an if($^O eq "MSWin32") {} block??
The purpose of this is scanning a subnet for peer-to-peer webserver-thingies
like the script itself and making a map of the entire network. This is the
course of action when there's no list of hosts specified and the router
doesn't support multicasting (gah). Depending on circumstances, quite some
hosts might need to be tried so it gets awfully slow with only one thread
using IO::Socket::INET (which, despite the manpage, has the timeout code
commented out by the way).
Thanks in advance,
Steven Mocking.
--
I never forget a face, but in your case I'll make an exception.
-- Groucho Marx
------------------------------
Date: Sat, 24 Jan 2004 12:24:10 +0000 (UTC)
From: Ben Morrow <usenet@morrow.me.uk>
Subject: Re: {ipc - windows} -| no such command?
Message-Id: <buto5a$9be$3@wisteria.csv.warwick.ac.uk>
Steven Mocking <ufo.removethisspamnote@quicknet.nl> wrote:
> According to perlipc you can open filehandles to the "filenames" -| and |-
> for forking a child and connecting the filehandle to either the child's
> STDOUT or STDIN and that works perfectly under unices.
>
> $pid = open(KID_TO_READ, "-|");
>
> I'm writing a script which needs to work on Windows machines as well. Now
> when i try to use that construct it dies with the dreaded
> no-such-internal-or-external-command-error. Is there any way to do this in
> a portable fashion? Or at least in an if($^O eq "MSWin32") {} block??
Have you tried the most recent version of Perl? I think this has
improved quite a lot on Win32 in 5.8.
Alternatively, you can use Win32::Process to invoke a command with the
same std filehandles as the current process.
Ben
--
For the last month, a large number of PSNs in the Arpa[Inter-]net have been
reporting symptoms of congestion ... These reports have been accompanied by an
increasing number of user complaints ... As of June,... the Arpanet contained
47 nodes and 63 links. [ftp://rtfm.mit.edu/pub/arpaprob.txt] * ben@morrow.me.uk
------------------------------
Date: Sat, 24 Jan 2004 15:50:14 GMT
From: Uri Guttman <uri@stemsystems.com>
Subject: Re: {ipc - windows} -| no such command?
Message-Id: <x7u12lz6q1.fsf@mail.sysarch.com>
>>>>> "SM" == Steven Mocking <ufo.removethisspamnote@quicknet.nl> writes:
SM> According to perlipc you can open filehandles to the "filenames" -| and |-
SM> for forking a child and connecting the filehandle to either the child's
SM> STDOUT or STDIN and that works perfectly under unices.
SM> $pid = open(KID_TO_READ, "-|");
SM> I'm writing a script which needs to work on Windows machines as
SM> well. Now when i try to use that construct it dies with the
SM> dreaded no-such-internal-or-external-command-error. Is there any
SM> way to do this in a portable fashion? Or at least in an if($^O eq
SM> "MSWin32") {} block??
that construct isn't supported on windows
read perlopentut which covers a way around it. unfortunately only one of
the directions in the examples seems to work. the other is either a perl
bug or a doc bug. i have reported this to p5p but with not much reaction
from them.
uri
--
Uri Guttman ------ uri@stemsystems.com -------- http://www.stemsystems.com
--Perl Consulting, Stem Development, Systems Architecture, Design and Coding-
Search or Offer Perl Jobs ---------------------------- http://jobs.perl.org
------------------------------
Date: Mon, 26 Jan 2004 23:42:51 +0100
From: Steven Mocking <ufo.removethisspamnote@quicknet.nl>
Subject: Re: {ipc - windows} -| no such command?
Message-Id: <101b5vdli7dtab0@corp.supernews.com>
Ben Morrow wrote:
>
> Steven Mocking <ufo.removethisspamnote@quicknet.nl> wrote:
>> According to perlipc you can open filehandles to the "filenames" -| and
>> |- for forking a child and connecting the filehandle to either the
>> child's STDOUT or STDIN and that works perfectly under unices.
>>
>> $pid = open(KID_TO_READ, "-|");
>>
>> I'm writing a script which needs to work on Windows machines as well. Now
>> when i try to use that construct it dies with the dreaded
>> no-such-internal-or-external-command-error. Is there any way to do this
>> in a portable fashion? Or at least in an if($^O eq "MSWin32") {} block??
>
> Have you tried the most recent version of Perl? I think this has
> improved quite a lot on Win32 in 5.8.
I'm using 5.8 all along. The fork emulation works, but not this.
>
> Alternatively, you can use Win32::Process to invoke a command with the
> same std filehandles as the current process.
Hm I don't think that would be a good idea. I actually need refs to the
STDOUTs of the children in an array:
Bit shortened:
for(my $i = 1; $i < 20; $i++)
{
my $handle[$i] = new IO::File;
my $pid = ($handle[$i]->open("-|"));
next if ($pid); # Parent? Next patient!
# <snip>Scan a port on 192.168.1.$i for a certain server...
}
# <snip> process the results
--
Sight is a faculty; seeing is an art.
------------------------------
Date: Mon, 26 Jan 2004 23:09:22 +0000 (UTC)
From: Ben Morrow <usenet@morrow.me.uk>
Subject: Re: {ipc - windows} -| no such command?
Message-Id: <bv46n2$q0b$1@wisteria.csv.warwick.ac.uk>
Steven Mocking <ufo.removethisspamnote@quicknet.nl> wrote:
> Ben Morrow wrote:
> > Have you tried the most recent version of Perl? I think this has
> > improved quite a lot on Win32 in 5.8.
>
> I'm using 5.8 all along. The fork emulation works, but not this.
OK, right.
> > Alternatively, you can use Win32::Process to invoke a command with the
> > same std filehandles as the current process.
>
> Hm I don't think that would be a good idea. I actually need refs to the
> STDOUTs of the children in an array:
I think it's the only answer. Hang on... you're not execing anything,
just using forking open? Right, that's easier.
> for(my $i = 1; $i < 20; $i++)
> {
> my $handle[$i] = new IO::File;
This is never going to work. IO::File is for *file*-handles, and what
you have here is a pipe.
> my $pid = ($handle[$i]->open("-|"));
> next if ($pid); # Parent? Next patient!
> # <snip>Scan a port on 192.168.1.$i for a certain server...
> }
# untested:
my @handle;
for my $i (1..20) {
{
my $TMP;
pipe $handle[$i], $TMP or die "pipe failed: $!";
my $pid = fork;
defined $pid or die "fork failed: $!";
next if $pid;
open STDOUT, ">&=", $TMP or die "dup2 failed: $!";
}
# scan for server
}
If you'd rather use the OO stuff (can't see why, but just in case...)
my @handle;
for my $i (1..20) {
$handle[$i] = new IO::Pipe or die "pipe failed: $!";
my $pid = fork;
defined $pid or die "fork failed: $!";
if ($pid) {
$handle[$i]->reader;
next;
}
$handle[$i]->writer;
open STDOUT, ">&=", fileno $handle[$i]
or die "dup2 failed: $!";
# scan
}
Ben
--
Musica Dei donum optimi, trahit homines, trahit deos. |
Musica truces mollit animos, tristesque mentes erigit. | ben@morrow.me.uk
Musica vel ipsas arbores et horridas movet feras. |
------------------------------
Date: Mon, 26 Jan 2004 23:19:13 GMT
From: Uri Guttman <uri@stemsystems.com>
Subject: Re: {ipc - windows} -| no such command?
Message-Id: <x74qui9u32.fsf@mail.sysarch.com>
>>>>> "SM" == Steven Mocking <ufo.removethisspamnote@quicknet.nl> writes:
>> Have you tried the most recent version of Perl? I think this has
>> improved quite a lot on Win32 in 5.8.
SM> I'm using 5.8 all along. The fork emulation works, but not this.
did you read the section on that in perlopentut? i told you open | is
not supported under winblows. and there are workaround subs in that doc
(of which one works and the other doesn't according to my tests).
uri
--
Uri Guttman ------ uri@stemsystems.com -------- http://www.stemsystems.com
--Perl Consulting, Stem Development, Systems Architecture, Design and Coding-
Search or Offer Perl Jobs ---------------------------- http://jobs.perl.org
------------------------------
Date: 6 Apr 2001 21:33:47 GMT (Last modified)
From: Perl-Users-Request@ruby.oce.orst.edu (Perl-Users-Digest Admin)
Subject: Digest Administrivia (Last modified: 6 Apr 01)
Message-Id: <null>
Administrivia:
#The Perl-Users Digest is a retransmission of the USENET newsgroup
#comp.lang.perl.misc. For subscription or unsubscription requests, send
#the single line:
#
# subscribe perl-users
#or:
# unsubscribe perl-users
#
#to almanac@ruby.oce.orst.edu.
NOTE: due to the current flood of worm email banging on ruby, the smtp
server on ruby has been shut off until further notice.
To submit articles to comp.lang.perl.announce, send your article to
clpa@perl.com.
#To request back copies (available for a week or so), send your request
#to almanac@ruby.oce.orst.edu with the command "send perl-users x.y",
#where x is the volume number and y is the issue number.
#For other requests pertaining to the digest, send mail to
#perl-users-request@ruby.oce.orst.edu. Do not waste your time or mine
#sending perl questions to the -request address, I don't have time to
#answer them even if I did know the answer.
------------------------------
End of Perl-Users Digest V10 Issue 6052
***************************************
