[21746] in Perl-Users-Digest
Perl-Users Digest, Issue: 3950 Volume: 10
daemon@ATHENA.MIT.EDU (Perl-Users Digest)
Thu Oct 10 18:06:03 2002
Date: Thu, 10 Oct 2002 15:05:14 -0700 (PDT)
From: Perl-Users Digest <Perl-Users-Request@ruby.OCE.ORST.EDU>
To: Perl-Users@ruby.OCE.ORST.EDU (Perl-Users Digest)
Perl-Users Digest Thu, 10 Oct 2002 Volume: 10 Number: 3950
Today's topics:
Re: flock related issue <pobugfix@peterlink.ru>
Re: flock related issue <camerond@mail.uca.edu>
Re: flock related issue <zoips@eesoft.org>
Re: flock related issue <jhalpin@nortelnetworks.com_.nospam>
Re: flock related issue <zoips@eesoft.org>
Re: How to test for blessedness? <comdog@panix.com>
Re: How to test for blessedness? <comdog@panix.com>
Re: How to test for blessedness? <dd@4pro.net>
Re: How to test for blessedness? <dd@4pro.net>
Re: ithreads, perl 5.8 and shared objects <goldbb2@earthlink.net>
Re: ithreads, perl 5.8 and shared objects <uri@stemsystems.com>
Re: join lines in @array <No_Mail_Address@cox.net>
Re: join lines in @array <uri@stemsystems.com>
Re: Multiple Pings/Second <troc@netrus.net>
Re: Multiple Pings/Second <flavell@mail.cern.ch>
Re: Multiple Pings/Second <goldbb2@earthlink.net>
Re: NDBM_File <danb@mail.dnttm.ro>
Need to convert from seconds since Unix epoch to GMT (J Hall)
Net::FTP: Unexpected EOF on command channel (jend)
Re: Perl Journal? (David Combs)
Digest Administrivia (Last modified: 6 Apr 01) (Perl-Users-Digest Admin)
----------------------------------------------------------------------
Date: Thu, 10 Oct 2002 22:11:02 +0400
From: "Andrew V. Tkachenko" <pobugfix@peterlink.ru>
Subject: Re: flock related issue
Message-Id: <ao4fkp$502$1@news.sovam.com>
Matt Eberts wrote:
> I'm trying to figure out if this is a bug in ActiveState Perl, or if
> it's because on Linux Perl is ignoring the file lock.
>
> Here is the simplest test case. On Win2k running ActiveState Perl 5.6.1,
> the script will die attempting to copy test.xml.temp back over test.xml.
> However, on Linux, Perl version 5.6.1, it copies the file over just fine.
>
> #!/usr/bin/perl
>
> use Fcntl qw(:DEFAULT :flock);
> use File::Copy;
>
> sysopen(FILE, "test.xml", O_RDWR) or die("Unable to open test.xml");
> flock(FILE, LOCK_EX);
>
> copy("test.xml", "test.xml.temp") or die("Unable to copy test.xml to
> test.xml.temp");
> copy("test.xml.temp","test.xml") or die("Unable to copy test.xml.temp to
> test.xml");
>
> close(FILE);
>
> Am I misunderstanding some basic concept of how flock works, or is this
> really a bug in ActiveState's Perl implementation?
>
> Thanks.
>
As I know, 'flock' is not supported on win32 platform. So you'll have to
do locking manually.
Regards, Andrew
------------------------------
Date: Thu, 10 Oct 2002 14:47:15 -0500
From: Cameron Dorey <camerond@mail.uca.edu>
Subject: Re: flock related issue
Message-Id: <3DA5D943.4070706@mail.uca.edu>
Andrew V. Tkachenko wrote:
> Matt Eberts wrote:
>
>> I'm trying to figure out if this is a bug in ActiveState Perl, or if
>> it's because on Linux Perl is ignoring the file lock.
>>
>> Here is the simplest test case. On Win2k running ActiveState Perl
>> 5.6.1, the script will die attempting to copy test.xml.temp back over
>> test.xml. However, on Linux, Perl version 5.6.1, it copies the file
>> over just fine.
>>
>> #!/usr/bin/perl
>>
>> use Fcntl qw(:DEFAULT :flock);
>> use File::Copy;
>>
>> sysopen(FILE, "test.xml", O_RDWR) or die("Unable to open test.xml");
>> flock(FILE, LOCK_EX);
>>
>> copy("test.xml", "test.xml.temp") or die("Unable to copy test.xml to
>> test.xml.temp");
>> copy("test.xml.temp","test.xml") or die("Unable to copy test.xml.temp
>> to test.xml");
>>
>> close(FILE);
>>
>> Am I misunderstanding some basic concept of how flock works, or is
>> this really a bug in ActiveState's Perl implementation?
>>
>> Thanks.
>>
>
> As [far as] I know, 'flock' is not supported on win32 platform. So you'll have to
> do locking manually.
This is not correct. flock is not supported under Win9x, but is just
fine under NT, 2K, and (I figure) XP. If it *weren't* supported on 2K,
you'd copy the file just fine (and someone else could write to it, too).
In order to write to the file here (i.e., copy to it in your case), you
will have to close the filehandle to unlock it first. It's not due to a
problem with AS Perl, but the way that Window$ locks files (I seem to
remember that it's this way on VMS, too, but don't jump on me if it isn't).
Cameron
--
Cameron Dorey
Associate Professor of Chemistry
University of Central Arkansas
Phone: 501-450-5938
camerond@mail.uca.edu
------------------------------
Date: Thu, 10 Oct 2002 19:47:39 GMT
From: Matt Eberts <zoips@eesoft.org>
Subject: Re: flock related issue
Message-Id: <vPkp9.20240$lV3.1958689@newsread1.prod.itd.earthlink.net>
>
> As I know, 'flock' is not supported on win32 platform. So you'll have to
> do locking manually.
>
> Regards, Andrew
>
I've tested it, and the files do get locked (aside from the fact that the
Java applications launched by the Perl script are complaining at the moment
that the file has been locked, other instances of the Perl interpreter
running that script will sit and wait for the file to become unlocked). But
I think you are right in the sense that file locking through flock is
broken.
Matt
------------------------------
Date: 10 Oct 2002 14:40:41 -0500
From: Joe Halpin <jhalpin@nortelnetworks.com_.nospam>
Subject: Re: flock related issue
Message-Id: <yxs7ofa2m69x.fsf@nortelnetworks.com_.nospam>
Matt Eberts <zoips@eesoft.org> writes:
> I'm trying to figure out if this is a bug in ActiveState Perl, or if
> it's because on Linux Perl is ignoring the file lock.
>
> Here is the simplest test case. On Win2k running ActiveState Perl
> 5.6.1, the script will die attempting to copy test.xml.temp back
> over test.xml. However, on Linux, Perl version 5.6.1, it copies the
> file over just fine.
>
> #!/usr/bin/perl
>
> use Fcntl qw(:DEFAULT :flock);
> use File::Copy;
>
> sysopen(FILE, "test.xml", O_RDWR) or die("Unable to open test.xml");
> flock(FILE, LOCK_EX);
>
> copy("test.xml", "test.xml.temp") or die("Unable to copy test.xml to
> test.xml.temp");
> copy("test.xml.temp","test.xml") or die("Unable to copy test.xml.temp to
> test.xml");
>
> close(FILE);
>
> Am I misunderstanding some basic concept of how flock works, or is
> this really a bug in ActiveState's Perl implementation?
It's been a long time since I knew much about Windows, but I recall
being surprised about how locks work there. You might try asking
someone with Windows knowledge about this.
Joe
------------------------------
Date: Thu, 10 Oct 2002 19:51:52 GMT
From: Matt Eberts <zoips@eesoft.org>
Subject: Re: flock related issue
Message-Id: <sTkp9.20246$lV3.1958689@newsread1.prod.itd.earthlink.net>
> In order to write to the file here (i.e., copy to it in your case), you
> will have to close the filehandle to unlock it first. It's not due to a
> problem with AS Perl, but the way that Window$ locks files (I seem to
> remember that it's this way on VMS, too, but don't jump on me if it
> isn't).
Arg. Okay. Hmm...
Thanks.
Matt
------------------------------
Date: Thu, 10 Oct 2002 15:37:45 -0500
From: brian d foy <comdog@panix.com>
Subject: Re: How to test for blessedness?
Message-Id: <101020021537459069%comdog@panix.com>
In article <ao42e7$irg4i$1@ID-159100.news.dfncis.de>, Domizio Demichelis <dd@4pro.net> wrote:
> perldoc -f ref
ref() won't tell you if something is blessed. you would have to do extra
work to check the return value.
see the blessed() function in Scalar::Util.
--
brian d foy <comdog@panix.com> - Perl services for hire
The Perl Review - a new magazine devoted to Perl
<http://www.theperlreview.com>
------------------------------
Date: Thu, 10 Oct 2002 15:44:45 -0500
From: brian d foy <comdog@panix.com>
Subject: Re: How to test for blessedness?
Message-Id: <101020021544454278%comdog@panix.com>
In article <ao42lp$j2b8e$1@ID-159100.news.dfncis.de>, Domizio Demichelis <dd@4pro.net> wrote:
> perldoc UNIVERSAL
> use UNIVERSAL;
you don't need to use UNIVERSAL. it's automatically at the end of @ISA, too.
> UNIVERSAL::isa($ref, "class"); #return true if $ref is a blessed ref of
> class "class"
isa() works for non-blessed things, too:
UNIVERSAL::isa( [], 'ARRAY' )
see the blessed() function in Scalar::Util instead. :)
--
brian d foy <comdog@panix.com> - Perl services for hire
The Perl Review - a new magazine devoted to Perl
<http://www.theperlreview.com>
------------------------------
Date: Thu, 10 Oct 2002 16:55:35 -0400
From: "Domizio Demichelis" <dd@4pro.net>
Subject: Re: How to test for blessedness?
Message-Id: <ao4pha$jbrqb$1@ID-159100.news.dfncis.de>
> see the blessed() function in Scalar::Util instead. :)
nice... I didn't know it. Why it is not in the standard distribution of
perl?
--
-.. --- -- .. --.. .. ---
-.. . -- .. -.-. .... . .-.. .. ...
------------------------------
Date: Thu, 10 Oct 2002 17:06:29 -0400
From: "Domizio Demichelis" <dd@4pro.net>
Subject: Re: How to test for blessedness?
Message-Id: <ao4q5o$jb9uc$1@ID-159100.news.dfncis.de>
> nice... I didn't know it. Why it is not in the standard distribution of
> perl?
oops... it's in the 5.8
I need to upgrade :-/
--
-.. --- -- .. --.. .. ---
-.. . -- .. -.-. .... . .-.. .. ...
------------------------------
Date: Thu, 10 Oct 2002 15:41:19 -0400
From: Benjamin Goldberg <goldbb2@earthlink.net>
Subject: Re: ithreads, perl 5.8 and shared objects
Message-Id: <3DA5D7DF.D1E244BB@earthlink.net>
Ilya Zakharevich wrote:
> [A complimentary Cc of this posting was sent to
> Benjamin Goldberg <goldbb2@earthlink.net>],
Please don't.
> who wrote in article <3DA47FEC.CDEB13CC@earthlink.net>:
> > > > > Now suppose two modules - which do not know about each other -
> > > > > try to subclass select(). How would they do it?
> > > >
> > > > That's not how you would do it. Let's suppose you have written...
> > >
> > > I do not discuss applications. I discuss two *modules*. Both use
> > > POE. Both substitute their own event loop.
> >
> > If they use POE, they should use POE's event loop.
>
> > Stubstituting their own event loops is a *dumb* thing to do in
> > general,
>
> Maybe I use this "substitute" word incorrect? I see no way how a
> serious project can work without replacing the select()-based loop by
> something more capable.
>
> > and should only be done if no other way will work.
>
> I gave you examples (Tk and a wait()-based loop) where no other way
> will work.
>
> > > You have a BourneShell module. This module is given a program (a
> > > string) of a form
> > >
> > > (( a && ( b || c ) ) || (e && (f && h)) & (x || y) ; z ) |
> > > (zz || v zzz)
> > >
> > > and starts the applications *without any forking*. The event loop
> > > of this module is based on wait().
> >
> > Why not wait for EOF from the STDOUT of the last process in the
> > chain?
>
> Hmm? The STDOUT of the last process is not redirected... *There is*
> no last process in the chain. BourneShell runs several of the a..zzz
> processes asyncroneously.
For each group of asynchronous processes, there is a last one in that
chain. If you were to have a pipe from that last process to your own,
and copy the output from that last process to your own stdout, you could
wait for eof from that process.
Thinking on this some more, I do realize that this wouldn't really work
all that well, since it's more efficient, and prefered, for processes to
inherit your own std streams (especially if one of them is a tty) and
talk to them directly.
> When it sees that b ended unsuccessfully, it starts c with necessary
> redirections.
>
> > POE cannot use an externally supplied event loop except via a bridge
> > class, such as the one which allows it to use Tk's event loop.
>
> I read it as "POE can use an externally supplied event loop". Right?
Yes, if those externally supplied event loops will indicate readability
and writability on filehandles.
That is, it must do *at least* as much as POE's own event loop would do.
> > POE can only use one externally supplied event loop at a time.
>
> This is exactly my point. POE does not scale to more than one
> "extravagant" client. This makes it a toy only. 640K should be
> enough for everything mentality.
>
> The sad think is that I thought about this, and I saw no way to design
> a framework a subclassable event loop - even with best support from
> most advanced OSes... I saw several solutions which are pretty close
> to be implementable, but I was pretty sure that on legacy systems they
> will just testsuites for kernel bugs...
>
> The problem: there is a dispatcher X which knows how to multiplex
> events of types A,B,C; there is another dispatcher Y which knows how
> to multiplex events of types A,D,E. Design the API which allows them
> to cooperate with the dispatch kernel so that the kernel+X+Y can
> multiplex A,B,C,D,E. X and Y know everything about the kernel; X and
> Y do not know anything about each other; kernel know only the things
> X and Y pass it via the API.
Considering more real-worldish situations, let's consider the following
event types:
F: filehandle ready for reading/writing (select)
P: process has die (wait)
L: locking events (flock)
S: signals
R: RPC (using semaphores and shared memory)
[wouldn't it be nice if we had an event system which handled *all* of
these? I may as well wish for the moon, considering how likely that is.]
POE handles events of type F.
BourneShell handles events of type P.
POE can only use BourneShell as a backend, with a bridge, if it *at
least* handles events of type F. If BS handled both F and P, then it
would be doable, otherwise no.
Now, SMTT need events of some other type F, and lets you replace it's
event loop with any other event loop which supports type F ... since POE
always supports events of type F, you can use POE as a backend.
If you want to use another module's event loop as a backend, using a POE
bridge class, that backend *must* support events of type F (it may do
other things as well, such as P, but F is a minimum).
> > Ok, so ShowMeTheTree has it's own bridge class to use POE (whereas
> > BourneShell had a POE bridge class so POE can use *it*).
>
> > > I think: now I can use these two modules and write a debugger for
> > > bourne shell programs which is essentially a Perl one-liner. Can
> > I?
>
> > If POE is using BS's event loop, and SMTT is using the normal
> > interface to POE's event loop (which gets routed through the bridge
> > to BS), then it might indeed be possible.
>
> BS handles wait() events only.
Then you can't use BS as POE's backend.
If BS were designed to handle wait() events and select() events, then
you could use BS as a backend for POE.
> It does not know that somebody may need to examine the GUI event queue
> of Win32.
How are Win32 GUI events done? Type R, maybe?
> Likewise SMTT does not know that somebody may want to
> wait()...
Fortunatly, SMTT does not need to know about someone wanting to wait()
if it's using POE for it's backend, since POE provides a kind of
contract, in that it will make callbacks for whatever F type events you
tell it to watch for.
--
my $n = 2; print +(split //, 'e,4c3H r ktulrnsJ2tPaeh'
."\n1oa! er")[map $n = ($n * 24 + 30) % 31, (42) x 26]
------------------------------
Date: Thu, 10 Oct 2002 20:00:36 GMT
From: Uri Guttman <uri@stemsystems.com>
Subject: Re: ithreads, perl 5.8 and shared objects
Message-Id: <x7vg4anjx8.fsf@mail.sysarch.com>
>>>>> "BG" == Benjamin Goldberg <goldbb2@earthlink.net> writes:
BG> Considering more real-worldish situations, let's consider the following
BG> event types:
BG> F: filehandle ready for reading/writing (select)
BG> P: process has die (wait)
BG> L: locking events (flock)
BG> S: signals
BG> R: RPC (using semaphores and shared memory)
BG> [wouldn't it be nice if we had an event system which handled *all* of
BG> these? I may as well wish for the moon, considering how likely that is.]
BG> POE handles events of type F.
BG> BourneShell handles events of type P.
if you handle sigchld then you can handle type P as well. AFAIK bourne
shell is not a real event system as it just does a blocking wait on
child procs. event systems handle things asynchronously and multiple
types of things.
rpc via shared memory can be done with sockets for the semaphores so any
normal event loop could support that. in fact sockets as a replacement
for semaphores works well in a threaded system too. i did just that in a
very complex single event loop process in c. a pipe (from the process to
itself!) was used to synchronize a thread worker farm to the main event
loop. worked like a charm.
locking can be simulated with serializing external procs (or even
threads in some cases). see the other posts in this thread about
that. the pipe idea mentioned above can also do that.
so any decent event loop can do all of those. stem handles them all
right now and it piggybacks on event.pm.
uri
--
Uri Guttman ------ uri@stemsystems.com -------- http://www.stemsystems.com
----- Stem and Perl Development, Systems Architecture, Design and Coding ----
Search or Offer Perl Jobs ---------------------------- http://jobs.perl.org
------------------------------
Date: Thu, 10 Oct 2002 20:04:17 GMT
From: Fred <No_Mail_Address@cox.net>
Subject: Re: join lines in @array
Message-Id: <3DA5DD6A.D14DA9FE@cox.net>
Brian McCauley wrote:
>
> Fred <No_Mail_Address@cox.net> writes:
>
> > I want to join some lines from mail-headers into one value of an
> > array, like:
> >
> > From: somebody == Date: somedate == X-Mailer: Mozilla 4.04 [en]
> > (Win95; I) ==
> >
> > But I always end up with a separate value for every header line:
>
> Simply wanting to do something is not enough. You have to actually do
> it. To join the elements of a list into a single string you can use the
> join() function. Alternatively use the .= operator.
>
> --
OK - I used the .= operator on the individual lines, but still no joy
(same result as before):
while (<IN>) {
if (m/^From -.*?\s/i .. /^$/) {
next unless /^(?:From:.+|Date:.+|X-Mailer|From -)/i ;
if (s/^(From:.+|Date:.+|X-Mailer:.+)/$1 == /i ) {
chomp ;
$mylist .= $_ ;
}
push (@bigarray, $mylist) ;
$mylist = "" ;
}
}
foreach (@bigarray) {print OUT $_,"\n" };
--
Fred
------------------------------
Date: Thu, 10 Oct 2002 20:10:47 GMT
From: Uri Guttman <uri@stemsystems.com>
Subject: Re: join lines in @array
Message-Id: <x7it0anjg9.fsf@mail.sysarch.com>
>>>>> "F" == Fred <No_Mail_Address@cox.net> writes:
F> OK - I used the .= operator on the individual lines, but still no joy
F> (same result as before):
F> while (<IN>) {
F> if (m/^From -.*?\s/i .. /^$/) {
F> next unless /^(?:From:.+|Date:.+|X-Mailer|From -)/i ;
F> if (s/^(From:.+|Date:.+|X-Mailer:.+)/$1 == /i ) {
don't do a test and then a s/// on the same thing. you can test s/// for
success
next unless s/^(From:.+|Date:.+|X-Mailer:.+)/$1 == /i ;
F> chomp ;
F> $mylist .= $_ ;
that looks fine. you don't show what the input and output are so it is
hard to figure out what is wrong.
'same result as before' is not helpful.
F> }
F> push (@bigarray, $mylist) ;
F> $mylist = "" ;
F> }
F> }
there are modules which mung standard mail headers. you should look into
them as they do all you want and are easy to use. search cpan for them.
uri
--
Uri Guttman ------ uri@stemsystems.com -------- http://www.stemsystems.com
----- Stem and Perl Development, Systems Architecture, Design and Coding ----
Search or Offer Perl Jobs ---------------------------- http://jobs.perl.org
------------------------------
Date: Thu, 10 Oct 2002 18:24:28 -0000
From: Rocco Caputo <troc@netrus.net>
Subject: Re: Multiple Pings/Second
Message-Id: <slrnaqbheb.8ma.troc@eyrie.homenet>
On Thu, 10 Oct 2002 02:05:20 -0400, Benjamin Goldberg wrote:
> Rocco Caputo wrote:
>> See http://poe.perl.org/?POE_Cookbook/Pinging_Multiple_Hosts for a
>> very simple example of pinging several hosts at once from a single
>> process. Be aware, though, that it pings its list of hosts all at
>> once, which won't work in your case.
>
> First -- why do you say that pinging the whole list at once won't work?
I've seen it fail for UDP, which is about as unreliable as ICMP.
One guy wrote a POE program to poll his corporate network's Norton
AntiVirus servers. NAV servers talk via UDP, which we've already
established stands for Undeliverable Data Packets.
I know, I know. It really means User Datagram Protocol.
Anyway, he shotgunned several hundred of these UDP packets onto his
company's network and dutifully waited the timeout period for his
responses. The problem was, only a few percent of his packets were
acknowledged.
The first solution he came up with was to just resend the
unacknowledged requests again and again if necessary. He counted the
number number of times they were sent so he could eventually bail out
on ones that would never answer.
His program took a very long time, though, because it would send a
full volley of requests, wait the full timeout period, and in the end
only complete a small number of transactions. It needed to do this
many times to go through his list of hosts. It also generated several
false negatives because most of the requests were repeated many times.
The main cause of his problem turned out to be that he wasn't
receiving responses and sending requests at the same time. He was
using a simple loop to transmit all the requests together, then he was
listening for responses. As a result, the operating system's receive
queue for his socket would fill up, and the OS would silently drop the
overflow.
> Second -- who says you *have* to feed it the whole list at once? You
> could feed it some reasonable sized block, wait for them to respond or
> time out, and then send more.
Nobody says the requests need to be fed all at once, but the program I
cited does this. I felt the need to point out why this won't work.
Anyway, I covered request throttling next.
>> The problem there is that ICMP isn't a reliable protocol. 1500 hosts
>> over 60 seconds is an average of 25 ICMP echo/reply pairs (50 packets)
>> per second. Some packets are bound to get lost in the shuffle,
>> resulting in erratic host statuses.
>
> The obvious thing to do is send your pings as fast as you can, read as
> many responses as you can within a timeout, and then resend to those who
> didn't respond.
Reliability aside, it doesn't seem very responsible to squirt packets
onto a network faster than the network (the local interface included)
can handle them. The result is a program that generates more traffic
and takes longer than necessary.
To be fair, ICMP packets tend to be smaller than UDP ones, so the
amount of redundant traffic in each volley will be lower.
>> You may be able to send far fewer than 1500 echo requests if your
>> operating system and routers allow you to ping subnet broadcast
>> addresses.
>
> Since he hasn't said that he's pinging broadcast addresses, I would
> *assume* that he's pinging actual hosts. Thus, this shouldn't be
> relevant (I think).
I didn't know for sure what he was doing, so I presented this as just
one option among others. After all I've seen on IRC, I can't casually
dismiss the possibility someone might ping 1500 discrete addresses
just to discover hosts on a few corporate subnets.
-- Rocco Caputo / troc@pobox.com / poe.perl.org / poe.sf.net
------------------------------
Date: Thu, 10 Oct 2002 21:55:00 +0200
From: "Alan J. Flavell" <flavell@mail.cern.ch>
Subject: Re: Multiple Pings/Second
Message-Id: <Pine.LNX.4.40.0210102147240.3295-100000@lxplus075.cern.ch>
On Oct 10, Rocco Caputo inscribed on the eternal scroll:
[along with much that I'd have no argument with...]
> Reliability aside, it doesn't seem very responsible to squirt packets
> onto a network faster than the network (the local interface included)
> can handle them. The result is a program that generates more traffic
> and takes longer than necessary.
>
> To be fair, ICMP packets tend to be smaller than UDP ones,
If you're using them to ping a host, they can be as large - up to some
max - or as small, down to some minimum - as you choose them to be -
whether ICMP or UDP.
If you're looking at the population of packets on a network, then
sure, many of the UDP packets will have a large payload of data, and
many of the ICMP packets will be simple management functions with
little payload. But that's irrelevant in a discussion of which sort
to choose for the purpose that's currently under discussion, surely?
The choice would be made by some quite other considerations, IMHO.
all the best
------------------------------
Date: Thu, 10 Oct 2002 16:51:02 -0400
From: Benjamin Goldberg <goldbb2@earthlink.net>
Subject: Re: Multiple Pings/Second
Message-Id: <3DA5E836.60B58B3D@earthlink.net>
Rocco Caputo wrote:
>
> On Thu, 10 Oct 2002 02:05:20 -0400, Benjamin Goldberg wrote:
> > Rocco Caputo wrote:
> >> See http://poe.perl.org/?POE_Cookbook/Pinging_Multiple_Hosts for a
> >> very simple example of pinging several hosts at once from a single
> >> process. Be aware, though, that it pings its list of hosts all at
> >> once, which won't work in your case.
> >
> > First -- why do you say that pinging the whole list at once won't
> > work?
>
> I've seen it fail for UDP, which is about as unreliable as ICMP.
>
> One guy wrote a POE program to poll his corporate network's Norton
> AntiVirus servers. NAV servers talk via UDP, which we've already
> established stands for Undeliverable Data Packets.
>
> I know, I know. It really means User Datagram Protocol.
>
> Anyway, he shotgunned several hundred of these UDP packets onto his
> company's network and dutifully waited the timeout period for his
> responses. The problem was, only a few percent of his packets were
> acknowledged.
>
> The first solution he came up with was to just resend the
> unacknowledged requests again and again if necessary. He counted the
> number number of times they were sent so he could eventually bail out
> on ones that would never answer.
<uh oh>This sounds a bit like what my suggested code does...</uh oh>
> His program took a very long time, though, because it would send a
> full volley of requests, wait the full timeout period, and in the end
> only complete a small number of transactions. It needed to do this
> many times to go through his list of hosts. It also generated several
> false negatives because most of the requests were repeated many times.
What do you mean, "false negatives"?
> The main cause of his problem turned out to be that he wasn't
> receiving responses and sending requests at the same time. He was
> using a simple loop to transmit all the requests together, then he was
> listening for responses. As a result, the operating system's receive
> queue for his socket would fill up, and the OS would silently drop the
> overflow.
I think I see.
So if response handling was interleaved with request sending, the
problem would [mostly] go away?
Ok, revised code:
my $socket = ...;
my @hosts = ...; # 4 byte ip addresses.
my %hosts = map {; pack_sockaddr_in(0, $_) => $_ } @hosts;
my @checksums;
vec( my $svec = "", fileno($socket), 1 ) = 1;
my $expire = 30 + time;
my ($lastsent, $sentinel, $wvec) = scalar(each %hosts);
while( (my $seq = $expire - time) >= 0 ) {
my ($msg) = @{$checksums[$seq] ||= do {
$sentinel = $lastsent;
$wvec = $svec;
# code, subcode, checksum, pid, seq, [data ommited]
my $msg = pack("C2S3", 8, 0, 0, $$ & 0xFFFF, $seq);
my $chk = unpack("%32S*", $msg);
# $chk += ord(substr( $msg, -1 )) if length($msg) % 2;
$chk = ($chk & 0xFFFF) + ($chk >> 16) for 1..2;
$chk = 0xFFFF & ~$chk;
$msg = pack("C2S3", 8, 0, $chk, $$ & 0xFFFF, $seq);
[ $msg, $chk ];
}};
my $rvec = $svec;
(my ($n), $t) = select($rvec, $wvec, '', 0.5);
die "select failed: $!" if !defined($n) or $n < 0;
next if $n == 0; # timeout.
if( $wvec =~ tr/^\0// ) {
my $ip; $ip ||= each %hosts for 1..2;
send( $s, $msg, 0, $lastsent = $ip ) or die;
$wvec = "" if $ip eq $sentinel;
}
$rvec =~ tr/^\0// or next;
my $responder = recv( $s, my ($resp), 8, 0 )
or die "Error in recv: $!"; # shouldn't happen.
my ($port, $ip) = unpack_sockaddr_in($responder);
die "Expected port to be zero" if $port;
my ($rtype, undef, $rchk, $rpid, $rseq) =
unpack("C2S3", $resp);
# make sure this is a valid packet:
$rtype == 0 and $rpid == ($$ & 0xFFFF)
and defined $checksums[$rseq]
and $rchk == $checksums[$rseq][1] or next;
# remove this host from the %hosts hash, and
# if wasn't previously there, ignore the packet.
delete $hosts{$responder} or next;
# print out the fact that we've discovered
# that the host is alive.
print inet_ntoa($ip) . " is alive\n";
last unless %hosts;
} # end while(1)
print inet_ntoa($_)." is down\n" for values %hosts;
[untested]
The use of $sentinel and $lastsent is to ensure that the whole list is
sent no more than once per second, and that the next time we start
sending, we begin where we left off.
> > Second -- who says you *have* to feed it the whole list at once?
> > You could feed it some reasonable sized block, wait for them to
> > respond or time out, and then send more.
>
> Nobody says the requests need to be fed all at once, but the program I
> cited does this. I felt the need to point out why this won't work.
>
> Anyway, I covered request throttling next.
>
> >> The problem there is that ICMP isn't a reliable protocol. 1500
> >> hosts over 60 seconds is an average of 25 ICMP echo/reply pairs (50
> >> packets) per second. Some packets are bound to get lost in the
> >> shuffle, resulting in erratic host statuses.
> >
> > The obvious thing to do is send your pings as fast as you can, read
> > as many responses as you can within a timeout, and then resend to
> > those who didn't respond.
>
> Reliability aside, it doesn't seem very responsible to squirt packets
> onto a network faster than the network (the local interface included)
> can handle them. The result is a program that generates more traffic
> and takes longer than necessary.
What, you mean the most obvious thing to do isn't the most responsible
thing to do? :)
> To be fair, ICMP packets tend to be smaller than UDP ones, so the
> amount of redundant traffic in each volley will be lower.
>
> >> You may be able to send far fewer than 1500 echo requests if your
> >> operating system and routers allow you to ping subnet broadcast
> >> addresses.
> >
> > Since he hasn't said that he's pinging broadcast addresses, I would
> > *assume* that he's pinging actual hosts. Thus, this shouldn't be
> > relevant (I think).
>
> I didn't know for sure what he was doing, so I presented this as just
> one option among others. After all I've seen on IRC, I can't casually
> dismiss the possibility someone might ping 1500 discrete addresses
> just to discover hosts on a few corporate subnets.
Uhh, is it even possible to *ping* a broadcast address? And if you
*could,* the response packet's "sender" surely would be from the machine
which sent the response, not the broadcast address.
The logic for testing if ping packet is really one we care about would
need significant improvement if it's possible to ping one address and
get the corresponding response from a different address.
--
my $n = 2; print +(split //, 'e,4c3H r ktulrnsJ2tPaeh'
."\n1oa! er")[map $n = ($n * 24 + 30) % 31, (42) x 26]
------------------------------
Date: Thu, 10 Oct 2002 21:20:34 +0200
From: "Dan Borlovan" <danb@mail.dnttm.ro>
Subject: Re: NDBM_File
Message-Id: <ao4gdc$bil$1@nebula.dnttm.ro>
> can I download this module? Or how else can I tie a ash to a file in
> NDBM-format?
man NDBM_File:
NAME
NDBM_File - Tied access to ndbm files
SYNOPSIS
use NDBM_File;
use Fcntl; # for O_ constants
tie(%h, 'NDBM_File', 'Op.dbmx', O_RDWR|O_CREAT, 0640);
untie %h;
DESCRIPTION
See the tie entry in the perlfunc manpage
Dan
------------------------------
Date: 10 Oct 2002 14:43:57 -0700
From: gmnotyet@yahoo.com (J Hall)
Subject: Need to convert from seconds since Unix epoch to GMT
Message-Id: <ab25f06e.0210101343.29ca92ca@posting.google.com>
Dear Perl users,
Can someone easily tell me how to convert from seconds since
the Unix epoch (midnight, Jan 1, 1970) to GMT? Seems straightforward,
but all the docs I keep seeing address other problems. Thanks.
------------------------------
Date: 10 Oct 2002 12:30:56 -0700
From: dejen321@yahoo.com (jend)
Subject: Net::FTP: Unexpected EOF on command channel
Message-Id: <cc028093.0210101130.5fc8d9c1@posting.google.com>
hi,
i have this basic Net::FTP script that goes out to pull a file
everynight.
It has been working reliability for the past year until recently we
switch
to PASSIVE FTP. So what I did is set env variable FTP_PASSIVE to 1
and Net::FTP picks up the passive mode from there.
Now the script still works probably 4 or 5 days of the week, but
fails 2 or 3 times per week saying:
Net::FTP: Exporter(5.562)
Net::FTP: Net::Cmd(2.21)
Net::FTP: IO::Socket::INET(1.25)
Net::FTP: IO::Socket(1.26)
Net::FTP: IO::Handle(1.21)
Net::FTP=GLOB(0xf30e8)<<< 220-FTP SERVER
Net::FTP=GLOB(0xf30e8)<<< 331 User name okay, need password.
Net::FTP=GLOB(0xf30e8)>>> PASS ....
Net::FTP=GLOB(0xf30e8)<<< 230 User logged in, proceed.
Net::FTP=GLOB(0xf30e8)>>> CWD abc
Net::FTP=GLOB(0xf30e8)<<< 250 Directory changed to /abc
Net::FTP=GLOB(0xf30e8)>>> PASV
Net::FTP=GLOB(0xf30e8)<<< 227 Entering Passive Mode (blah blah)
Net::FTP=GLOB(0xf30e8)>>> NLST -lR
Net::FTP=GLOB(0xf30e8)<<< 150 Opening ASCII mode data connection for
/bin/ls.
Net::FTP=GLOB(0xf30e8)<<< 226 Transfer complete.
Net::FTP=GLOB(0xf30e8)>>> TYPE I
Net::FTP=GLOB(0xf30e8)<<< 200 Type set to I.
Net::FTP=GLOB(0xf30e8)>>> PASV
Net::FTP: Unexpected EOF on command channel at ./data_pull.pl line 52
Could not get 'daily_file.gz', skipped: Bad file number at
./data_pull.pl line 52.
btw, line 52 is:
$ftp->get($name,$filename)
and $ftp is:
$ftp = Net::FTP->new($host, Timeout => $timeout, Debug => 1) or die
"Can't connect: $@\n";
what i don't understand is why it runs most of the time but fails
sometimes (which is really bad for my production process) and the
weirdest part is
it only started recently when we use PASSIVE ftp so now it fails at
the command PASV whenever it fails..
i read many many posts about this topic but didn't see anything
similar to my situation and/or definite solutions...
if anyone has any ideas or suggestions, please reply here...
thanks. M.
------------------------------
Date: Thu, 10 Oct 2002 20:00:09 +0000 (UTC)
From: dkcombs@panix.com (David Combs)
Subject: Re: Perl Journal?
Message-Id: <ao4m89$k12$1@reader1.panix.com>
In article <250920020126107554%comdog@panix.com>,
brian d foy <comdog@panix.com> wrote:
>In article <5j7k9.11382$Lg2.3058384@news2.news.adelphia.net>, Bob X <bobx@linuxmail.org> wrote:
>
>> "brian d foy" <comdog@panix.com> wrote in message
>> news:240920021826548170%comdog@panix.com...
>
>> > Both TPJ and TPR need to build a minimum paid subscriber base to make it
>
>> So "out of ignorance he asks"...why do both? Why not support one with a
>> larger subscriber base?
>
>when I started TPR i assumed TPJ was dead. neither EarthWeb nor CMP showed
>much motivation to actually publish it.
>
>after talking to the editors at TPJ recently, i think that if TPR had not existed
>they would not have thought to make TPJ a PDF-only release.
>
>sometimes the world turns out differently than you thought it would, but i still
>don't see any reason that both can't survive. i certainly do not want to run TPJ
>out of existence. :)
>
>--
>brian d foy <comdog@panix.com> - Perl services for hire
>The Perl Review - a new magazine devoted to Perl
><http://www.theperlreview.com>
------------------------------
Date: 6 Apr 2001 21:33:47 GMT (Last modified)
From: Perl-Users-Request@ruby.oce.orst.edu (Perl-Users-Digest Admin)
Subject: Digest Administrivia (Last modified: 6 Apr 01)
Message-Id: <null>
Administrivia:
The Perl-Users Digest is a retransmission of the USENET newsgroup
comp.lang.perl.misc. For subscription or unsubscription requests, send
the single line:
subscribe perl-users
or:
unsubscribe perl-users
to almanac@ruby.oce.orst.edu.
To submit articles to comp.lang.perl.announce, send your article to
clpa@perl.com.
To request back copies (available for a week or so), send your request
to almanac@ruby.oce.orst.edu with the command "send perl-users x.y",
where x is the volume number and y is the issue number.
For other requests pertaining to the digest, send mail to
perl-users-request@ruby.oce.orst.edu. Do not waste your time or mine
sending perl questions to the -request address, I don't have time to
answer them even if I did know the answer.
------------------------------
End of Perl-Users Digest V10 Issue 3950
***************************************
