[31396] in Perl-Users-Digest
Perl-Users Digest, Issue: 2648 Volume: 11
daemon@ATHENA.MIT.EDU (Perl-Users Digest)
Fri Oct 23 14:09:41 2009
Date: Fri, 23 Oct 2009 11:09:08 -0700 (PDT)
From: Perl-Users Digest <Perl-Users-Request@ruby.OCE.ORST.EDU>
To: Perl-Users@ruby.OCE.ORST.EDU (Perl-Users Digest)
Perl-Users Digest Fri, 23 Oct 2009 Volume: 11 Number: 2648
Today's topics:
File locking using threads <timothy.hill@gmail.com>
Re: File locking using threads (Jens Thoms Toerring)
Re: File locking using threads <ben@morrow.me.uk>
How to prevent hanging when writing lots of text to a p <jl_post@hotmail.com>
Re: How to prevent hanging when writing lots of text to <c7eqjyg02@sneakemail.com>
Re: How to prevent hanging when writing lots of text to <jurgenex@hotmail.com>
Re: How to prevent hanging when writing lots of text to <ben@morrow.me.uk>
Re: Indirection in a hash <mvdwege@mail.com>
Re: Indirection in a hash <burner+usenet@imf.au.dk>
Re: Indirection in a hash <jurgenex@hotmail.com>
Re: Indirection in a hash <uri@StemSystems.com>
Posting Guidelines for comp.lang.perl.misc ($Revision: tadmc@seesig.invalid
Re: Want to write a script to note specific IP addresse <paduille.4061.mumia.w+nospam@earthlink.net>
Re: Want to write a script to note specific IP addresse <hongyi.zhao@gmail.com>
Re: Want to write a script to note specific IP addresse <someone@example.com>
Digest Administrivia (Last modified: 6 Apr 01) (Perl-Users-Digest Admin)
----------------------------------------------------------------------
Date: Fri, 23 Oct 2009 08:59:47 -0700 (PDT)
From: Prince Al <timothy.hill@gmail.com>
Subject: File locking using threads
Message-Id: <48e6e1e3-bc67-4171-83a5-d39b4b5d38ac@r36g2000vbn.googlegroups.com>
Hi,
I am attempting to write a some code that spawns various threads. Each
thread will need to access a common file and to ensure that this file
is only written to by one thread at a time, it needs to be locked.
However, I am having trouble even creating a test script to try out
the logic, which is below. What happens is that both threads are able
to acquire the lock, even though the first thread should have it and
release it before the second thread is able to lock the file. Any
assistance will be very gratefully received!
Cheers
Tim
#!usr/bin/perl
use strict;
use threads;
my $thr_1 = threads->new(\&flock_test,5, 20, "Thread 1");
my $thr_2 = threads->new(\&flock_test,10, 5, "Thread 2");
grep {$_->join;} ($thr_1,$thr_2);
# Just In Case!
while ( my(@list)=threads->list()) {
print "$#list\n";
grep { $_->join } @list;
};
exit;
sub flock_test {
open(my $fh, ">", "/tmp/flock_test.dat");
my $sleep_1 = $_[0];
my $sleep_2 = $_[1];
my $thread_name = $_[2];
sleep $sleep_1;
flock($fh,2) || die "Could not acquire the lock\n";
print "$thread_name: acquired lock!\n";
sleep $sleep_2;
close $fh;
print "$thread_name: released lock!\n";
}
------------------------------
Date: 23 Oct 2009 17:06:32 GMT
From: jt@toerring.de (Jens Thoms Toerring)
Subject: Re: File locking using threads
Message-Id: <7ke64oF38i9k6U1@mid.uni-berlin.de>
Prince Al <timothy.hill@gmail.com> wrote:
> I am attempting to write a some code that spawns various threads. Each
> thread will need to access a common file and to ensure that this file
> is only written to by one thread at a time, it needs to be locked.
> However, I am having trouble even creating a test script to try out
> the logic, which is below. What happens is that both threads are able
> to acquire the lock, even though the first thread should have it and
> release it before the second thread is able to lock the file. Any
> assistance will be very gratefully received!
> #!usr/bin/perl
> use strict;
> use threads;
> my $thr_1 = threads->new(\&flock_test,5, 20, "Thread 1");
> my $thr_2 = threads->new(\&flock_test,10, 5, "Thread 2");
> grep {$_->join;} ($thr_1,$thr_2);
> # Just In Case!
> while ( my(@list)=threads->list()) {
> print "$#list\n";
> grep { $_->join } @list;
> };
> exit;
> sub flock_test {
> open(my $fh, ">", "/tmp/flock_test.dat");
> my $sleep_1 = $_[0];
> my $sleep_2 = $_[1];
> my $thread_name = $_[2];
> sleep $sleep_1;
> flock($fh,2) || die "Could not acquire the lock\n";
Works ok on my machine. My guess is that on yours LOCK_EX (that's
what you need here) isn't 2. Better use
use Fcntl ':flock';
to import the constants used for flock() and then write
flock($fh,LOCK_EX) || die "Could not acquire the lock\n";
Regards, Jens
--
\ Jens Thoms Toerring ___ jt@toerring.de
\__________________________ http://toerring.de
------------------------------
Date: Fri, 23 Oct 2009 18:14:32 +0100
From: Ben Morrow <ben@morrow.me.uk>
Subject: Re: File locking using threads
Message-Id: <os49r6-rmj.ln1@osiris.mauzo.dyndns.org>
Quoth Prince Al <timothy.hill@gmail.com>:
>
> I am attempting to write a some code that spawns various threads. Each
> thread will need to access a common file and to ensure that this file
> is only written to by one thread at a time, it needs to be locked.
> However, I am having trouble even creating a test script to try out
> the logic, which is below. What happens is that both threads are able
> to acquire the lock, even though the first thread should have it and
> release it before the second thread is able to lock the file. Any
> assistance will be very gratefully received!
Which OS are you using, and does it emulate flock(2) using fcntl(2)?
Normal flock(2) semantics don't allow a single process to acquire the
same lock twice, but the POSIX semantics used by fcntl locking do.
> #!usr/bin/perl
>
> use strict;
> use threads;
>
> my $thr_1 = threads->new(\&flock_test,5, 20, "Thread 1");
> my $thr_2 = threads->new(\&flock_test,10, 5, "Thread 2");
>
> grep {$_->join;} ($thr_1,$thr_2);
Goodness me. map-in-void-context is a known (if questionable) idiom, but
grep-in-void-context is just daft. You want
$_->join for $thr_1, $thr_2;
> flock($fh,2) || die "Could not acquire the lock\n";
Don't rely on magic numbers like that: it's unclear and unportable.
Import the appropriate constants from the Fcntl module:
use Fcntl qw/:flock/;
flock($fh, LOCK_EX) || die ...;
Ben
------------------------------
Date: Fri, 23 Oct 2009 08:47:30 -0700 (PDT)
From: "jl_post@hotmail.com" <jl_post@hotmail.com>
Subject: How to prevent hanging when writing lots of text to a pipe?
Message-Id: <c2e9dc54-4f4d-48f7-aeb3-912a89f03a03@s21g2000prm.googlegroups.com>
Hi,
I'm trying to write text to a piped writeHandle (created with the
pipe() function) so that I can later read the text by extracting it
from a readHandle. However, I discovered that if I write a lot of
text to the pipe, my script just hangs. Here is an example program:
#!/usr/bin/perl
use strict;
use warnings;
print "Enter a number: ";
my $number = <STDIN>;
chomp($number);
my @lines = do
{
pipe(my $readHandle, my $writeHandle);
# Autoflush $writeHandle:
my $oldHandle = select($writeHandle);
$| = 1;
select($oldHandle);
print $writeHandle "$_\n" foreach 1 .. $number;
close($writeHandle);
# Extract the output, line-by-line:
<$readHandle>
};
print "Extracted output lines:\n @lines";
__END__
When I run this program, I notice that it runs perfectly for small
values of $number (like 10). But on high values (like ten thousand),
the program hangs.
From testing, I discovered that the limit on the Windows platform
I'm using is 155, and the limit on the Linux platform I'm using is
1040. Any higher number causes the program to hang.
As for why this is happening, my best guess is that a program can
only stuff so much output into a piped writeHandle before it gets
full. Therefore, deadlock occurs, as the reading won't happen until
the writing is finished.
However, I'm not fully convinced this is the case, because I
replaced the lines:
print $writeHandle "$_\n" foreach 1 .. $number;
close($writeHandle);
with:
if (fork() == 0)
{
# Only the child process gets here:
print $writeHandle "$_\n" foreach 1 .. $number;
close($writeHandle);
exit(0);
}
and now the Perl script hangs on both Windows and Linux platforms,
even with low values of $number (such as 5). My intent was to make
the child process solely responsible for stuffing the output into the
pipe, while the parent process read from the $readHandle as data
became available. That way we would avoid the pipe getting stuffed to
capacity.
But as I've said, that fork()ing code change doesn't even work for
any values, so I must be doing something wrong somewhere.
So my question is: How do I prevent my script from hanging when I
have a lot of text to send through the pipe?
Thanks in advance for any help.
-- Jean-Luc
------------------------------
Date: Fri, 23 Oct 2009 09:27:01 -0700 (PDT)
From: ilovelinux <c7eqjyg02@sneakemail.com>
Subject: Re: How to prevent hanging when writing lots of text to a pipe?
Message-Id: <3c852ba2-9254-41fc-a19d-6c51bc473d94@d23g2000vbm.googlegroups.com>
On 23 okt, 17:47, "jl_p...@hotmail.com" <jl_p...@hotmail.com> wrote:
> =A0 =A0I'm trying to write text to a piped writeHandle (created with the
> pipe() function) so that I can later read the text by extracting it
> from a readHandle. =A0However, I discovered that if I write a lot of
> text to the pipe, my script just hangs. =A0Here is an example program:
[snip]
> =A0 =A0As for why this is happening, my best guess is that a program can
> only stuff so much output into a piped writeHandle before it gets
> full. =A0Therefore, deadlock occurs, as the reading won't happen until
> the writing is finished.
That's right. Pipes have a limited capacity. See http://linux.die.net/man/7=
/pipe.
Posix prescribes a minimum capacity of 512, which is exactly
implemented on your windows machine:
$ perl -we 'print "$_\n" for 1..155'| wc -c
512
$
Linux has 4096 on your machine:
$ perl -we 'print "$_\n" for 1..1040'| wc -c
4093
$
but in modern kernels it is 2^16.
As for your fork()ing test program: you should close both the write
descriptor of the pipe in the reading process and the read descriptor
in the writing process.
------------------------------
Date: Fri, 23 Oct 2009 09:31:10 -0700
From: Jürgen Exner <jurgenex@hotmail.com>
Subject: Re: How to prevent hanging when writing lots of text to a pipe?
Message-Id: <p6m3e59bsuh5819lror19mtv1ka86vaoh8@4ax.com>
"jl_post@hotmail.com" <jl_post@hotmail.com> wrote:
> I'm trying to write text to a piped writeHandle (created with the
>pipe() function) so that I can later read the text by extracting it
>from a readHandle. However, I discovered that if I write a lot of
>text to the pipe, my script just hangs. Here is an example program:
>
[...]
> When I run this program, I notice that it runs perfectly for small
>values of $number (like 10). But on high values (like ten thousand),
>the program hangs.
>[...]
>Therefore, deadlock occurs, as the reading won't happen until
>the writing is finished.
And that is your problem. Pipes are an IPC method, they are not meant
and not designed to store any data. You are abusing the buffer for
long-term storage. Of course that is not going to work.
You need a different design, e.g. using a file as storage medium. There
are other options, too, like the Storable module which may or may not be
of help.
jue
------------------------------
Date: Fri, 23 Oct 2009 18:01:51 +0100
From: Ben Morrow <ben@morrow.me.uk>
Subject: Re: How to prevent hanging when writing lots of text to a pipe?
Message-Id: <v449r6-rmj.ln1@osiris.mauzo.dyndns.org>
Quoth "jl_post@hotmail.com" <jl_post@hotmail.com>:
> Hi,
>
> I'm trying to write text to a piped writeHandle (created with the
> pipe() function) so that I can later read the text by extracting it
> from a readHandle. However, I discovered that if I write a lot of
> text to the pipe, my script just hangs. Here is an example program:
>
> #!/usr/bin/perl
>
> use strict;
> use warnings;
>
> print "Enter a number: ";
> my $number = <STDIN>;
> chomp($number);
>
> my @lines = do
> {
> pipe(my $readHandle, my $writeHandle);
>
> # Autoflush $writeHandle:
> my $oldHandle = select($writeHandle);
> $| = 1;
> select($oldHandle);
>
> print $writeHandle "$_\n" foreach 1 .. $number;
> close($writeHandle);
>
> # Extract the output, line-by-line:
> <$readHandle>
> };
>
> print "Extracted output lines:\n @lines";
>
> __END__
>
> When I run this program, I notice that it runs perfectly for small
> values of $number (like 10). But on high values (like ten thousand),
> the program hangs.
>
> From testing, I discovered that the limit on the Windows platform
> I'm using is 155, and the limit on the Linux platform I'm using is
> 1040. Any higher number causes the program to hang.
>
> As for why this is happening, my best guess is that a program can
> only stuff so much output into a piped writeHandle before it gets
> full. Therefore, deadlock occurs, as the reading won't happen until
> the writing is finished.
This is correct.
> However, I'm not fully convinced this is the case, because I
> replaced the lines:
>
> print $writeHandle "$_\n" foreach 1 .. $number;
> close($writeHandle);
>
> with:
>
> if (fork() == 0)
> {
> # Only the child process gets here:
> print $writeHandle "$_\n" foreach 1 .. $number;
> close($writeHandle);
> exit(0);
> }
>
> and now the Perl script hangs on both Windows and Linux platforms,
> even with low values of $number (such as 5).
What is happening here is that the parent process never closes the
writing end of the pipe, so it never gets EOF on the read end. If you
replace the <>-in-list-context with a loop like
while (<$readHandle>) {
warn "read line: $_";
push @lines, $_;
}
you will see that it succeeds in reading all the lines, but hangs after
the last waiting for EOF.
The solution is to add
close($writeHandle);
as the first statement executed in the parent after the child is forked;
you should also close $readHandle in the parent, as otherwise the child
won't get notified if the parent exits early.
Note: I have no idea if this will work on Win32. Win32 perl's fork
emulation can sometimes be a little peculiar.
Ben
------------------------------
Date: Fri, 23 Oct 2009 09:29:21 +0200
From: Mart van de Wege <mvdwege@mail.com>
Subject: Re: Indirection in a hash
Message-Id: <86fx9atvou.fsf@gareth.avalon.lan>
"Uri Guttman" <uri@StemSystems.com> writes:
>>>>>> "MvdW" == Mart van de Wege <mvdwege@mail.com> writes:
>
> MvdW> "Uri Guttman" <uri@StemSystems.com> writes:
> >>>>>>> "MvdW" == Mart van de Wege <mvdwege@mail.com> writes:
> >>
> MvdW> jt@toerring.de (Jens Thoms Toerring) writes:
> >> >> "multidimensional arrays" (I put that in parantheses since there
> >> >> aren't real multi- dimensional arrays in Perl but instead you use
> >> >> arrays of array references to emulate them).
> >>
> MvdW> (ITYM quotes, not parentheses).
> >>
> MvdW> What is the difference? Surely this is the most obvious way to implement
> MvdW> multidimensional arrays on a lower level?
> >>
> MvdW> And if Perl's "array of references" is functionally equivalent to a
> MvdW> multidimensional array (and it is), then isn't it a multidimensional
> MvdW> array?
> >>
> >> by the ways classic compiled langs like c does them, then no, perl
> >> doesn't support multidimensional arrays. in those langs you usually
> >> predeclare all the array's dimensions and you can directly access
> >> elements with a fast calculation of the correct offset given the
> >> indices. in perl's version you have to loop over each dimension then
> >> index to get the next level down.
>
> MvdW> Surely this is only true if the dimensions are of unknown
> MvdW> size. If your data model is such that you expect to have fixed
> MvdW> size arrays, you can still directly index into it, even with
> MvdW> Perl's model.
>
> no you can't. with perl all array accesses require looping over each
> dimension. there is no way to directly access a lower level. the code
> looks like it does direct access but internally it does a loop over the
> indices.
I think we're talking past each other. That the compiler can't directly
access a value does not mean that the language syntax doesn't allow
it. It merely makes this an inefficient operation.
This also means that if your data model heavily depends on two- or
more-dimensional arrays, Perl may not be the right tool for the job,
or your data model needs rethinking.
Mart
--
"We will need a longer wall when the revolution comes."
--- AJS, quoting an uncertain source.
------------------------------
Date: Fri, 23 Oct 2009 10:13:45 +0200
From: Rasmus Villemoes <burner+usenet@imf.au.dk>
Subject: Re: Indirection in a hash
Message-Id: <u0locnywmrq.fsf@orc10.imf.au.dk>
Mart van de Wege <mvdwege@mail.com> writes:
> 'Most obvious' != 'Most efficient'.
s/!=/ne/
;-)
--
Rasmus Villemoes
<http://rasmusvillemoes.dk/>
------------------------------
Date: Fri, 23 Oct 2009 02:45:44 -0700
From: Jürgen Exner <jurgenex@hotmail.com>
Subject: Re: Indirection in a hash
Message-Id: <vpt2e5t191a8ggm0spcj71tltiarmn8on2@4ax.com>
Mart van de Wege <mvdwege@mail.com> wrote:
>Jürgen Exner <jurgenex@hotmail.com> writes:
>>>What is the difference? Surely this is the most obvious way to implement
>>>multidimensional arrays on a lower level?
>>
>> Actually no because individual elements cannot be accessed directly with
>> O(1) by pre-calculating the memory location.
>
>Erm.
>
>'Most obvious' != 'Most efficient'.
>
>Certainly pre-calculating is faster, and can't be done in Perl because
>the size of the dimensions is unknown, but that's a compiler issue.
True. But when creating a compiler you thrive for efficieny of the
generated code. Therefore the non-efficient way is usually not the
obvious choice for a compiler architect.
>I am however not a Perl internals hacker. I am a sysadmin who does a bit
>of Perl programming to automate tasks and to support business. For me,
>if my data model consists of matrix-like structures, an array of
>references to arrays looks just like a multidimensional array.
Then the design goal for Perl arrays works very well for you.
However there are situations where the wrong mental model can bite you.
One example:
@fourthRow = @multiArray[4];
In a true multi-dimensional model @multiArray[4] would yield the whole
fourth (actually fifth but let's not go there) row of the
multi-dimensional array.
In Perl however you get a result array with a single element: the
reference to the sub-array.
Same vice-versa:
@multiArray[4] = @fourthRow;
does not assign the fourth (fifth) row of multiArray but puts the length
of @fourthRow into the fourth element of multiArray.
To me that is a very relevant difference and actually questions like
this pop up here every now and then when newbies to Perl who have never
dealt with Perl references before are lost as to what is happening.
jue
------------------------------
Date: Fri, 23 Oct 2009 13:18:54 -0400
From: "Uri Guttman" <uri@StemSystems.com>
Subject: Re: Indirection in a hash
Message-Id: <8763a681vl.fsf@quad.sysarch.com>
>>>>> "MvdW" == Mart van de Wege <mvdwege@mail.com> writes:
MvdW> "Uri Guttman" <uri@StemSystems.com> writes:
>>>>>>> "MvdW" == Mart van de Wege <mvdwege@mail.com> writes:
>>
MvdW> "Uri Guttman" <uri@StemSystems.com> writes:
>> >>>>>>> "MvdW" == Mart van de Wege <mvdwege@mail.com> writes:
>> >>
MvdW> jt@toerring.de (Jens Thoms Toerring) writes:
>> >> >> "multidimensional arrays" (I put that in parantheses since there
>> >> >> aren't real multi- dimensional arrays in Perl but instead you use
>> >> >> arrays of array references to emulate them).
>> >>
MvdW> (ITYM quotes, not parentheses).
>> >>
MvdW> What is the difference? Surely this is the most obvious way to implement
MvdW> multidimensional arrays on a lower level?
>> >>
MvdW> And if Perl's "array of references" is functionally equivalent to a
MvdW> multidimensional array (and it is), then isn't it a multidimensional
MvdW> array?
>> >>
>> >> by the ways classic compiled langs like c does them, then no, perl
>> >> doesn't support multidimensional arrays. in those langs you usually
>> >> predeclare all the array's dimensions and you can directly access
>> >> elements with a fast calculation of the correct offset given the
>> >> indices. in perl's version you have to loop over each dimension then
>> >> index to get the next level down.
>>
MvdW> Surely this is only true if the dimensions are of unknown
MvdW> size. If your data model is such that you expect to have fixed
MvdW> size arrays, you can still directly index into it, even with
MvdW> Perl's model.
>>
>> no you can't. with perl all array accesses require looping over each
>> dimension. there is no way to directly access a lower level. the code
>> looks like it does direct access but internally it does a loop over the
>> indices.
MvdW> I think we're talking past each other. That the compiler can't directly
MvdW> access a value does not mean that the language syntax doesn't allow
MvdW> it. It merely makes this an inefficient operation.
no, i get both sides very well. i worked in a compiler company way back
and know perl well too. i was just emphasizing the difference between
the abstraction of multidim arrays which perl supports and the
implementation which is slower in perl. it does matter sometimes and you
can't just say perl supports multidim arrays without that context.
MvdW> This also means that if your data model heavily depends on two- or
MvdW> more-dimensional arrays, Perl may not be the right tool for the job,
MvdW> or your data model needs rethinking.
not really. speed is only an issue if it becomes an issue. i have dealt
with largish data trees (my term for mixed hash/array structures)
without any issues. the flexibility overrides any speed issues i may
have seen. but i still do my own optimization by factoring out common
deep accesses. i see this way too often:
$x = $foo{bar}[1]{baz}[0] ;
$y = $foo{bar}[1]{baz}[1] ;
and so on. the higher level accesses would be factored out by a good
optimizing compiler in fortran but not in perl. and it is also harder to
read and edit. and when you remove the common stuff, you end up with
access to a simpler shallower tree which is easy to code and
handle. just one of the things i try to teach and the reasons are both
speed and code clarity. just because you can write code like the above
and deal directly with multidim structures doesn't mean you should do it
that way. the data can still be in that form but you code it to make
accessing it better and faster.
uri
--
Uri Guttman ------ uri@stemsystems.com -------- http://www.sysarch.com --
----- Perl Code Review , Architecture, Development, Training, Support ------
--------- Gourmet Hot Cocoa Mix ---- http://bestfriendscocoa.com ---------
------------------------------
Date: Fri, 23 Oct 2009 02:25:38 -0500
From: tadmc@seesig.invalid
Subject: Posting Guidelines for comp.lang.perl.misc ($Revision: 1.9 $)
Message-Id: <6pednWipeM7vx3zXnZ2dnUVZ_h2dnZ2d@giganews.com>
Outline
Before posting to comp.lang.perl.misc
Must
- Check the Perl Frequently Asked Questions (FAQ)
- Check the other standard Perl docs (*.pod)
Really Really Should
- Lurk for a while before posting
- Search a Usenet archive
If You Like
- Check Other Resources
Posting to comp.lang.perl.misc
Is there a better place to ask your question?
- Question should be about Perl, not about the application area
How to participate (post) in the clpmisc community
- Carefully choose the contents of your Subject header
- Use an effective followup style
- Speak Perl rather than English, when possible
- Ask perl to help you
- Do not re-type Perl code
- Provide enough information
- Do not provide too much information
- Do not post binaries, HTML, or MIME
Social faux pas to avoid
- Asking a Frequently Asked Question
- Asking a question easily answered by a cursory doc search
- Asking for emailed answers
- Beware of saying "doesn't work"
- Sending a "stealth" Cc copy
Be extra cautious when you get upset
- Count to ten before composing a followup when you are upset
- Count to ten after composing and before posting when you are upset
-----------------------------------------------------------------
Posting Guidelines for comp.lang.perl.misc ($Revision: 1.9 $)
This newsgroup, commonly called clpmisc, is a technical newsgroup
intended to be used for discussion of Perl related issues (except job
postings), whether it be comments or questions.
As you would expect, clpmisc discussions are usually very technical in
nature and there are conventions for conduct in technical newsgroups
going somewhat beyond those in non-technical newsgroups.
The article at:
http://www.catb.org/~esr/faqs/smart-questions.html
describes how to get answers from technical people in general.
This article describes things that you should, and should not, do to
increase your chances of getting an answer to your Perl question. It is
available in POD, HTML and plain text formats at:
http://www.rehabitation.com/clpmisc.shtml
For more information about netiquette in general, see the "Netiquette
Guidelines" at:
http://andrew2.andrew.cmu.edu/rfc/rfc1855.html
A note to newsgroup "regulars":
Do not use these guidelines as a "license to flame" or other
meanness. It is possible that a poster is unaware of things
discussed here. Give them the benefit of the doubt, and just
help them learn how to post, rather than assume that they do
know and are being the "bad kind" of Lazy.
A note about technical terms used here:
In this document, we use words like "must" and "should" as
they're used in technical conversation (such as you will
encounter in this newsgroup). When we say that you *must* do
something, we mean that if you don't do that something, then
it's unlikely that you will benefit much from this group.
We're not bossing you around; we're making the point without
lots of words.
Do *NOT* send email to the maintainer of these guidelines. It will be
discarded unread. The guidelines belong to the newsgroup so all
discussion should appear in the newsgroup. I am just the secretary that
writes down the consensus of the group.
Before posting to comp.lang.perl.misc
Must
This section describes things that you *must* do before posting to
clpmisc, in order to maximize your chances of getting meaningful replies
to your inquiry and to avoid getting flamed for being lazy and trying to
have others do your work.
The perl distribution includes documentation that is copied to your hard
drive when you install perl. Also installed is a program for looking
things up in that (and other) documentation named 'perldoc'.
You should either find out where the docs got installed on your system,
or use perldoc to find them for you. Type "perldoc perldoc" to learn how
to use perldoc itself. Type "perldoc perl" to start reading Perl's
standard documentation.
Check the Perl Frequently Asked Questions (FAQ)
Checking the FAQ before posting is required in Big 8 newsgroups in
general, there is nothing clpmisc-specific about this requirement.
You are expected to do this in nearly all newsgroups.
You can use the "-q" switch with perldoc to do a word search of the
questions in the Perl FAQs.
Check the other standard Perl docs (*.pod)
The perl distribution comes with much more documentation than is
available for most other newsgroups, so in clpmisc you should also
see if you can find an answer in the other (non-FAQ) standard docs
before posting.
It is *not* required, or even expected, that you actually *read* all of
Perl's standard docs, only that you spend a few minutes searching them
before posting.
Try doing a word-search in the standard docs for some words/phrases
taken from your problem statement or from your very carefully worded
"Subject:" header.
Really Really Should
This section describes things that you *really should* do before posting
to clpmisc.
Lurk for a while before posting
This is very important and expected in all newsgroups. Lurking means
to monitor a newsgroup for a period to become familiar with local
customs. Each newsgroup has specific customs and rituals. Knowing
these before you participate will help avoid embarrassing social
situations. Consider yourself to be a foreigner at first!
Search a Usenet archive
There are tens of thousands of Perl programmers. It is very likely
that your question has already been asked (and answered). See if you
can find where it has already been answered.
One such searchable archive is:
http://groups.google.com/advanced_search
If You Like
This section describes things that you *can* do before posting to
clpmisc.
Check Other Resources
You may want to check in books or on web sites to see if you can
find the answer to your question.
But you need to consider the source of such information: there are a
lot of very poor Perl books and web sites, and several good ones
too, of course.
Posting to comp.lang.perl.misc
There can be 200 messages in clpmisc in a single day. Nobody is going to
read every article. They must decide somehow which articles they are
going to read, and which they will skip.
Your post is in competition with 199 other posts. You need to "win"
before a person who can help you will even read your question.
These sections describe how you can help keep your article from being
one of the "skipped" ones.
Is there a better place to ask your question?
Question should be about Perl, not about the application area
It can be difficult to separate out where your problem really is,
but you should make a conscious effort to post to the most
applicable newsgroup. That is, after all, where you are the most
likely to find the people who know how to answer your question.
Being able to "partition" a problem is an essential skill for
effectively troubleshooting programming problems. If you don't get
that right, you end up looking for answers in the wrong places.
It should be understood that you may not know that the root of your
problem is not Perl-related (the two most frequent ones are CGI and
Operating System related), so off-topic postings will happen from
time to time. Be gracious when someone helps you find a better place
to ask your question by pointing you to a more applicable newsgroup.
How to participate (post) in the clpmisc community
Carefully choose the contents of your Subject header
You have 40 precious characters of Subject to win out and be one of
the posts that gets read. Don't waste them. Take care while
composing them, they are the key that opens the door to getting an
answer.
Spend them indicating what aspect of Perl others will find if they
should decide to read your article.
Do not spend them indicating "experience level" (guru, newbie...).
Do not spend them pleading (please read, urgent, help!...).
Do not spend them on non-Subjects (Perl question, one-word
Subject...)
For more information on choosing a Subject see "Choosing Good
Subject Lines":
http://www.cpan.org/authors/id/D/DM/DMR/subjects.post
Part of the beauty of newsgroup dynamics, is that you can contribute
to the community with your very first post! If your choice of
Subject leads a fellow Perler to find the thread you are starting,
then even asking a question helps us all.
Use an effective followup style
When composing a followup, quote only enough text to establish the
context for the comments that you will add. Always indicate who
wrote the quoted material. Never quote an entire article. Never
quote a .signature (unless that is what you are commenting on).
Intersperse your comments *following* each section of quoted text to
which they relate. Unappreciated followup styles are referred to as
"top-posting", "Jeopardy" (because the answer comes before the
question), or "TOFU" (Text Over, Fullquote Under).
Reversing the chronology of the dialog makes it much harder to
understand (some folks won't even read it if written in that style).
For more information on quoting style, see:
http://web.presby.edu/~nnqadmin/nnq/nquote.html
Speak Perl rather than English, when possible
Perl is much more precise than natural language. Saying it in Perl
instead will avoid misunderstanding your question or problem.
Do not say: I have variable with "foo\tbar" in it.
Instead say: I have $var = "foo\tbar", or I have $var = 'foo\tbar',
or I have $var = <DATA> (and show the data line).
Ask perl to help you
You can ask perl itself to help you find common programming mistakes
by doing two things: enable warnings (perldoc warnings) and enable
"strict"ures (perldoc strict).
You should not bother the hundreds/thousands of readers of the
newsgroup without first seeing if a machine can help you find your
problem. It is demeaning to be asked to do the work of a machine. It
will annoy the readers of your article.
You can look up any of the messages that perl might issue to find
out what the message means and how to resolve the potential mistake
(perldoc perldiag). If you would like perl to look them up for you,
you can put "use diagnostics;" near the top of your program.
Do not re-type Perl code
Use copy/paste or your editor's "import" function rather than
attempting to type in your code. If you make a typo you will get
followups about your typos instead of about the question you are
trying to get answered.
Provide enough information
If you do the things in this item, you will have an Extremely Good
chance of getting people to try and help you with your problem!
These features are a really big bonus toward your question winning
out over all of the other posts that you are competing with.
First make a short (less than 20-30 lines) and *complete* program
that illustrates the problem you are having. People should be able
to run your program by copy/pasting the code from your article. (You
will find that doing this step very often reveals your problem
directly. Leading to an answer much more quickly and reliably than
posting to Usenet.)
Describe *precisely* the input to your program. Also provide example
input data for your program. If you need to show file input, use the
__DATA__ token (perldata.pod) to provide the file contents inside of
your Perl program.
Show the output (including the verbatim text of any messages) of
your program.
Describe how you want the output to be different from what you are
getting.
If you have no idea at all of how to code up your situation, be sure
to at least describe the 2 things that you *do* know: input and
desired output.
Do not provide too much information
Do not just post your entire program for debugging. Most especially
do not post someone *else's* entire program.
Do not post binaries, HTML, or MIME
clpmisc is a text only newsgroup. If you have images or binaries
that explain your question, put them in a publically accessible
place (like a Web server) and provide a pointer to that location. If
you include code, cut and paste it directly in the message body.
Don't attach anything to the message. Don't post vcards or HTML.
Many people (and even some Usenet servers) will automatically filter
out such messages. Many people will not be able to easily read your
post. Plain text is something everyone can read.
Social faux pas to avoid
The first two below are symptoms of lots of FAQ asking here in clpmisc.
It happens so often that folks will assume that it is happening yet
again. If you have looked but not found, or found but didn't understand
the docs, say so in your article.
Asking a Frequently Asked Question
It should be understood that you may have missed the applicable FAQ
when you checked, which is not a big deal. But if the Frequently
Asked Question is worded similar to your question, folks will assume
that you did not look at all. Don't become indignant at pointers to
the FAQ, particularly if it solves your problem.
Asking a question easily answered by a cursory doc search
If folks think you have not even tried the obvious step of reading
the docs applicable to your problem, they are likely to become
annoyed.
If you are flamed for not checking when you *did* check, then just
shrug it off (and take the answer that you got).
Asking for emailed answers
Emailed answers benefit one person. Posted answers benefit the
entire community. If folks can take the time to answer your
question, then you can take the time to go get the answer in the
same place where you asked the question.
It is OK to ask for a *copy* of the answer to be emailed, but many
will ignore such requests anyway. If you munge your address, you
should never expect (or ask) to get email in response to a Usenet
post.
Ask the question here, get the answer here (maybe).
Beware of saying "doesn't work"
This is a "red flag" phrase. If you find yourself writing that,
pause and see if you can't describe what is not working without
saying "doesn't work". That is, describe how it is not what you
want.
Sending a "stealth" Cc copy
A "stealth Cc" is when you both email and post a reply without
indicating *in the body* that you are doing so.
Be extra cautious when you get upset
Count to ten before composing a followup when you are upset
This is recommended in all Usenet newsgroups. Here in clpmisc, most
flaming sub-threads are not about any feature of Perl at all! They
are most often for what was seen as a breach of netiquette. If you
have lurked for a bit, then you will know what is expected and won't
make such posts in the first place.
But if you get upset, wait a while before writing your followup. I
recommend waiting at least 30 minutes.
Count to ten after composing and before posting when you are upset
After you have written your followup, wait *another* 30 minutes
before committing yourself by posting it. You cannot take it back
once it has been said.
AUTHOR
Tad McClellan and many others on the comp.lang.perl.misc newsgroup.
--
Tad McClellan
email: perl -le "print scalar reverse qq/moc.noitatibaher\100cmdat/"
------------------------------
Date: Fri, 23 Oct 2009 03:38:58 -0500
From: "Mumia W." <paduille.4061.mumia.w+nospam@earthlink.net>
Subject: Re: Want to write a script to note specific IP addresses.
Message-Id: <qt2dnfmjT9SB8XzXnZ2dnUVZ_oOdnZ2d@earthlink.com>
On 10/22/2009 03:13 AM, Hongyi Zhao wrote:
> On Tue, 20 Oct 2009 13:20:18 -0500, "J. Gleixner"
> <glex_no-spam@qwest-spam-no.invalid> wrote:
>
>> Quite a few modules that help with IPs, however
>> you could translate the IPs to integers and simply
>> check if one IP is between the start and end.
>>
>> Place to start:
>>
>> use Net::Netmask qw(quad2int);
>>
>>
>> To open/read/process files:
>>
>> perldoc -f open
>> perldoc perlopentut
>
> Firstly, thank you very much for your help.
>
> Here, I'll describe the solution used by me now and some issues when
> use this method:
>
> Currently, I use a complete binary IPdatabase named QQWry.Dat which
> include 373374 IP blocks oall over the world, you can download this
> IPdatabase from the following url:
>
> [...]
> My issue is that: if the IPfile is a huge one, say including several
> thousands entries in it, the above process will time consuming. So
> how can I revise the above perl script with perl's multithread module
> to improve the efficiency?
>
> Thanks in advance.
Hello Mr. Zhao. I suggest you do a few things to improve the speed of
your program:
Don't open and close the file on each read of an IP address from IPList.
Instead, keep QQWry.Dat open during the entire run. The shell script is
probably not needed, and the entire program can be written in Perl.
Use a database module such as DB_File or DBD::mysql to access the data.
Databases are designed to deal with large amounts of data quickly.
Can you write a program to convert QQWry.Dat into another form such as
.csv or tab-delimited data? I've become a little interested in this problem.
------------------------------
Date: Fri, 23 Oct 2009 17:47:49 +0800
From: Hongyi Zhao <hongyi.zhao@gmail.com>
Subject: Re: Want to write a script to note specific IP addresses.
Message-Id: <8pu2e59u0sidcn2dik1fqtqq2hocbubvq9@4ax.com>
On Fri, 23 Oct 2009 03:38:58 -0500, "Mumia W."
<paduille.4061.mumia.w+nospam@earthlink.net> wrote:
>Hello Mr. Zhao. I suggest you do a few things to improve the speed of
>your program:
This is NOT my program, it's written by others.
>
>Don't open and close the file on each read of an IP address from IPList.
>Instead, keep QQWry.Dat open during the entire run. The shell script is
>probably not needed, and the entire program can be written in Perl.
>
>Use a database module such as DB_File or DBD::mysql to access the data.
>Databases are designed to deal with large amounts of data quickly.
Thanks for your suggestion.
>
>Can you write a program to convert QQWry.Dat into another form such as
>.csv or tab-delimited data? I've become a little interested in this problem.
This sort of program also has been written by others, just search in
Google with "IPLook".
Best regards.
--
.: Hongyi Zhao [ hongyi.zhao AT gmail.com ] Free as in Freedom :.
------------------------------
Date: Fri, 23 Oct 2009 04:28:38 -0700
From: "John W. Krahn" <someone@example.com>
Subject: Re: Want to write a script to note specific IP addresses.
Message-Id: <HrgEm.48328$Lw1.20478@newsfe03.iad>
Hongyi Zhao wrote:
> On Fri, 23 Oct 2009 03:38:58 -0500, "Mumia W."
> <paduille.4061.mumia.w+nospam@earthlink.net> wrote:
>
>> Hello Mr. Zhao. I suggest you do a few things to improve the speed of
>> your program:
>
> This is NOT my program, it's written by others.
Then, do you have their permission to post their code to
comp.lang.perl.misc and beginners@perl.org?
John
--
The programmer is fighting against the two most
destructive forces in the universe: entropy and
human stupidity. -- Damian Conway
------------------------------
Date: 6 Apr 2001 21:33:47 GMT (Last modified)
From: Perl-Users-Request@ruby.oce.orst.edu (Perl-Users-Digest Admin)
Subject: Digest Administrivia (Last modified: 6 Apr 01)
Message-Id: <null>
Administrivia:
To submit articles to comp.lang.perl.announce, send your article to
clpa@perl.com.
Back issues are available via anonymous ftp from
ftp://cil-www.oce.orst.edu/pub/perl/old-digests.
#For other requests pertaining to the digest, send mail to
#perl-users-request@ruby.oce.orst.edu. Do not waste your time or mine
#sending perl questions to the -request address, I don't have time to
#answer them even if I did know the answer.
------------------------------
End of Perl-Users Digest V11 Issue 2648
***************************************
