[25884] in Perl-Users-Digest
Perl-Users Digest, Issue: 8112 Volume: 10
daemon@ATHENA.MIT.EDU (Perl-Users Digest)
Wed May 25 03:05:33 2005
Date: Wed, 25 May 2005 00:05:08 -0700 (PDT)
From: Perl-Users Digest <Perl-Users-Request@ruby.OCE.ORST.EDU>
To: Perl-Users@ruby.OCE.ORST.EDU (Perl-Users Digest)
Perl-Users Digest Wed, 25 May 2005 Volume: 10 Number: 8112
Today's topics:
Re: clone a directory w/ copying its contents <zzzzzz@xxx.yyy>
Re: FormMail Problem <emschwar@pobox.com>
Re: FormMail Problem <apeiron+usenet@coitusmentis.info>
Re: FormMail Problem <spamtrap@dot-app.org>
Re: FormMail Problem <djames@thehub.com.au>
Re: HTML::TableExtract punctuation parsing <see.sig@rochester.rr.com>
Need help with proper file locking of flatfile data <rbutcher.nospam@hotmail.com>
Re: Need help with proper file locking of flatfile data <someone@example.com>
Re: Need help with proper file locking of flatfile data <rbutcher.nospam@hotmail.com>
Re: Need help with proper file locking of flatfile data <mark.clementsREMOVETHIS@wanadoo.fr>
simplest method for renaming files / Windows ? <geoff.cox@notquitecorrectfreeuk.com>
Re: simplest method for renaming files / Windows ? <john@castleamber.com>
Re: simplest method for renaming files / Windows ? <geoff.cox@notquitecorrectfreeuk.com>
Re: simplest method for renaming files / Windows ? <geoff.cox@notquitecorrectfreeuk.com>
Re: Strange 'for' behaviour? <tadmc@augustmail.com>
Re: substitution <tadmc@augustmail.com>
Re: substitution <tadmc@augustmail.com>
Re: substitution <1usa@llenroc.ude.invalid>
Re: substitution <dz_ny@hotmail.com>
Digest Administrivia (Last modified: 6 Apr 01) (Perl-Users-Digest Admin)
----------------------------------------------------------------------
Date: Wed, 25 May 2005 04:54:49 GMT
From: Kohn J. Wrahn <zzzzzz@xxx.yyy>
Subject: Re: clone a directory w/ copying its contents
Message-Id: <tyTke.5661$X92.2417@newsread2.news.pas.earthlink.net>
On comp.unix.shell, in <TGSke.13427$wr.6818@clgrps12>, "John W. Krahn" wrote:
One more time, Fool:
This is not a perl group.
If we wanted perl solutions, we'd go to one of the
145+ perl groups.
No one with any sense runs the offered programs of a person
that is too stupid to choose the right tool for a particular
job.
That's where competent programming BEGINS.
I'd guess that you are so bad at perl that you've been
run off of all the perl groups.
And are reduced to wandering around the Usenet offering
perl as a solution where 20K utilities will do a better job.
<snip>
AC
--
alanconnor AT earthlink DOT net
Use your real return address or I'll never know you
even tried to mail me. http://tinyurl.com/2t5kp
------------------------------
Date: Tue, 24 May 2005 19:12:54 -0600
From: Eric Schwartz <emschwar@pobox.com>
Subject: Re: FormMail Problem
Message-Id: <etohdgskt0p.fsf@wilson.emschwar>
Gunnar Hjalmarsson <noreply@gunnar.cc> writes:
> Damian James wrote:
>> I am baffled slightly that there's been an ongoing discussion of one
>> of MW's old broken cgi scripts here
>
> The discussion has been about whether it's accurate to describe
> version 1.92 of Matt's FormMail script as "horribly, horrendously
> insecure and thoroughly broken" without providing any evidence.
Here's some evidence: without getting into whether or not it's a good
idea to write your own CGI-parsing code, the CGI-parsing code in there
doesn't catch a few of the most common error cases (query string
delimited by ; not & for GET, and read not reading all data in the
buffer at once for POST). That qualifies, to me, as broken, at
least. Also, the regex for valid emails will flag as failed a number
of valid addresses.
As for security holes, there's the obvious one that HTTP_REFERER is
easily spoofable. If you're paying attention, you can restrict
somewhat the domains to which a DDOS can be directed, but it's next to
impossible to use without opening yourself up to one.
There's also a lot of ugly syntax and bad coding practises in there
I'd hate to see anybody emulate-- every function is called with
leading &, the whole get_date routine would be better off as
appropriate calls to POSIX::strftime().
So yeah, on further study I'd agree that 'broken' and 'insecure' are
empirically valid adjectives. You may disagree with the adverbs, if
you like; those are clearly value judgements.
-=Eric
--
Come to think of it, there are already a million monkeys on a million
typewriters, and Usenet is NOTHING like Shakespeare.
-- Blair Houghton.
------------------------------
Date: 25 May 2005 01:35:54 GMT
From: Christopher Nehren <apeiron+usenet@coitusmentis.info>
Subject: Re: FormMail Problem
Message-Id: <slrnd97ljq.17h8.apeiron+usenet@prophecy.dyndns.org>
On 2005-05-23, Gunnar Hjalmarsson scribbled these
curious markings:
> Even if I'm not sure, I have a feeling that you are referring to
> previous versions. If also v. 1.92 deserves that verdict, it would be
> nice if you (or somebody else) could explain *how* it's broken.
First, no taint checking. Sure, this in itself doesn't make the code
insecure, but it's certainly not helping things.
The lack of 'use CGI;', while similarly not instantly insecure, bothers
me in a similar fashion. Okay, it was written in the days of Perl 4, but
that's no reason that it couldn't have been updated.
I wouldn't rely on something like /usr/lib/sendmail (and realistically
couldn't, if taint checking were enabled), and would opt to go with
a module. This would give one portability in most cases as well as
security (assuming no external program is used).
In &parse_form, the /e operator to s/// is applied on completely
unprocessed CGI variables -- not once, but twice. I don't even want to
think about what that can cause.
Best Regards,
Christopher Nehren
--
I abhor a system designed for the "user", if that word is a coded
pejorative meaning "stupid and unsophisticated". -- Ken Thompson
If you ask the wrong people questions, you get "Joel on Software".
Unix is user friendly. However, it isn't idiot friendly.
------------------------------
Date: Tue, 24 May 2005 23:04:44 -0400
From: Sherm Pendley <spamtrap@dot-app.org>
Subject: Re: FormMail Problem
Message-Id: <84CdndtaWOzQdg7fRVn-tw@adelphia.com>
Christopher Nehren wrote:
> The lack of 'use CGI;', while similarly not instantly insecure, bothers
> me in a similar fashion. Okay, it was written in the days of Perl 4
A weak excuse at best - Perl 4 may not have had CGI.pm, but it *did*
have cgi-lib.pl.
sherm--
--
Cocoa programming in Perl: http://camelbones.sourceforge.net
Hire me! My resume: http://www.dot-app.org
------------------------------
Date: Wed, 25 May 2005 03:38:17 GMT
From: Damian James <djames@thehub.com.au>
Subject: Re: FormMail Problem
Message-Id: <slrnd97spa.coq.djames@stumble.qimr.edu.au>
On Wed, 25 May 2005 02:42:54 +0200, Gunnar Hjalmarsson said:
> Damian James wrote:
>> I am baffled slightly that there's been an ongoing discussion of one
>> of MW's old broken cgi scripts here
>
> The discussion has been about whether it's accurate to describe version
> 1.92 of Matt's FormMail script as "horribly, horrendously insecure and
> thoroughly broken" without providing any evidence. I notice that also
> you refer to it as "broken", but that does not make a more accurate
> description just because it's being repeated, does it?
I am amazes that you consider it worth anyone's time to do that. It has been
done, and done many, many times. Perhaps not in this thread but over several
years in this newgroup, quite possibly hundreds of times. I refuse to be
specific or enumerate that, and insist that it is worth noone's time to do so
again, and again, and again. Search the archives, for goodness' sake.
> ...
> You could have viewed my first reply in this thread instead:
> http://groups-beta.google.com/group/comp.lang.perl.misc/msg/c8b264fe6eb8bec8
>
Fine. Glad to see that someone had the sense to point that out. Given that
an example of a known-good version of the program exists, why is this being
discussed at all? Move on. Next!
;)
--damian
------------------------------
Date: Wed, 25 May 2005 02:40:09 GMT
From: Bob Walton <see.sig@rochester.rr.com>
Subject: Re: HTML::TableExtract punctuation parsing
Message-Id: <dARke.15262$zd5.2984@twister.nyroc.rr.com>
Maqo wrote:
> Bob Walton wrote:
>
>> My browser says that web page is Unicode with UTF-8 encoding. If you
>> process it as Unicode with UTF-8 encoding, you'll probably be fine.
>> Otherwise, as you noted, you'll get gibberish. If you view the
>> results of your print() with a Unicode with UTF-8 viewer, you should
>> be OK, as you are doing nothing that should alter the non-ASCII
>> characters.
>
>
> Thanks Bob, that's what I had suspected as well, which is why I can't
> for the life of me understand why this is still giving me gibberish (I
> must be missing something with respect to proper decoding of UTF-8):
>
> use LWP::UserAgent;
> use HTML::Encoding 'encoding_from_http_message';
> use Encode;
>
> my $URL =
> "http://www.pimco.com/LeftNav/Late+Breaking+Commentary/IO/2005/IO+May-June+2005.htm";
>
>
> my $content = LWP::UserAgent->new->get($URL, 'Accept-Charset'=>'UTF-8');
> my $enco = encoding_from_http_message($content);
> my $utf8 = decode($enco => $content->content());
> open (OUT, ">:encoding(utf8)", "out.html");
> print OUT $utf8;
> close (OUT);
Well, I'm certainly no expert at all these encodings, but I note
that when running your program above verbatim, one still ends up
with "out.html" containing UTF-8 encoded Unicode. In fact,
out.html is character-for-character identical with the file
generated from:
use LWP::Simple;
open OUT,">out1.html" or die "Oops, $!";
print OUT get('http://www.p...');
#[trailing portion of long URL elided]
It seems that what you really want to do is convert the "weird"
quote and apostrophe characters and the em-dash from Unicode to
their nearest ASCII equivalents. There is certainly no
general-purpose converter to take Unicode and make "best guess"
ASCII out of it (what would it do with Chinese characters, for
example?). Perl can convert the UTF-8 encoding to true Unicode
in Perl strings (which apparently is happening with your $utf8
variable), and one could then use the tr/// operator to convert
the unwanted codes to the ASCII characters you want to use as
their approximation.
For example, try adding this line to your above program just
after your "my $utf8..." line and before the open():
$utf8=~tr/\x{2019}\x{201c}\x{201d}\x{2013}/'""-/;
and see if that will suffice. It appears as if the call to
->decode() of the Encode module is needed to convert the UTF-8
encoding from the web page to a true Unicode string. It may thus
be misleading to call it $utf8 -- perhaps $unicode would be more
descriptive?
BTW, you should test your open to ensure it executed
successfully. The typical paradigm is:
open(...) or die "Your error message, $!";
--
Bob Walton
Email: http://bwalton.com/cgi-bin/emailbob.pl
------------------------------
Date: Wed, 25 May 2005 05:01:53 GMT
From: "Randy" <rbutcher.nospam@hotmail.com>
Subject: Need help with proper file locking of flatfile data
Message-Id: <5FTke.1476088$6l.321507@pd7tw2no>
Hello,
I seem to be having some difficulties locking a simple counter flatfile
database. To start off, here is the code I am presently using:
#!/usr/bin/perl
use Fcntl qw(:DEFAULT :flock);
open (LOCKERFILE, "<$activeuser/locker.txt") or die "Can't open: $!";
flock (LOCKERFILE, LOCK_EX) or die "Can't lock: $!";
open (GETCOUNT, "<$activeuser/counter.txt") or die "Can't open file: $!";
flock (GETCOUNT, LOCK_SH) or die "Can't lock: $!";
$currentcount=<GETCOUNT>;
($date,$count,$users)=split(/\|/,$currentcount);
close(GETCOUNT);
$count = $count + 1;
open (WRITECOUNT, ">$activeuser/counter.txt") or die "Can't open file: $!";
flock (WRITECOUNT, LOCK_SH) or die "Can't lock: $!";
print WRITECOUNT "$date|$count|$users";
close(WRITECOUNT);
close(LOCKERFILE);
exit;
I use the locker.txt file to start a primary lock before the reading and
writing begins, suggested to me by a user from this newsgroup long ago. The
script seems to work most of the time, but every now and then I check the
data and its completely gone ... only leaving the data separators | | | So I
guess it doesnt really work well after all.
I would be interested in knowing if this script is infact flawed in my
approach or if the data loss is being caused by something else. I have
really done lots of reading on locking flatfile databases and there seems to
be alot of opinions, but few hard facts on how to do it correctly.
This script performs a very simple function but is very important to me to
make it work right without losing my data. I would very much value feedback
on this. Many thanks in advance.
Robert
------------------------------
Date: Wed, 25 May 2005 05:20:54 GMT
From: "John W. Krahn" <someone@example.com>
Subject: Re: Need help with proper file locking of flatfile data
Message-Id: <WWTke.9376$HI.2387@edtnps84>
Randy wrote:
>
> I seem to be having some difficulties locking a simple counter flatfile
> database.
That is covered by the FAQ:
perldoc -q "I still don't get locking. I just want to increment the number in
the file. How can I do this"
John
--
use Perl;
program
fulfillment
------------------------------
Date: Wed, 25 May 2005 05:40:59 GMT
From: "Randy" <rbutcher.nospam@hotmail.com>
Subject: Re: Need help with proper file locking of flatfile data
Message-Id: <LdUke.1473649$Xk.954847@pd7tw3no>
"John W. Krahn" <someone@example.com> wrote in message
news:WWTke.9376$HI.2387@edtnps84...
> Randy wrote:
> >
> > I seem to be having some difficulties locking a simple counter flatfile
> > database.
>
> That is covered by the FAQ:
>
> perldoc -q "I still don't get locking. I just want to increment the
number in
> the file. How can I do this"
I have checked perldoc many times. perldoc has only limited information on
locking, and it's method is not adequate for high traffic flatfile database
locking. I have learned that a sentinel locking file is essential, which
perldoc does not cover.
Robert
------------------------------
Date: 25 May 2005 06:34:33 GMT
From: "Mark Clements" <mark.clementsREMOVETHIS@wanadoo.fr>
Subject: Re: Need help with proper file locking of flatfile data
Message-Id: <42941c79$0$11707$8fcfb975@news.wanadoo.fr>
Randy wrote:
> Hello,
>
> I seem to be having some difficulties locking a simple counter
> flatfile database. To start off, here is the code I am presently
> using:
>
> #!/usr/bin/perl
> use Fcntl qw(:DEFAULT :flock);
You need to have
use strict;
use warnings;
Read the posting guidelines for why.
>
> open (LOCKERFILE, "<$activeuser/locker.txt") or die "Can't open: $!";
> flock (LOCKERFILE, LOCK_EX) or die "Can't lock: $!";
You are trying to get an exclusive lock on a file opened read-only.
I get:
bob 548 $ cat testflock.pl
#!/usr/local/bin/perl
use strict;
use warnings;
use Fcntl qw(:DEFAULT :flock);
my $filename = shift;
open IN,"<$filename" or die $!;
flock IN,LOCK_EX or die $!;
bob 549 $ perl testflock.pl testlock
Bad file number at testflock.pl line 9.
bob 550 $ ll testlock
-rw-r--r-- 1 mark cvs 0 May 25 07:13 testlock
The following comments are sort of academic because you are using an
external lockfile, but...
>
> open (GETCOUNT, "<$activeuser/counter.txt") or die "Can't open file:
> $!"; flock (GETCOUNT, LOCK_SH) or die "Can't lock: $!";
> $currentcount=<GETCOUNT>;
> ($date,$count,$users)=split(/\|/,$currentcount);
> close(GETCOUNT);
You have a race condition here: you have opened the file read-only with
a shared lock, and then you close it. It is possible that another
process will come and read the file in between this close and the next
open.
> $count = $count + 1;
>
> open (WRITECOUNT, ">$activeuser/counter.txt") or die "Can't open
> file: $!"; flock (WRITECOUNT, LOCK_SH) or die "Can't lock: $!";
> print WRITECOUNT "$date|$count|$users";
> close(WRITECOUNT);
>
> close(LOCKERFILE);
> exit;
What happens if you duplicate the code in the perldoc that you were
pointed to? Are you running over NFS?
Mark
------------------------------
Date: Wed, 25 May 2005 05:41:35 GMT
From: Geoff Cox <geoff.cox@notquitecorrectfreeuk.com>
Subject: simplest method for renaming files / Windows ?
Message-Id: <in3891pqnjhadgo4ctmi8led8o2fktqu5k@4ax.com>
Hello
At the moment I
1. use Find::File
2. open each file and print OUT each line to files with a new name
3. delete the original files
4. rename the new files to the original ones with another Perl script
I know there are better ways - but am confused as to which to use.
Which is the simplest?
Thanks
Geoff
------------------------------
Date: 25 May 2005 06:00:23 GMT
From: John Bokma <john@castleamber.com>
Subject: Re: simplest method for renaming files / Windows ?
Message-Id: <Xns9661A3C58D45castleamber@130.133.1.4>
Geoff Cox wrote:
> Hello
>
> At the moment I
>
> 1. use Find::File
> 2. open each file and print OUT each line to files with a new name
> 3. delete the original files
> 4. rename the new files to the original ones with another Perl script
>
> I know there are better ways - but am confused as to which to use.
> Which is the simplest?
rename, *but* be careful not to rename new files over old files etc.
--
John Small Perl scripts: http://johnbokma.com/perl/
Perl programmer available: http://castleamber.com/
Happy Customers: http://castleamber.com/testimonials.html
------------------------------
Date: Wed, 25 May 2005 06:04:59 GMT
From: Geoff Cox <geoff.cox@notquitecorrectfreeuk.com>
Subject: Re: simplest method for renaming files / Windows ?
Message-Id: <b658915dkcfl62877063b66hepekcnbeut@4ax.com>
On 25 May 2005 06:00:23 GMT, John Bokma <john@castleamber.com> wrote:
>rename, *but* be careful not to rename new files over old files etc.
John,
I thought rename did not apply to Perl with Windows? Only for unix? am
I wrong?
Assuming it does work do I have to?
1. rename orig files to new names
2. delete orig files
3. rename new files to orig file names
Cannot just rename files in one step?
Geoff
------------------------------
Date: Wed, 25 May 2005 06:27:32 GMT
From: Geoff Cox <geoff.cox@notquitecorrectfreeuk.com>
Subject: Re: simplest method for renaming files / Windows ?
Message-Id: <vi6891tknivdpmqn60tg8bbtgnjpk4jmim@4ax.com>
On 25 May 2005 06:00:23 GMT, John Bokma <john@castleamber.com> wrote:
John,
I have just tried
rename ("d://perl/progs/test/file1.txt","file2.txt") ;
and
this appears to delete the orig file and create new file with the new
name ...
Cheers
Geoff
------------------------------
Date: Tue, 24 May 2005 22:46:05 -0500
From: Tad McClellan <tadmc@augustmail.com>
Subject: Re: Strange 'for' behaviour?
Message-Id: <slrnd97t7t.3u0.tadmc@magna.augustmail.com>
Tim Hammerquist <tim@vegeta.ath.cx> wrote:
> A. Sinan Unur <1usa@llenroc.ude.invalid> wrote:
>> Tim Hammerquist <tim@vegeta.ath.cx> wrote:
>> > for (my $year = 2005; $year >= 1999; $year++) {...}
>>
>> ITYM
>>
>> for (my $year = 2005; $year >= 1999; $year--) {...}
>
> Indeed. Thanks for the correction.
>
> /me reverts to "lurk" mode until he can learn to post while not half
> asleep.
No, no! Let's simply turn it into a Learning Opportunity and you'll
end up being credited with starting a valuable sub-thread.
Lessee...
Before I even noticed the typo, I was going to suggest
foreach my $year ( reverse 1999 .. 2005 )
because it would be harder to screw that one up.
Then I noticed that you had proven my case!
Thanks man.
Moral (IM(NS)HO, of course):
TMTOWTDI allows you to choose the way that is easiest
to follow and maintain, so choose that one.
Sub-Moral:
Review the Three Rules of Software Development:
Optimize for labor, optimize for labor and optimize for labor.
There. That ought to do it.
--
Tad McClellan SGML consulting
tadmc@augustmail.com Perl programming
Fort Worth, Texas
------------------------------
Date: Tue, 24 May 2005 22:18:08 -0500
From: Tad McClellan <tadmc@augustmail.com>
Subject: Re: substitution
Message-Id: <slrnd97rjg.3u0.tadmc@magna.augustmail.com>
Eric Schwartz <emschwar@pobox.com> wrote:
> s/^\$ERROR.*$//;
or maybe
$_ = '' if /^\$ERROR/;
easier to see, to my eye.
--
Tad McClellan SGML consulting
tadmc@augustmail.com Perl programming
Fort Worth, Texas
------------------------------
Date: Tue, 24 May 2005 22:23:29 -0500
From: Tad McClellan <tadmc@augustmail.com>
Subject: Re: substitution
Message-Id: <slrnd97rth.3u0.tadmc@magna.augustmail.com>
Alexandre Jaquet <alexj@freesurf.ch> wrote:
> Eric Schwartz a écrit :
>> Please show us a complete script, with example data, showing what your
>> data looks like before and after.
> here is a part of my cgi script :
It is both too big of a part and too little of a part.
Too big because it is more that 20-30 lines.
Too little because it is not a complete script as was requested.
Post a short and complete program *that we can run*, and we will
surely be able to help solve your problem.
Don't, and we probably can't. Your choice.
Have you seen the Posting Guidelines that are posted here frequently?
--
Tad McClellan SGML consulting
tadmc@augustmail.com Perl programming
Fort Worth, Texas
------------------------------
Date: Wed, 25 May 2005 04:02:51 GMT
From: "A. Sinan Unur" <1usa@llenroc.ude.invalid>
Subject: Re: substitution
Message-Id: <Xns96614A89059asu1cornelledu@127.0.0.1>
Alexandre Jaquet <alexj@freesurf.ch> wrote in
news:4293b8cc$0$1162$5402220f@news.sunrise.ch:
<a ton of code>
You should seriusly consider using CGI::Application with HTML::Template.
--
A. Sinan Unur <1usa@llenroc.ude.invalid>
(reverse each component and remove .invalid for email address)
comp.lang.perl.misc guidelines on the WWW:
http://mail.augustmail.com/~tadmc/clpmisc/clpmisc_guidelines.html
------------------------------
Date: Wed, 25 May 2005 04:11:05 GMT
From: "David" <dz_ny@hotmail.com>
Subject: Re: substitution
Message-Id: <tVSke.1621$XB2.487758@twister.nyc.rr.com>
s/\$ERROR(\S+)?/ /g;
-David
"Alexandre Jaquet" <alexj@freesurf.ch> wrote in message
news:4293adb0$0$1157$5402220f@news.sunrise.ch...
> Hi,
>
> I would like to replace all fields values who begin by $ERROR with empty
> value:
>
> I've try something like :
> s/\$ERROR{'([\w]+)'}/ /;
>
> but didn't work
>
> any help ?
>
> cheers
>
> Alexandre Jaquet
------------------------------
Date: 6 Apr 2001 21:33:47 GMT (Last modified)
From: Perl-Users-Request@ruby.oce.orst.edu (Perl-Users-Digest Admin)
Subject: Digest Administrivia (Last modified: 6 Apr 01)
Message-Id: <null>
Administrivia:
#The Perl-Users Digest is a retransmission of the USENET newsgroup
#comp.lang.perl.misc. For subscription or unsubscription requests, send
#the single line:
#
# subscribe perl-users
#or:
# unsubscribe perl-users
#
#to almanac@ruby.oce.orst.edu.
NOTE: due to the current flood of worm email banging on ruby, the smtp
server on ruby has been shut off until further notice.
To submit articles to comp.lang.perl.announce, send your article to
clpa@perl.com.
#To request back copies (available for a week or so), send your request
#to almanac@ruby.oce.orst.edu with the command "send perl-users x.y",
#where x is the volume number and y is the issue number.
#For other requests pertaining to the digest, send mail to
#perl-users-request@ruby.oce.orst.edu. Do not waste your time or mine
#sending perl questions to the -request address, I don't have time to
#answer them even if I did know the answer.
------------------------------
End of Perl-Users Digest V10 Issue 8112
***************************************
