[31757] in Perl-Users-Digest
Perl-Users Digest, Issue: 3019 Volume: 11
daemon@ATHENA.MIT.EDU (Perl-Users Digest)
Tue Jul 6 11:26:55 2010
Date: Tue, 6 Jul 2010 08:21:14 -0700 (PDT)
From: Perl-Users Digest <Perl-Users-Request@ruby.OCE.ORST.EDU>
To: Perl-Users@ruby.OCE.ORST.EDU (Perl-Users Digest)
Perl-Users Digest Tue, 6 Jul 2010 Volume: 11 Number: 3019
Today's topics:
Are there any MySQL queries or software packages for "f <ignoramus12110@NOSPAM.12110.invalid>
Re: Are there any MySQL queries or software packages fo <npeelman@cfl.rr.com>
Re: Are there any MySQL queries or software packages fo <npeelman@cfl.rr.com>
Re: Are there any MySQL queries or software packages fo <ignoramus12110@NOSPAM.12110.invalid>
Re: Are there any MySQL queries or software packages fo <npeelman@cfl.rr.com>
Re: Are there any MySQL queries or software packages fo <ignoramus12110@NOSPAM.12110.invalid>
Re: Are there any MySQL queries or software packages fo <jstucklex@attglobal.net>
Re: Are there any MySQL queries or software packages fo <ignoramus12110@NOSPAM.12110.invalid>
Re: Are there any MySQL queries or software packages fo <jstucklex@attglobal.net>
Re: Are there any MySQL queries or software packages fo (Marc Espie)
close STDERR considered puzzling <nospam-abuse@ilyaz.org>
Re: close STDERR considered puzzling <ben@morrow.me.uk>
Re: close STDERR considered puzzling <nospam-abuse@ilyaz.org>
Re: close STDERR considered puzzling <ben@morrow.me.uk>
Re: close STDERR considered puzzling <nospam-abuse@ilyaz.org>
Re: close STDERR considered puzzling <ben@morrow.me.uk>
Digest Administrivia (Last modified: 6 Apr 01) (Perl-Users-Digest Admin)
----------------------------------------------------------------------
Date: Mon, 05 Jul 2010 15:16:12 -0500
From: Ignoramus12110 <ignoramus12110@NOSPAM.12110.invalid>
Subject: Are there any MySQL queries or software packages for "finding similar items"
Message-Id: <maudnQZR9LURoK_RnZ2dnUVZ_qadnZ2d@giganews.com>
I have a MySQL database of answered algebra questions. Questions are
stored as text strings.
Examples are
``two dice are rolled. find the odds that the score on the dice is either 10 or at most 5''
``if x is the first of three consecutive even intethe product of twice a number and three is the same as the difference''
``Write the equation of the line with a slope of -1/3 and passing through the point (6, -4).''
``A flagpole casts a shadow of 32 ft, Nearby, a 10-ft tree casts a shadow of 2 ft. What is the height of the flag pole?''
``A flag pole casts a shadow of 32 feet. Nearby, a 10 foot tree casts a shadow of 2 ft. Find the height of the flag pole?''
When students ask questions, often (if not usually) there is already
something similar answered in the database. Note that I am not
defining what is "similar" and I do realize that it is a difficult
definition to make.
Example: to my human mind, questions
``A flagpole casts a shadow of 32 ft, Nearby, a 10-ft tree casts a shadow of 2 ft. What is the height of the flag pole?''
and
``A flag pole casts a shadow of 32 feet. Nearby, a 10 foot tree casts a shadow of 2 ft. Find the height of the flag pole?''
are similar.
I am hoping that, perhaps, there is some free package that could take
a few hundreds of thousands of text strings and could provide me with
"find similar" functionality.
Realizing the potential difficulty of the task, I would be content if
it worked only moderately well. I just want something along the lines.
Are there any MySQL functions or other software packages or perl
modules that provide something of the sort.
I have seen some web forums that provide "do perhaps those other
threads answer your question?" functionality by giving a list of
matching threads. None of them seems to have super cow powers, but it
looks like decent start.
So... Any suggestion for software to ran strings by similarity and
provide "top 5" or something like that?
thanks
i
------------------------------
Date: Mon, 05 Jul 2010 16:42:56 -0400
From: Norman Peelman <npeelman@cfl.rr.com>
Subject: Re: Are there any MySQL queries or software packages for "finding similar items"
Message-Id: <4c3243df$0$4962$9a6e19ea@unlimited.newshosting.com>
Ignoramus12110 wrote:
> I have a MySQL database of answered algebra questions. Questions are
> stored as text strings.
>
> Examples are
>
> ``two dice are rolled. find the odds that the score on the dice is either 10 or at most 5''
> ``if x is the first of three consecutive even intethe product of twice a number and three is the same as the difference''
> ``Write the equation of the line with a slope of -1/3 and passing through the point (6, -4).''
> ``A flagpole casts a shadow of 32 ft, Nearby, a 10-ft tree casts a shadow of 2 ft. What is the height of the flag pole?''
> ``A flag pole casts a shadow of 32 feet. Nearby, a 10 foot tree casts a shadow of 2 ft. Find the height of the flag pole?''
>
> When students ask questions, often (if not usually) there is already
> something similar answered in the database. Note that I am not
> defining what is "similar" and I do realize that it is a difficult
> definition to make.
>
> Example: to my human mind, questions
>
> ``A flagpole casts a shadow of 32 ft, Nearby, a 10-ft tree casts a shadow of 2 ft. What is the height of the flag pole?''
>
> and
>
> ``A flag pole casts a shadow of 32 feet. Nearby, a 10 foot tree casts a shadow of 2 ft. Find the height of the flag pole?''
>
> are similar.
>
> I am hoping that, perhaps, there is some free package that could take
> a few hundreds of thousands of text strings and could provide me with
> "find similar" functionality.
>
> Realizing the potential difficulty of the task, I would be content if
> it worked only moderately well. I just want something along the lines.
>
> Are there any MySQL functions or other software packages or perl
> modules that provide something of the sort.
>
> I have seen some web forums that provide "do perhaps those other
> threads answer your question?" functionality by giving a list of
> matching threads. None of them seems to have super cow powers, but it
> looks like decent start.
>
> So... Any suggestion for software to ran strings by similarity and
> provide "top 5" or something like that?
>
> thanks
>
> i
Check:
http://us.php.net/manual/en/function.soundex.php
and other links/algorithms on that page.
If you created and stored the soundex of each question in the db
(indexed) you could do a search by soundex of input question.
--
Norman
Registered Linux user #461062
-Have you been to www.mysql.com yet?-
------------------------------
Date: Mon, 05 Jul 2010 16:46:24 -0400
From: Norman Peelman <npeelman@cfl.rr.com>
Subject: Re: Are there any MySQL queries or software packages for "finding similar items"
Message-Id: <4c3244af$0$15824$9a6e19ea@unlimited.newshosting.com>
Ignoramus12110 wrote:
> I have a MySQL database of answered algebra questions. Questions are
> stored as text strings.
>
> Examples are
>
> ``two dice are rolled. find the odds that the score on the dice is either 10 or at most 5''
> ``if x is the first of three consecutive even intethe product of twice a number and three is the same as the difference''
> ``Write the equation of the line with a slope of -1/3 and passing through the point (6, -4).''
> ``A flagpole casts a shadow of 32 ft, Nearby, a 10-ft tree casts a shadow of 2 ft. What is the height of the flag pole?''
> ``A flag pole casts a shadow of 32 feet. Nearby, a 10 foot tree casts a shadow of 2 ft. Find the height of the flag pole?''
>
> When students ask questions, often (if not usually) there is already
> something similar answered in the database. Note that I am not
> defining what is "similar" and I do realize that it is a difficult
> definition to make.
>
> Example: to my human mind, questions
>
> ``A flagpole casts a shadow of 32 ft, Nearby, a 10-ft tree casts a shadow of 2 ft. What is the height of the flag pole?''
>
> and
>
> ``A flag pole casts a shadow of 32 feet. Nearby, a 10 foot tree casts a shadow of 2 ft. Find the height of the flag pole?''
>
> are similar.
>
> I am hoping that, perhaps, there is some free package that could take
> a few hundreds of thousands of text strings and could provide me with
> "find similar" functionality.
>
> Realizing the potential difficulty of the task, I would be content if
> it worked only moderately well. I just want something along the lines.
>
> Are there any MySQL functions or other software packages or perl
> modules that provide something of the sort.
>
> I have seen some web forums that provide "do perhaps those other
> threads answer your question?" functionality by giving a list of
> matching threads. None of them seems to have super cow powers, but it
> looks like decent start.
>
> So... Any suggestion for software to ran strings by similarity and
> provide "top 5" or something like that?
>
> thanks
>
> i
Looks like perl has it also:
http://perldoc.perl.org/Text/Soundex.html
--
Norman
Registered Linux user #461062
-Have you been to www.mysql.com yet?-
------------------------------
Date: Mon, 05 Jul 2010 17:33:31 -0500
From: Ignoramus12110 <ignoramus12110@NOSPAM.12110.invalid>
Subject: Re: Are there any MySQL queries or software packages for "finding similar items"
Message-Id: <BOGdnUFshcAmwK_RnZ2dnUVZ_jmdnZ2d@giganews.com>
On 2010-07-05, Axel Schwenke <axel.schwenke@gmx.de> wrote:
> Ignoramus12110 <ignoramus12110@NOSPAM.12110.invalid> wrote:
>>
>> I am hoping that, perhaps, there is some free package that could take
>> a few hundreds of thousands of text strings and could provide me with
>> "find similar" functionality.
>>
>> Realizing the potential difficulty of the task, I would be content if
>> it worked only moderately well. I just want something along the lines.
>>
>> Are there any MySQL functions or other software packages or perl
>> modules that provide something of the sort.
>
> CPAN has some packages for approximate string matching. Levenstein has
> been named. And virtually all SQL databases have SOUNDEX(). Another
> approach is trigram counting.
Thanks. Do you know any package names?
> The problem ist hard, especially when you look for a solution that runs
> faster than O(n). Outside the database you cannot be faster than O(n)
> anyway. For "few thousands" candidates it will however be fast enough.
Right now I have 208,919 candidates and the number is growing by
appx. 200 per day.
::~==>algsql "select count(*) from XXXXXXX where yyyyy = 1"
count(*)
208919
I agree that it is a hard problem.
Perl levenshtein module seems to be more single word oriented.
i
------------------------------
Date: Mon, 05 Jul 2010 19:33:13 -0400
From: Norman Peelman <npeelman@cfl.rr.com>
Subject: Re: Are there any MySQL queries or software packages for "finding similar items"
Message-Id: <4c326bb9$0$22215$9a6e19ea@unlimited.newshosting.com>
Ignoramus12110 wrote:
> On 2010-07-05, Axel Schwenke <axel.schwenke@gmx.de> wrote:
>> Ignoramus12110 <ignoramus12110@NOSPAM.12110.invalid> wrote:
>>> I am hoping that, perhaps, there is some free package that could take
>>> a few hundreds of thousands of text strings and could provide me with
>>> "find similar" functionality.
>>>
>>> Realizing the potential difficulty of the task, I would be content if
>>> it worked only moderately well. I just want something along the lines.
>>>
>>> Are there any MySQL functions or other software packages or perl
>>> modules that provide something of the sort.
>> CPAN has some packages for approximate string matching. Levenstein has
>> been named. And virtually all SQL databases have SOUNDEX(). Another
>> approach is trigram counting.
>
> Thanks. Do you know any package names?
>
>> The problem ist hard, especially when you look for a solution that runs
>> faster than O(n). Outside the database you cannot be faster than O(n)
>> anyway. For "few thousands" candidates it will however be fast enough.
>
> Right now I have 208,919 candidates and the number is growing by
> appx. 200 per day.
>
> ::~==>algsql "select count(*) from XXXXXXX where yyyyy = 1"
> count(*)
> 208919
>
> I agree that it is a hard problem.
>
> Perl levenshtein module seems to be more single word oriented.
>
> i
So does soundex now that i've tried it a bit.
--
Norman
Registered Linux user #461062
-Have you been to www.mysql.com yet?-
------------------------------
Date: Mon, 05 Jul 2010 20:52:32 -0500
From: Ignoramus12110 <ignoramus12110@NOSPAM.12110.invalid>
Subject: Re: Are there any MySQL queries or software packages for "finding similar items"
Message-Id: <mYGdnQUoqoP9Ea_RnZ2dnUVZ_vadnZ2d@giganews.com>
On 2010-07-05, Norman Peelman <npeelman@cfl.rr.com> wrote:
> Ignoramus12110 wrote:
>> On 2010-07-05, Axel Schwenke <axel.schwenke@gmx.de> wrote:
>>> Ignoramus12110 <ignoramus12110@NOSPAM.12110.invalid> wrote:
>>>> I am hoping that, perhaps, there is some free package that could take
>>>> a few hundreds of thousands of text strings and could provide me with
>>>> "find similar" functionality.
>>>>
>>>> Realizing the potential difficulty of the task, I would be content if
>>>> it worked only moderately well. I just want something along the lines.
>>>>
>>>> Are there any MySQL functions or other software packages or perl
>>>> modules that provide something of the sort.
>>> CPAN has some packages for approximate string matching. Levenstein has
>>> been named. And virtually all SQL databases have SOUNDEX(). Another
>>> approach is trigram counting.
>>
>> Thanks. Do you know any package names?
>>
>>> The problem ist hard, especially when you look for a solution that runs
>>> faster than O(n). Outside the database you cannot be faster than O(n)
>>> anyway. For "few thousands" candidates it will however be fast enough.
>>
>> Right now I have 208,919 candidates and the number is growing by
>> appx. 200 per day.
>>
>> ::~==>algsql "select count(*) from XXXXXXX where yyyyy = 1"
>> count(*)
>> 208919
>>
>> I agree that it is a hard problem.
>>
>> Perl levenshtein module seems to be more single word oriented.
>>
>> i
>
> So does soundex now that i've tried it a bit.
>
>
Yes, soundex ias for misspellings.
------------------------------
Date: Mon, 05 Jul 2010 21:56:41 -0400
From: Jerry Stuckle <jstucklex@attglobal.net>
Subject: Re: Are there any MySQL queries or software packages for "finding similar items"
Message-Id: <i0u2gq$pb7$1@news.eternal-september.org>
Ignoramus12110 wrote:
> On 2010-07-05, Norman Peelman <npeelman@cfl.rr.com> wrote:
>> Ignoramus12110 wrote:
>>> On 2010-07-05, Axel Schwenke <axel.schwenke@gmx.de> wrote:
>>>> Ignoramus12110 <ignoramus12110@NOSPAM.12110.invalid> wrote:
>>>>> I am hoping that, perhaps, there is some free package that could take
>>>>> a few hundreds of thousands of text strings and could provide me with
>>>>> "find similar" functionality.
>>>>>
>>>>> Realizing the potential difficulty of the task, I would be content if
>>>>> it worked only moderately well. I just want something along the lines.
>>>>>
>>>>> Are there any MySQL functions or other software packages or perl
>>>>> modules that provide something of the sort.
>>>> CPAN has some packages for approximate string matching. Levenstein has
>>>> been named. And virtually all SQL databases have SOUNDEX(). Another
>>>> approach is trigram counting.
>>> Thanks. Do you know any package names?
>>>
>>>> The problem ist hard, especially when you look for a solution that runs
>>>> faster than O(n). Outside the database you cannot be faster than O(n)
>>>> anyway. For "few thousands" candidates it will however be fast enough.
>>> Right now I have 208,919 candidates and the number is growing by
>>> appx. 200 per day.
>>>
>>> ::~==>algsql "select count(*) from XXXXXXX where yyyyy = 1"
>>> count(*)
>>> 208919
>>>
>>> I agree that it is a hard problem.
>>>
>>> Perl levenshtein module seems to be more single word oriented.
>>>
>>> i
>> So does soundex now that i've tried it a bit.
>>
>>
>
> Yes, soundex ias for misspellings.
Agreed. Soundex is not for trying to understand phrases or sentences.
It is to find words which "sound" alike - i.e. misspelled words.
It can't, for instance, tell the difference between "here" and "hear" -
but it can tell they sound alike.
--
==================
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.
jstucklex@attglobal.net
==================
------------------------------
Date: Mon, 05 Jul 2010 21:11:46 -0500
From: Ignoramus12110 <ignoramus12110@NOSPAM.12110.invalid>
Subject: Re: Are there any MySQL queries or software packages for "finding similar items"
Message-Id: <FpydnSL0p-9_Da_RnZ2dnUVZ_qKdnZ2d@giganews.com>
On 2010-07-06, Jerry Stuckle <jstucklex@attglobal.net> wrote:
> Ignoramus12110 wrote:
>> On 2010-07-05, Norman Peelman <npeelman@cfl.rr.com> wrote:
>>> Ignoramus12110 wrote:
>>>> On 2010-07-05, Axel Schwenke <axel.schwenke@gmx.de> wrote:
>>>>> Ignoramus12110 <ignoramus12110@NOSPAM.12110.invalid> wrote:
>>>>>> I am hoping that, perhaps, there is some free package that could take
>>>>>> a few hundreds of thousands of text strings and could provide me with
>>>>>> "find similar" functionality.
>>>>>>
>>>>>> Realizing the potential difficulty of the task, I would be content if
>>>>>> it worked only moderately well. I just want something along the lines.
>>>>>>
>>>>>> Are there any MySQL functions or other software packages or perl
>>>>>> modules that provide something of the sort.
>>>>> CPAN has some packages for approximate string matching. Levenstein has
>>>>> been named. And virtually all SQL databases have SOUNDEX(). Another
>>>>> approach is trigram counting.
>>>> Thanks. Do you know any package names?
>>>>
>>>>> The problem ist hard, especially when you look for a solution that runs
>>>>> faster than O(n). Outside the database you cannot be faster than O(n)
>>>>> anyway. For "few thousands" candidates it will however be fast enough.
>>>> Right now I have 208,919 candidates and the number is growing by
>>>> appx. 200 per day.
>>>>
>>>> ::~==>algsql "select count(*) from XXXXXXX where yyyyy = 1"
>>>> count(*)
>>>> 208919
>>>>
>>>> I agree that it is a hard problem.
>>>>
>>>> Perl levenshtein module seems to be more single word oriented.
>>>>
>>>> i
>>> So does soundex now that i've tried it a bit.
>>>
>>>
>>
>> Yes, soundex ias for misspellings.
>
> Agreed. Soundex is not for trying to understand phrases or sentences.
> It is to find words which "sound" alike - i.e. misspelled words.
>
> It can't, for instance, tell the difference between "here" and "hear" -
> but it can tell they sound alike.
>
I actually looked quite a bit, and did not find anything. Maybe my
brother in law could find something.
i
------------------------------
Date: Tue, 06 Jul 2010 07:01:48 -0400
From: Jerry Stuckle <jstucklex@attglobal.net>
Subject: Re: Are there any MySQL queries or software packages for "finding similar items"
Message-Id: <i0v2fd$iah$1@news.eternal-september.org>
Ignoramus12110 wrote:
> On 2010-07-06, Jerry Stuckle <jstucklex@attglobal.net> wrote:
>> Ignoramus12110 wrote:
>>> On 2010-07-05, Norman Peelman <npeelman@cfl.rr.com> wrote:
>>>> Ignoramus12110 wrote:
>>>>> On 2010-07-05, Axel Schwenke <axel.schwenke@gmx.de> wrote:
>>>>>> Ignoramus12110 <ignoramus12110@NOSPAM.12110.invalid> wrote:
>>>>>>> I am hoping that, perhaps, there is some free package that could take
>>>>>>> a few hundreds of thousands of text strings and could provide me with
>>>>>>> "find similar" functionality.
>>>>>>>
>>>>>>> Realizing the potential difficulty of the task, I would be content if
>>>>>>> it worked only moderately well. I just want something along the lines.
>>>>>>>
>>>>>>> Are there any MySQL functions or other software packages or perl
>>>>>>> modules that provide something of the sort.
>>>>>> CPAN has some packages for approximate string matching. Levenstein has
>>>>>> been named. And virtually all SQL databases have SOUNDEX(). Another
>>>>>> approach is trigram counting.
>>>>> Thanks. Do you know any package names?
>>>>>
>>>>>> The problem ist hard, especially when you look for a solution that runs
>>>>>> faster than O(n). Outside the database you cannot be faster than O(n)
>>>>>> anyway. For "few thousands" candidates it will however be fast enough.
>>>>> Right now I have 208,919 candidates and the number is growing by
>>>>> appx. 200 per day.
>>>>>
>>>>> ::~==>algsql "select count(*) from XXXXXXX where yyyyy = 1"
>>>>> count(*)
>>>>> 208919
>>>>>
>>>>> I agree that it is a hard problem.
>>>>>
>>>>> Perl levenshtein module seems to be more single word oriented.
>>>>>
>>>>> i
>>>> So does soundex now that i've tried it a bit.
>>>>
>>>>
>>> Yes, soundex ias for misspellings.
>> Agreed. Soundex is not for trying to understand phrases or sentences.
>> It is to find words which "sound" alike - i.e. misspelled words.
>>
>> It can't, for instance, tell the difference between "here" and "hear" -
>> but it can tell they sound alike.
>>
>
> I actually looked quite a bit, and did not find anything. Maybe my
> brother in law could find something.
>
> i
But what you're looking for is to get a computer to be a natural
language processor, which is still beyond our current programming
capabilities. IBM has recently come up with a test system ("Watson")
which does a fair job, but still has a long ways to go. Once we get
there, we'll have a Star Trek capability :)
With that said, it doesn't mean all is hopeless. Levenstein can help,
as can trigram matching and other things mentioned (except SoundEx).
But it will also require a lot of work on your part to "train" the
system as to whether two questions are similar or not.
--
==================
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.
jstucklex@attglobal.net
==================
------------------------------
Date: Tue, 6 Jul 2010 11:21:05 +0000 (UTC)
From: espie@lain.home (Marc Espie)
Subject: Re: Are there any MySQL queries or software packages for "finding similar items"
Message-Id: <i0v3j1$g1u$1@saria.nerim.net>
In article <maudnQZR9LURoK_RnZ2dnUVZ_qadnZ2d@giganews.com>,
Ignoramus12110 <ignoramus12110@NOSPAM.12110.invalid> wrote:
>I have a MySQL database of answered algebra questions. Questions are
>stored as text strings.
>
>Examples are
>
>``two dice are rolled. find the odds that the score on the dice is
>either 10 or at most 5''
>``if x is the first of three consecutive even intethe product of twice a
>number and three is the same as the difference''
>``Write the equation of the line with a slope of -1/3 and passing
>through the point (6, -4).''
>``A flagpole casts a shadow of 32 ft, Nearby, a 10-ft tree casts a
>shadow of 2 ft. What is the height of the flag pole?''
>``A flag pole casts a shadow of 32 feet. Nearby, a 10 foot tree casts a
>shadow of 2 ft. Find the height of the flag pole?''
>
>When students ask questions, often (if not usually) there is already
>something similar answered in the database. Note that I am not
>defining what is "similar" and I do realize that it is a difficult
>definition to make.
Are you hell-bent on mysql ?
Because sqlite has a fts3 extension that looks like a prime candidate
for trying to locate similar questions before using some perl approximate
code to figure out whether it's the same or not...
------------------------------
Date: Mon, 5 Jul 2010 10:13:35 +0000 (UTC)
From: Ilya Zakharevich <nospam-abuse@ilyaz.org>
Subject: close STDERR considered puzzling
Message-Id: <slrni33c2f.ol2.nospam-abuse@powdermilk.math.berkeley.edu>
perl -wle "close STDERR; open F, q(>), q(/dev/tty) or print 11;
print fileno F; warn 22; print STDERR 33; print $x+77"
Try to guess the output (use /dev/con on DOSISH, ' otherwise). Solution below:
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
perl -wle "close STDERR; open F, q(>), q(/dev/tty) or print 11;
print fileno F; warn 22; print STDERR 33; print $x+77"
Name "main::x" used only once: possible typo at -e line 1.
2
22 at -e line 1.
print() on closed filehandle STDERR at -e line 1.
Use of uninitialized value in addition (+) at -e line 1.
77
=======================================================
Of course, 2=`fileno F' is output, as well as 77. And 33 is not.
However, `warn 22' IS output, as well as the warning for `undefined $x'.
Well, at least it was not what I expected... Did anybody succeed?
Which experiences of yours helped you to guess it?
Thanks,
Ilya
------------------------------
Date: Mon, 5 Jul 2010 12:41:17 +0100
From: Ben Morrow <ben@morrow.me.uk>
Subject: Re: close STDERR considered puzzling
Message-Id: <tvs8g7-sn52.ln1@osiris.mauzo.dyndns.org>
Quoth Ilya Zakharevich <nospam-abuse@ilyaz.org>:
>
> perl -wle "close STDERR; open F, q(>), q(/dev/tty) or print 11;
> print fileno F; warn 22; print STDERR 33; print $x+77"
> Name "main::x" used only once: possible typo at -e line 1.
> 2
> 22 at -e line 1.
> print() on closed filehandle STDERR at -e line 1.
> Use of uninitialized value in addition (+) at -e line 1.
> 77
>
> =======================================================
>
> Of course, 2=`fileno F' is output, as well as 77. And 33 is not.
>
> However, `warn 22' IS output, as well as the warning for `undefined $x'.
>
> Well, at least it was not what I expected... Did anybody succeed?
> Which experiences of yours helped you to guess it?
open(2) is documented as opening the lowest-numbered currently-closed
file descriptor. If you close STDERR, and thus close fd 2, then the
next-opened file handle will be associated with fd 2. If (the
Perl-level) STDERR is currently closed, warn will write its output to
(the C-level) stderr, which means fd 2.
As a rule you should never close STD{IN,OUT,ERR}. If you must, reopen
them to or from File::Spec->devnull.
Ben
------------------------------
Date: Mon, 5 Jul 2010 13:26:49 +0000 (UTC)
From: Ilya Zakharevich <nospam-abuse@ilyaz.org>
Subject: Re: close STDERR considered puzzling
Message-Id: <slrni33ncp.pgv.nospam-abuse@powdermilk.math.berkeley.edu>
On 2010-07-05, Ben Morrow <ben@morrow.me.uk> wrote:
> If (the Perl-level) STDERR is currently closed, warn will write its
> output to (the C-level) stderr, which means fd 2.
It is THIS behaviour which causes unexpected results. Is it
documented anywhere?
> As a rule you should never close STD{IN,OUT,ERR}. If you must, reopen
> them to or from File::Spec->devnull.
I MUST close them (in all possible combinations) to find bugs in some
UNRELATEDLY obscure ;-) part of perl. And inspecting the output leads to
some deep surprises...
Thanks,
Ilya
------------------------------
Date: Mon, 5 Jul 2010 21:48:17 +0100
From: Ben Morrow <ben@morrow.me.uk>
Subject: Re: close STDERR considered puzzling
Message-Id: <h1t9g7-oi82.ln1@osiris.mauzo.dyndns.org>
Quoth Ilya Zakharevich <nospam-abuse@ilyaz.org>:
> On 2010-07-05, Ben Morrow <ben@morrow.me.uk> wrote:
>
> > If (the Perl-level) STDERR is currently closed, warn will write its
> > output to (the C-level) stderr, which means fd 2.
>
> It is THIS behaviour which causes unexpected results. Is it
> documented anywhere?
I don't know. I don't think so.
> > As a rule you should never close STD{IN,OUT,ERR}. If you must, reopen
> > them to or from File::Spec->devnull.
>
> I MUST close them (in all possible combinations) to find bugs in some
> UNRELATEDLY obscure ;-) part of perl. And inspecting the output leads to
> some deep surprises...
I see.
Ben
------------------------------
Date: Tue, 6 Jul 2010 09:23:08 +0000 (UTC)
From: Ilya Zakharevich <nospam-abuse@ilyaz.org>
Subject: Re: close STDERR considered puzzling
Message-Id: <slrni35tfs.slq.nospam-abuse@powdermilk.math.berkeley.edu>
On 2010-07-05, Ben Morrow <ben@morrow.me.uk> wrote:
>> > If (the Perl-level) STDERR is currently closed, warn will write its
>> > output to (the C-level) stderr, which means fd 2.
>>
>> It is THIS behaviour which causes unexpected results. Is it
>> documented anywhere?
> I don't know. I don't think so.
Which does not make the behaviour less puzzling...
[ I note that I have seen this behaviour explicitly mentioned on p5p
in 90s. But at that time I did not realize how puzzling this
behaviour may be. ]
BTW, is there any backdoor way to redirect STDERR to some other fd
than 2?
Thanks again,
Ilya
------------------------------
Date: Tue, 6 Jul 2010 11:46:57 +0100
From: Ben Morrow <ben@morrow.me.uk>
Subject: Re: close STDERR considered puzzling
Message-Id: <16ebg7-r1r2.ln1@osiris.mauzo.dyndns.org>
Quoth Ilya Zakharevich <nospam-abuse@ilyaz.org>:
> On 2010-07-05, Ben Morrow <ben@morrow.me.uk> wrote:
> >> > If (the Perl-level) STDERR is currently closed, warn will write its
> >> > output to (the C-level) stderr, which means fd 2.
> >>
> >> It is THIS behaviour which causes unexpected results. Is it
> >> documented anywhere?
>
> > I don't know. I don't think so.
>
> Which does not make the behaviour less puzzling...
Well, it's kind-of a last ditch effort to send the error message
*somewhere* sensible. Perl perhaps ought to prevent you from closing
these three fds altogether (or rather, implicitly reopen them to
/dev/null if you close them) and make sure they stay permanently
attached to their proper filehandles.
> [ I note that I have seen this behaviour explicitly mentioned on p5p
> in 90s. But at that time I did not realize how puzzling this
> behaviour may be. ]
>
> BTW, is there any backdoor way to redirect STDERR to some other fd
> than 2?
Oh, that's easy:
close STDERR; close STDIN;
open STDERR, ">", "/dev/tty";
open STDIN, "<", "/dev/tty";
You now have STDIN on fd 2 and STDERR on fd 0. Since STDERR is now open,
error messages will go out of fd 0.
Ben
------------------------------
Date: 6 Apr 2001 21:33:47 GMT (Last modified)
From: Perl-Users-Request@ruby.oce.orst.edu (Perl-Users-Digest Admin)
Subject: Digest Administrivia (Last modified: 6 Apr 01)
Message-Id: <null>
Administrivia:
To submit articles to comp.lang.perl.announce, send your article to
clpa@perl.com.
Back issues are available via anonymous ftp from
ftp://cil-www.oce.orst.edu/pub/perl/old-digests.
#For other requests pertaining to the digest, send mail to
#perl-users-request@ruby.oce.orst.edu. Do not waste your time or mine
#sending perl questions to the -request address, I don't have time to
#answer them even if I did know the answer.
------------------------------
End of Perl-Users Digest V11 Issue 3019
***************************************
