[31789] in Perl-Users-Digest
Perl-Users Digest, Issue: 3052 Volume: 11
daemon@ATHENA.MIT.EDU (Perl-Users Digest)
Fri Jul 30 18:09:41 2010
Date: Fri, 30 Jul 2010 15:09:06 -0700 (PDT)
From: Perl-Users Digest <Perl-Users-Request@ruby.OCE.ORST.EDU>
To: Perl-Users@ruby.OCE.ORST.EDU (Perl-Users Digest)
Perl-Users Digest Fri, 30 Jul 2010 Volume: 11 Number: 3052
Today's topics:
avoiding min and max <bugbear@trim_papermule.co.uk_trim>
Re: avoiding min and max <willem@turtle.stack.nl>
Re: avoiding min and max <tzz@lifelogs.com>
Re: avoiding min and max <ben@morrow.me.uk>
Re: avoiding min and max sln@netherlands.com
Can this be done (by a noob :)) <thomas@tifozi.net>
Re: Can this be done (by a noob :)) <RedGrittyBrick@spamweary.invalid>
Re: Can this be done (by a noob :)) <tadmc@seesig.invalid>
Re: Can this be done (by a noob :)) <RedGrittyBrick@spamweary.invalid>
Re: Can this be done (by a noob :)) <bugbear@trim_papermule.co.uk_trim>
piped open and shell metacharacters <jak@isp2dial.com>
Re: piped open and shell metacharacters <nospam-abuse@ilyaz.org>
Re: piped open and shell metacharacters <jak@isp2dial.com>
Re: Posting Guidelines for comp.lang.perl.misc ($Revisi <ralph@happydays.com>
Thank you Rakudo-Star (and question about SciTE for Per <dilbert1999@gmail.com>
Digest Administrivia (Last modified: 6 Apr 01) (Perl-Users-Digest Admin)
----------------------------------------------------------------------
Date: Fri, 30 Jul 2010 11:16:11 +0100
From: bugbear <bugbear@trim_papermule.co.uk_trim>
Subject: avoiding min and max
Message-Id: <DZmdnYA_C-DxO8_RnZ2dnUVZ8oudnZ2d@brightview.co.uk>
After careful thought I have concluded that:
$x < $aa && $x < $bb
is an equivalent condition to
$x < min($aa, $bb)
I cannot presently see how this generalises
for all the comparators (< <=, == !=, >=, >) , and "max" as well
as "min".
Can anyone give me a general rule for replacing
$x <comparator> <min|max($aa, $bb)
My reasons for wanting to avoid min/max
are fairly theoretical and unimportant :-)
NugBear
------------------------------
Date: Fri, 30 Jul 2010 13:06:46 +0000 (UTC)
From: Willem <willem@turtle.stack.nl>
Subject: Re: avoiding min and max
Message-Id: <slrni55jj6.1fqi.willem@turtle.stack.nl>
bugbear wrote:
) After careful thought I have concluded that:
) $x < $aa && $x < $bb
) is an equivalent condition to
) $x < min($aa, $bb)
)
) I cannot presently see how this generalises
) for all the comparators (< <=, == !=, >=, >) , and "max" as well
) as "min".
It doesn't generalise. There are three separate cases.
The first case, you already mentioned.
The second case:
$x < max($aa, $bb)
is equivalent to
$x < $aa || $x < $bb
But the third case (equality) is more complicated:
$x == max($aa, $bb)
is equivalent to
($x == $aa && $aa <= $bb) || ($x == $bb && $bb <= $aa)
which can't be simplified further.
PS: Yes, theoretically the last case is general for all combinations,
but in the other cases it can be simplified.
SaSW, Willem
--
Disclaimer: I am in no way responsible for any of the statements
made in the above text. For all I know I might be
drugged or something..
No I'm not paranoid. You all think I'm paranoid, don't you !
#EOT
------------------------------
Date: Fri, 30 Jul 2010 09:02:55 -0500
From: Ted Zlatanov <tzz@lifelogs.com>
Subject: Re: avoiding min and max
Message-Id: <87fwz1nk34.fsf@lifelogs.com>
On Fri, 30 Jul 2010 11:16:11 +0100 bugbear <bugbear@trim_papermule.co.uk_trim> wrote:
b> After careful thought I have concluded that:
b> $x < $aa && $x < $bb
b> is an equivalent condition to
b> $x < min($aa, $bb)
b> I cannot presently see how this generalises
b> for all the comparators (< <=, == !=, >=, >) , and "max" as well
b> as "min".
b> Can anyone give me a general rule for replacing
b> $x <comparator> <min|max($aa, $bb)
You should look at Quantum::Superpositions (Perl 6 has built-in support
for the same concepts). min() and max() simply collapse the
superposition. So, for example, your conclusion is equivalent to
$x < all($aa, $bb) === $x < min($aa, $bb)
Similarly:
$x > any($aa, $bb) === $x > min($aa, $bb)
Ted
------------------------------
Date: Fri, 30 Jul 2010 14:54:19 +0100
From: Ben Morrow <ben@morrow.me.uk>
Subject: Re: avoiding min and max
Message-Id: <b52bi7-ji01.ln1@osiris.mauzo.dyndns.org>
Quoth Willem <willem@turtle.stack.nl>:
> bugbear wrote:
> ) After careful thought I have concluded that:
> ) $x < $aa && $x < $bb
> ) is an equivalent condition to
> ) $x < min($aa, $bb)
> )
> ) I cannot presently see how this generalises
> ) for all the comparators (< <=, == !=, >=, >) , and "max" as well
> ) as "min".
>
> It doesn't generalise. There are three separate cases.
> The first case, you already mentioned.
>
> The second case:
>
> $x < max($aa, $bb)
> is equivalent to
> $x < $aa || $x < $bb
>
> But the third case (equality) is more complicated:
>
> $x == max($aa, $bb)
> is equivalent to
> ($x == $aa && $aa <= $bb) || ($x == $bb && $bb <= $aa)
> which can't be simplified further.
Or to
$x == ($aa > $bb ? $aa : $bb)
which generalises like
$x == ($aa > $bb ? $aa : $bb > $cc ? $bb : $cc);
and is just a written-out max().
To the OP: max() is just
sub max {
my $rv = shift;
$_ > $rv and $rv = $_ for @_;
return $rv;
}
so there's really no need to make your code ugly to avoid it even if you
think you can't use List::Util.
(Now, of course, I'm wondering about a '<=' operator, which isn't the
usual 'less-than-or-equal' operator but instead a compare-and-assign
operator like ||=... It's hard to see what it might sensibly be called.)
Ben
------------------------------
Date: Fri, 30 Jul 2010 07:34:08 -0700
From: sln@netherlands.com
Subject: Re: avoiding min and max
Message-Id: <agn5569f5iv7cajisnape4cklis7kvr3st@4ax.com>
On Fri, 30 Jul 2010 11:16:11 +0100, bugbear <bugbear@trim_papermule.co.uk_trim> wrote:
>After careful thought I have concluded that:
> $x < $aa && $x < $bb
>is an equivalent condition to
> $x < min($aa, $bb)
>
>I cannot presently see how this generalises
>for all the comparators (< <=, == !=, >=, >) , and "max" as well
>as "min".
>
>Can anyone give me a general rule for replacing
>
>$x <comparator> <min|max($aa, $bb)
>
>My reasons for wanting to avoid min/max
>are fairly theoretical and unimportant :-)
>
> NugBear
Its hard to fathom that min/max are not functions
where its body could be stripped out and integrated
as an "equivalent condition" within a surrounding
boolean expression.
Boolean expressions say what you want, not what you
don't. Thinking this way about min/max will lead you
down the wrong path, that is a "math" path, which
doesen't really coincide with a programming "logic"
path. Code doesen't look like math, and visa versa.
Warning, pretty soon you will be disecting transendentals
for the equivalent condition.
-sln
------------------------------
Date: Fri, 30 Jul 2010 16:45:27 +0200
From: "Thomas Andersson" <thomas@tifozi.net>
Subject: Can this be done (by a noob :))
Message-Id: <8bg6qkF1gaU1@mid.individual.net>
I have set myself a task to create a script that can collect data from web
pages and insert them intoa MySQl database. I'm a complete noob at this
thougha nd not even sure what language I need (to learn), but think perl
might be it. What I ask now is not for you to tell me whow to do it, only if
it's feasible or if I'm barking up the wrong tree (pointers on where to find
relevant information is wellcome though.
First step would be to export a list of pids to be processed, each paired
with the last sid processed for the pid.
The script would read the list and set the first pid in list as current
Next step woud be for it to add current pid to a URL and load that page
containinga list.
From this page a list of sids needs to be collected untill I hit the "last
processed" one, these might be spread over severall pages so it need to keep
going either untill it finds "last processed" or there's no futher pages to
load (a fail I guess)
Next is the new sid list created in the previous step, each one need to be
processed and data collected
some basic data is collected frrom each sid and then 2 possible (but not
always excistant) lists.
The basic data collected for the sid cotains two values to be set as
variables, these decides how many data blocks needs to be collected lower
down on the page.
Go to first type block, collect the data I want and repeat as many times as
variable says
Go to seciodn type block and repeat.
Store the data collected from previous ina textfile named after pid, it
should contain 4sections of data to be inserted into 4 databases
First section update the pid with new last processed
Second section add sids with info to DB.
Third section add the data from type 1 blocks on sid pages to DB.
Fourth section section add the data from type 2 blocks on sid pages to DB.
Close the file, load next pid from list and repeat the process untill pid
list is empty.
A guess a bonus at the end would be if it could also insert all the data
collected into the db as well.
Is this something perl would be suitable for or is there a better choise?
My system is Win 7 64bit btw, running MySQL 5.1.
TIA
Thomas
------------------------------
Date: Fri, 30 Jul 2010 16:22:25 +0100
From: RedGrittyBrick <RedGrittyBrick@spamweary.invalid>
Subject: Re: Can this be done (by a noob :))
Message-Id: <4c52ee33$0$27996$db0fefd9@news.zen.co.uk>
On 30/07/2010 15:45, Thomas Andersson wrote:
> I have set myself a task to create a script that can collect data from web
> pages and insert them intoa MySQl database. I'm a complete noob at this
> thougha nd not even sure what language I need (to learn), but think perl
> might be it. What I ask now is not for you to tell me whow to do it, only if
> it's feasible or if I'm barking up the wrong tree (pointers on where to find
> relevant information is wellcome though.
It is feasible using Perl.
Other languages have subroutine libraries. Perl has "modules" for
handling specific tasks. Some modules are "core modules" that are
included with a normal Perl installation. Other modules can be found in
an online repository called CPAN. You can search it at
http://search.cpan.org/
You will need a module for fetching web pages and for extracting data
from the retrieved HTML.
You will need a module for working with MySQL. Perl's Database Interface
module is called DBI. See http://dbi.perl.org/
>
> First step would be to export a list of pids to be processed, each paired
> with the last sid processed for the pid.
> The script would read the list and set the first pid in list as current
> Next step woud be for it to add current pid to a URL and load that page
> containinga list.
> From this page a list of sids needs to be collected untill I hit the "last
> processed" one, these might be spread over severall pages so it need to keep
> going either untill it finds "last processed" or there's no futher pages to
> load (a fail I guess)
>
> Next is the new sid list created in the previous step, each one need to be
> processed and data collected
> some basic data is collected frrom each sid and then 2 possible (but not
> always excistant) lists.
> The basic data collected for the sid cotains two values to be set as
> variables, these decides how many data blocks needs to be collected lower
> down on the page.
> Go to first type block, collect the data I want and repeat as many times as
> variable says
> Go to seciodn type block and repeat.
>
> Store the data collected from previous ina textfile named after pid, it
> should contain 4sections of data to be inserted into 4 databases
> First section update the pid with new last processed
> Second section add sids with info to DB.
> Third section add the data from type 1 blocks on sid pages to DB.
> Fourth section section add the data from type 2 blocks on sid pages to DB.
>
> Close the file, load next pid from list and repeat the process untill pid
> list is empty.
>
> A guess a bonus at the end would be if it could also insert all the data
> collected into the db as well.
>
> Is this something perl would be suitable for or is there a better choise?
> My system is Win 7 64bit btw, running MySQL 5.1.
>
I confess I can't fully follow your description but I didn't notice
anything that would be difficult using Perl.
I suggest you start with a Perl script that just fetches a web page. If
you have problems, try to reproduce the problem in the smallest possible
Perl program and post that here with a short description of what you
expected to happen and what actually happened (cut & paste messages
rather than re-typing them).
There's a Posting FAQ posted regularly in this newsgroup, it is worth
reading.
--
RGB
------------------------------
Date: Fri, 30 Jul 2010 10:23:47 -0500
From: Tad McClellan <tadmc@seesig.invalid>
Subject: Re: Can this be done (by a noob :))
Message-Id: <slrni55rft.k65.tadmc@tadbox.sbcglobal.net>
Thomas Andersson <thomas@tifozi.net> wrote:
> I have set myself a task to create a script that can collect data from web
> pages and insert them intoa MySQl database.
> Is this something perl would be suitable for
Sure! I have written dozens of such programs in Perl.
perldoc -q HTML
How do I fetch an HTML file?
How do I automate an HTML form submission?
The WWW::Mechanize module is invaluable for this type of thing:
http://search.cpan.org/~petdance/WWW-Mechanize-1.64/
See also the Web Scraping Proxy:
http://www2.research.att.com/sw/tools/wsp/
--
Tad McClellan
email: perl -le "print scalar reverse qq/moc.liamg\100cm.j.dat/"
The above message is a Usenet post.
I don't recall having given anyone permission to use it on a Web site.
------------------------------
Date: Fri, 30 Jul 2010 16:26:24 +0100
From: RedGrittyBrick <RedGrittyBrick@spamweary.invalid>
Subject: Re: Can this be done (by a noob :))
Message-Id: <4c52ef23$0$28008$db0fefd9@news.zen.co.uk>
On 30/07/2010 16:22, RedGrittyBrick wrote:
> On 30/07/2010 15:45, Thomas Andersson wrote:
>> I'm a complete noob at this thougha nd not even sure what language
>> I need (to learn), but think perl might be it.
Oh yes, http://learn.perl.org/
--
RGB
------------------------------
Date: Fri, 30 Jul 2010 17:07:34 +0100
From: bugbear <bugbear@trim_papermule.co.uk_trim>
Subject: Re: Can this be done (by a noob :))
Message-Id: <COydnaroEtBaZc_RnZ2dnUVZ8sGdnZ2d@brightview.co.uk>
RedGrittyBrick wrote:
> On 30/07/2010 15:45, Thomas Andersson wrote:
>> I have set myself a task to create a script that can collect data from
>
> I confess I can't fully follow your description but I didn't notice
> anything that would be difficult using Perl.
I think he's re-inventing depth-first or breadth-first
search.
BugBear
------------------------------
Date: Fri, 30 Jul 2010 14:39:47 +0000
From: John Kelly <jak@isp2dial.com>
Subject: piped open and shell metacharacters
Message-Id: <6nn5561gou6jr71om72s922ot7psfa49nc@4ax.com>
The Camel book, 16.3.1. Anonymous Pipes says:
> Perl uses your default system shell (/bin/sh on Unix) whenever a pipe
> command contains special characters that the shell cares about. If
> you're only starting one command, and you don't need--or don't want--to
> use the shell, you can use the multi-argument form of a piped open ...
> ... But then you don't get I/O redirection, wildcard expansion, or
> multistage pipes, since Perl relies on your shell to do those.
and 29.2.104. open says
> Any pipe command containing shell metacharacters such as wildcards or
> I/O redirections is passed to your system's canonical shell (/bin/sh on
> Unix), so those shell-specific constructs can be processed first. If no
> metacharacters are found, Perl launches the new process itself without
> calling the shell.
I have a script that traps the standard output of any command passed in
as args to the script. My piped open uses 2>&1 to grab stderr as well
as stdout. I thought > was a shell metacharacter so I expected to see
/bin/sh between my script and the trapped command when doing ps ax. But
in many cases, Perl runs the trapped command directly, without needing
/bin/sh.
You can see that by running the script like this:
./myscript sleep 10
and then doing a ps ax before the sleep ends. Is the book wrong, or I
am I missing something? Here is the script:
#!/usr/bin/perl
# Define author
# John Kelly, July 28, 2010
# Define copyright
# Copyright John Kelly, 2010. All rights reserved.
# Define license
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this work except in compliance with the License.
# You may obtain a copy of the License at:
# http://www.apache.org/licenses/LICENSE-2.0
# Define symbols and (words)
# OT ........... Output Trap
# bas0 ......... basename of $0
# binx ......... binary executable
# tt ........... temporary time
use strict;
use FileHandle;
use File::Basename;
use POSIX qw (strftime);
STDOUT->autoflush (1);
STDERR->autoflush (1);
my $bas0 = basename ($0);
my $binx;
unless ($binx = shift @ARGV) {
print "Usage: ", $bas0, " binary.executable [args]\n";
exit 1;
}
my $basx = basename ($binx);
if (!defined (open OT, "$binx @ARGV 2>&1 |")) {
printf "%s -> %s: failure starting %s: $!\n", &tt, $bas0, $binx;
exit 1;
}
while (<OT>) {
/^\s*$/ && next;
printf "%s -> %s: ", &tt, $basx;
print $_;
}
if (!(close OT) && $!) {
printf "%s -> %s: failure closing OT: $!\n", &tt, $bas0;
} else {
if ($? & 127) {
printf "%s -> %s: %s signal %d, %s coredump\n", &tt, $bas0, $basx,
($? & 127), ($? & 128) ? 'with' : 'without';
} else {
printf "%s -> %s: %s exit value %d\n", &tt, $bas0, $basx, $? >> 8;
}
}
sub tt {
strftime "%a %b %e %H:%M:%S %Z %Y", localtime;
}
--
Web mail, POP3, and SMTP
http://www.beewyz.com/freeaccounts.php
------------------------------
Date: Fri, 30 Jul 2010 16:38:40 +0000 (UTC)
From: Ilya Zakharevich <nospam-abuse@ilyaz.org>
Subject: Re: piped open and shell metacharacters
Message-Id: <slrni5600g.p1i.nospam-abuse@powdermilk.math.berkeley.edu>
On 2010-07-30, John Kelly <jak@isp2dial.com> wrote:
> as args to the script. My piped open uses 2>&1 to grab stderr as well
> as stdout. I thought > was a shell metacharacter so I expected to see
> /bin/sh between my script and the trapped command when doing ps ax.
At least on Unix and OS/2, I implemented shell-less "2>&1". (For a
decade already I'm also sitting on code which does shell-less "FOO"
and 'bar', but have no time to run a test suite - so I never send it
to p5p.)
[As my other post this week shows, after I fixed the bug (in fact 2)
in the OS/2 branch I suspected for several years, my "stress test" now
passes on OS/2. On other architectures, Perl's pipe open() (and maybe
system() too???) is/are seriously buggy...]
Hope this helps,
Ilya
------------------------------
Date: Fri, 30 Jul 2010 17:40:13 +0000
From: John Kelly <jak@isp2dial.com>
Subject: Re: piped open and shell metacharacters
Message-Id: <f23656pgdmqlucb82vkgus0jcjov1gsogi@4ax.com>
On Fri, 30 Jul 2010 16:38:40 +0000 (UTC), Ilya Zakharevich
<nospam-abuse@ilyaz.org> wrote:
> on Unix and OS/2, I implemented shell-less "2>&1"
That's what I wondered. Makes sense to avoid an extra shell pid when
2>&1 is the only shell metacharacter present.
Works for me.
--
Web mail, POP3, and SMTP
http://www.beewyz.com/freeaccounts.php
------------------------------
Date: Fri, 30 Jul 2010 11:28:41 -0400
From: Ralph Malph <ralph@happydays.com>
Subject: Re: Posting Guidelines for comp.lang.perl.misc ($Revision: 1.9 $)
Message-Id: <eb206$4c52efa9$40779ac3$16718@news.eurofeeds.com>
tl, dnr
------------------------------
Date: Fri, 30 Jul 2010 07:02:33 -0700 (PDT)
From: Dilbert <dilbert1999@gmail.com>
Subject: Thank you Rakudo-Star (and question about SciTE for Perl 6)
Message-Id: <c6cac19d-e643-41a7-a783-39154143454d@f6g2000yqa.googlegroups.com>
I know, it's not really on topic in c.l.p.m, but I just can't contain
my excitement:
Perl 6/Rakudo-Star has been released yesterday:
http://rakudo.org/announce/rakudo-star/2010.07, the download page is
http://github.com/rakudo/star/downloads
I am just a simple Perl user, but I say: "Thank you very much to all
who have made this release possible".
Not all Perl 6 features are there, it is slow and it might contain
some bugs, but as far as the integration with Windows is concerned, it
works perfectly (no need to call for a Strawberry version of Rakudo-
Star).
Now, that my appetite for Perl 6 on Windows is satisfied (until the
next monthly release of Rakudo-Star) I am going to my next item on my
wish list, which is...
...does anybody know whether there is a project that integrates Perl 6
syntax for the SciTE editor ( http://www.scintilla.org/SciTE.html ) ?
(I am currently using SciTE for Perl 5 and it works perfectly)
------------------------------
Date: 6 Apr 2001 21:33:47 GMT (Last modified)
From: Perl-Users-Request@ruby.oce.orst.edu (Perl-Users-Digest Admin)
Subject: Digest Administrivia (Last modified: 6 Apr 01)
Message-Id: <null>
Administrivia:
To submit articles to comp.lang.perl.announce, send your article to
clpa@perl.com.
Back issues are available via anonymous ftp from
ftp://cil-www.oce.orst.edu/pub/perl/old-digests.
#For other requests pertaining to the digest, send mail to
#perl-users-request@ruby.oce.orst.edu. Do not waste your time or mine
#sending perl questions to the -request address, I don't have time to
#answer them even if I did know the answer.
------------------------------
End of Perl-Users Digest V11 Issue 3052
***************************************
