[30081] in Perl-Users-Digest
Perl-Users Digest, Issue: 1324 Volume: 11
daemon@ATHENA.MIT.EDU (Perl-Users Digest)
Sun Mar 2 00:10:10 2008
Date: Sat, 1 Mar 2008 21:09:06 -0800 (PST)
From: Perl-Users Digest <Perl-Users-Request@ruby.OCE.ORST.EDU>
To: Perl-Users@ruby.OCE.ORST.EDU (Perl-Users Digest)
Perl-Users Digest Sat, 1 Mar 2008 Volume: 11 Number: 1324
Today's topics:
A question about multithreading <r.ted.byers@rogers.com>
Re: A question about multithreading <joost@zeekat.nl>
Re: A question about multithreading <ben@morrow.me.uk>
Re: A question about multithreading xhoster@gmail.com
Re: Converting "’" to an Apostrophe? <nospam-abuse@ilyaz.org>
Re: Converting "’" to an Apostrophe? <stoupa@practisoft.cz>
Re: Converting "’" to an Apostrophe? <uri@stemsystems.com>
Re: Printing to screen and to file <ldolan@thinkinghatbigpond.net.au>
Re: Printing to screen and to file <ldolan@thinkinghatbigpond.net.au>
Re: Regular Expression to Replace UPPER Case Text with <joost@zeekat.nl>
Re: Regular Expression to Replace UPPER Case Text with <joost@zeekat.nl>
Re: Regular Expression to Replace UPPER Case Text with <someone@example.com>
Digest Administrivia (Last modified: 6 Apr 01) (Perl-Users-Digest Admin)
----------------------------------------------------------------------
Date: Sat, 1 Mar 2008 14:10:18 -0800 (PST)
From: Ted <r.ted.byers@rogers.com>
Subject: A question about multithreading
Message-Id: <f69a0485-5d12-40ea-8eb0-c40cbb680de4@d21g2000prf.googlegroups.com>
I managed to get my threading script to work, to a point. I can
launch an arbitrary number of threads, and have them run
concurrently. But the processor is not being fully exploited. If I
launch ONE child process in one thread, one of the four available
cores is nearly fully utilized (about 95% of that one core's cycles.
But if I launch four, the load is distributed evenly over ALL
available cores, but only about 5% of the processor's capacity is
used This means that the time I have to wait for results is twnety
times longer than it would be if I could get the processor fully
utilitized! WHY? Might it have something to do with thread priority?
If I launch the same four child processes manually, each in its own
separate commandline window, then I get the processor being fully
utilized. How do I get the same effect by launching threads or child
processes? But this gets quite tedious very quickly when there are
dozens of scripts to run!
Any suggestions would be appreciated.
Thanks
Ted
------------------------------
Date: Sun, 02 Mar 2008 00:01:52 +0100
From: Joost Diepenmaat <joost@zeekat.nl>
Subject: Re: A question about multithreading
Message-Id: <87mypi82vz.fsf@zeekat.nl>
Ted <r.ted.byers@rogers.com> writes:
> If I launch ONE child process in one thread, one of the four available
> cores is nearly fully utilized (about 95% of that one core's cycles.
This is ofcourse expected.
> But if I launch four, the load is distributed evenly over ALL
> available cores, but only about 5% of the processor's capacity is
> used This means that the time I have to wait for results is twnety
> times longer than it would be if I could get the processor fully
> utilitized! WHY? Might it have something to do with thread priority?
Since I can't see your code, I would assume this is because of the usual
cause: the threads are actually waiting on some external even like IO to
finish before they can contintue.
If your code is actually badly written, I would suspect your threads are
locking each other out.
> If I launch the same four child processes manually, each in its own
> separate commandline window, then I get the processor being fully
> utilized. How do I get the same effect by launching threads or child
> processes? But this gets quite tedious very quickly when there are
> dozens of scripts to run!
So your threads aren't sharing any data and run faster as separate
processes? use fork() instead. It may be more efficient anyway, Or show
your code.
--
Joost Diepenmaat | blog: http://joost.zeekat.nl/ | work: http://zeekat.nl/
------------------------------
Date: Sat, 1 Mar 2008 23:05:23 +0000
From: Ben Morrow <ben@morrow.me.uk>
Subject: Re: A question about multithreading
Message-Id: <j24p95-tfd.ln1@osiris.mauzo.dyndns.org>
Quoth Ted <r.ted.byers@rogers.com>:
> I managed to get my threading script to work, to a point. I can
> launch an arbitrary number of threads, and have them run
> concurrently. But the processor is not being fully exploited. If I
> launch ONE child process in one thread, one of the four available
> cores is nearly fully utilized (about 95% of that one core's cycles.
> But if I launch four, the load is distributed evenly over ALL
> available cores, but only about 5% of the processor's capacity is
> used This means that the time I have to wait for results is twnety
> times longer than it would be if I could get the processor fully
> utilitized! WHY? Might it have something to do with thread priority?
>
> If I launch the same four child processes manually, each in its own
> separate commandline window, then I get the processor being fully
> utilized. How do I get the same effect by launching threads or child
> processes? But this gets quite tedious very quickly when there are
> dozens of scripts to run!
Forget threads. They really aren't helping, and are just making things
more complicated. Just start the processes in the background, using
system 1, "...", or Win32::Process, or IPC::Run, or something else
equivalent. If necessary you should be able to start a new command
window for each process with something like system 'start cmd /c "..."'.
After that, you need to look at how your OS schedules processes, which
is probably off-topic for this group. Is there some sort of limit on the
total processor usage of a single process group (or whatever Win32 calls
it: I think it's a 'Job'?)? If so you may be able to get around this
with Win32::Job. Are you able to write a single batch file which invokes
several processes simultaneously (IIRC recent versions of cmd support
the '... &' syntax to run a process in the background)? Does that have
the same effect? If it doesn't, there may be something about the way
perl runs processes that's causing a problem, but I'd only consider that
option as a last resort.
Ben
------------------------------
Date: 02 Mar 2008 02:47:42 GMT
From: xhoster@gmail.com
Subject: Re: A question about multithreading
Message-Id: <20080301214746.004$6Q@newsreader.com>
Ted <r.ted.byers@rogers.com> wrote:
> I managed to get my threading script to work, to a point. I can
> launch an arbitrary number of threads, and have them run
> concurrently. But the processor is not being fully exploited. If I
> launch ONE child process in one thread, one of the four available
> cores is nearly fully utilized (about 95% of that one core's cycles.
> But if I launch four, the load is distributed evenly over ALL
> available cores, but only about 5% of the processor's capacity is
> used This means that the time I have to wait for results is twnety
> times longer than it would be if I could get the processor fully
> utilitized! WHY? Might it have something to do with thread priority?
>
> If I launch the same four child processes manually, each in its own
> separate commandline window, then I get the processor being fully
> utilized.
Just one processor fully utilized, or all of them fully utilized?
Is exactly the same work being accomplished via both methods? For example,
is one method running task A four times but the other is running tasks A,
B, C, and D?
> How do I get the same effect by launching threads or child
> processes? But this gets quite tedious very quickly when there are
> dozens of scripts to run!
>
> Any suggestions would be appreciated.
Create a script that fires off other scripts using whatever method is best
suited to your OS, and stop screwing around with threads.
Xho
--
-------------------- http://NewsReader.Com/ --------------------
The costs of publication of this article were defrayed in part by the
payment of page charges. This article must therefore be hereby marked
advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate
this fact.
------------------------------
Date: Sat, 1 Mar 2008 19:47:30 +0000 (UTC)
From: Ilya Zakharevich <nospam-abuse@ilyaz.org>
Subject: Re: Converting "’" to an Apostrophe?
Message-Id: <fqcbsi$18fd$1@agate.berkeley.edu>
[A complimentary Cc of this posting was sent to
Gunnar Hjalmarsson
<noreply@gunnar.cc>], who wrote in article <62pdc2F24jiheU1@mid.individual.net>:
> A. Sinan Unur wrote:
> > In the mean time, here in 2008, in the docs shipped with Perl 5.10
> >
> > perldoc perlop
> >
> > s Treat string as single line. (Make . match a newline)
>
> Great, that's a clarification I wasn't aware of.
In which way it is a clarification? IMO, the semantic of these "( )"
is clear ONLY if you know the answer already... :-( [I.e., how would
one guess that what is in () is the ONLY effect?]
Puzzled,
Ilya
------------------------------
Date: Sun, 2 Mar 2008 04:37:07 +0100
From: "Petr Vileta" <stoupa@practisoft.cz>
Subject: Re: Converting "’" to an Apostrophe?
Message-Id: <fqd7jp$1f61$1@ns.felk.cvut.cz>
Peter J. Holzer wrote:
> Works for me:
>
> #!/usr/bin/perl
> use warnings;
> use strict;
>
> my $string = 'foo ’ bar';
>
Hmm, but try this
my $string = "foo\n’\nbar";
$string =~ s/’/'/g;
print "$string\n";
__END__
prints:
foo
'
bar
perl -v
This is perl, v5.6.1 built for MSWin32-x86-multi-thread
(with 1 registered patch, see perl -V for more detail)
--
Petr Vileta, Czech republic
(My server rejects all messages from Yahoo and Hotmail. Send me your
mail from another non-spammer site please.)
Please reply to <petr AT practisoft DOT cz>
------------------------------
Date: Sun, 02 Mar 2008 04:03:46 GMT
From: Uri Guttman <uri@stemsystems.com>
Subject: Re: Converting "’" to an Apostrophe?
Message-Id: <x7r6et4vry.fsf@mail.sysarch.com>
>>>>> "PV" == Petr Vileta <stoupa@practisoft.cz> writes:
PV> Peter J. Holzer wrote:
>> Works for me:
>> #!/usr/bin/perl
>> use warnings;
>> use strict;
>> my $string = 'foo ’ bar';
>>
PV> Hmm, but try this
PV> my $string = "foo\n’\nbar";
PV> $string =~ s/’/'/g;
PV> print "$string\n";
PV> __END__
PV> prints:
PV> foo
PV> '
PV> bar
PV> This is perl, v5.6.1 built for MSWin32-x86-multi-thread
PV> (with 1 registered patch, see perl -V for more detail)
and where is the bug in that result? it is exactly what i expect perl to
print. it prints foo, a newline, a single quote, a newline, bar and a
final newline. what do you expect it to print? that regex does not use
. so /s will have no effect either enabled or disabled.
uri
--
Uri Guttman ------ uri@stemsystems.com -------- http://www.sysarch.com --
----- Perl Architecture, Development, Training, Support, Code Review ------
----------- Search or Offer Perl Jobs ----- http://jobs.perl.org ---------
--------- Gourmet Hot Cocoa Mix ---- http://bestfriendscocoa.com ---------
------------------------------
Date: Sun, 02 Mar 2008 04:49:01 GMT
From: "Peter Jamieson" <ldolan@thinkinghatbigpond.net.au>
Subject: Re: Printing to screen and to file
Message-Id: <1lqyj.21253$421.10228@news-server.bigpond.net.au>
Hi Paul, Thanks for your assistance!
I have been absent from home for some time and am only now catching up
on things.
I will try your advice asap.
Thanks again, Cheers, Peter
"Paul Lalli" <mritty@gmail.com> wrote in message
news:618543b6-6dc7-4b15-95db-89e95d42f76f@l32g2000hse.googlegroups.com...
On Feb 9, 7:09 pm, "Peter Jamieson" <ldo...@thinkinghatbigpond.net.au>
wrote:
> I can print to a file by using:
>
> open(STDOUT, ">>tmp.txt");
>
> and print warnings to a file
>
> open(STDERR, ">>tmp.txt");
>
> But what I would like to do is both print STDOUT and STDERR to a file
> but also to still view both on screen.
>
> I looked in perldoc and elsewhere but could not see how to do this.
> Any help appreciated!
Use the IO::Tee module from CPAN to create a filehandle that prints to
both the file and STDOUT, select() that filehandle, and then setup a
warn handler to print to the default filehandle rather than STDERR:
$ perl -MIO::Tee -le'
open my $fh, ">", "log.txt" or die $!;
my $tee = IO::Tee->new(\*STDOUT, $fh);
select $tee;
$SIG{__WARN__} = sub { print @_ };
print "Info";
warn "Warning";
'
Info
Warning at -e line 7.
$ cat log.txt
Info
Warning at -e line 7.
Hope this helps,
Paul Lalli
------------------------------
Date: Sun, 02 Mar 2008 04:50:18 GMT
From: "Peter Jamieson" <ldolan@thinkinghatbigpond.net.au>
Subject: Re: Printing to screen and to file
Message-Id: <emqyj.21254$421.3741@news-server.bigpond.net.au>
Hi David, Thanks for your detailed response!
I have been absent from home for some time and am only now catching up
on things.
I will try out code soon.
Thank you!, Cheers, Peter
"David Filmer" <usenet@davidfilmer.com> wrote in message
news:B-2dnchC_O6ACTPaRVn_vwA@giganews.com...
> Peter Jamieson wrote:
>
>> But what I would like to do is both print STDOUT and STDERR to a file
>> but also to still view both on screen.
>
> If you want to get really fancy and versatile you can do this with a
> logging module.
>
> My favorite is Log::Dispatch. Here is some sample code that I use in my
> "skeleton" boilerplate starter script to do all the work of setting up
> the logging methods (you need to do a bunch of stuff up-front, but once
> it's done, the actual logging is VERY simple, as you will see):
>
> ######################################################################
> ### LOGGING. Levels are: #############################################
> ### debug info notice warning error critical alert emergency ##
> ######################################################################
> ##
> use Mail::Sendmail; ##
> use Log::Dispatch; ##
> use Log::Dispatch::Screen; ##
> use Log::Dispatch::File; ##
> use Log::Dispatch::Email::MailSendmail; ##
> ##
> my $log; ##
> ##
> LOG_METHODS: { #lexically isolate variables in a block ##
> my $logdir = $cfg{'dir'}{'log'} || "/var/tmp"; ##
> my @admin_email = @{$cfg{'email'}{'admin'}} || ##
> sprintf('%s@%s', scalar getpwuid($<), hostname); ##
> ##
> unshift @{$Mail::Sendmail::mailcfg{'smtp'}}, ##
> $cfg{'email'}{'smtp'} || 'localhost'; ##
> ##
> my $add_lf = sub { my %p = @_; "$p{'message'}\n"}; ##
> my $add_indent = sub {my %p = @_; " $p{'message'}"}; #for Outlook ##
> my $add_timestamp = sub { my %p = @_; ##
> sprintf "%s - %s", scalar(localtime), ##
> $p{'message'}; }; ##
> my $add_level = sub { my %p = @_; ##
> sprintf "%-10s %s", ($p{'level'} =~ /debug/i) ##
> ? lc $p{'level'} ##
> : uc $p{'level'},##
> $p{'message'} }; ##
> ##
> $log = Log::Dispatch->new ( callbacks => [$add_level, $add_lf] ); ##
> ##
> $log ->add( Log::Dispatch::Screen ->new( ##
> name => 'screen', ##
> min_level => 'debug', ##
> stderr => 0, ) ##
> ); ##
> $log ->add( Log::Dispatch::File ->new( ##
> name => 'file', ##
> min_level => 'info', ##
> filename => sprintf ( "%s/%s.log", ##
> $logdir, ##
> $FindBin::Script ), ##
> mode => 'append', ##
> callbacks => $add_timestamp, ##
> )); ##
> $log ->add( Log::Dispatch::Email::MailSendmail ->new( ##
> name => 'email', ##
> min_level => 'error', ##
> to => \@admin_email, ##
> subject => "ERROR in $PROGRAM_NAME", ##
> callbacks => $add_indent, ##
> from => sprintf ("SERVER<%s\@%s>", ##
> (hostname =~ /^([^\.]*)/)[0], ##
> 'do-not-reply.com' ) , ##
> smtp => $cfg{'email'}{'smtp'} || 'localhost', ##
> )); ##
> }; #do ##
>
> #dispatch our very first message - print all the runtime info ##
> $log -> debug(sprintf ##
> "[%s] Begin %s\n\tVersion %s on %s as %s\n" ##
> ."\tConfigFile: %s\n\tKillfile(s):\n%s", ##
> __LINE__, __FILE__, ##
> $main::VERSION, ##
> hostname(), ##
> "$REAL_USER_ID ($ENV{'USER'})", ##
> $cfg{'inifile'}, ##
> map {"\t\t$_\n"} keys %{$cfg{'killfile'}}, ##
> ); ##
> ######################################################################
>
> (I'm using a configuration hash (%cfg) which is populated earlier, and
> I'm allowing for numerous default values within my method definitions)
>
> This creates three different types of logging methods (console, file,
> and e-mail). When we pass a message to the logger, it will fire ANY and
> ALL methods which are at (or above) the log level specified for the
> various methods. For example,
>
> $log -> debug("This message goes to the screen only");
> $log -> info("This message goes to BOTH screen and file.");
> $log -> error("This goes to screen, file, AND e-mail");
>
> I could add many other methods as well (database logging, multiple
> levels of file logging, etc).
>
> As you can see, I've added several callback functions to do things like
> add a timestamp and automatically add linefeeds. I've found that
> Outlook can do some funny things with wrapping, so I add an indention
> to e-mail messages to avoid Outlook reformatting.
>
> Check it out; it's an amazing module:
>
> http://search.cpan.org/~drolsky/Log-Dispatch-2.11/lib/Log/Dispatch.pm
------------------------------
Date: Sat, 01 Mar 2008 22:32:49 +0100
From: Joost Diepenmaat <joost@zeekat.nl>
Subject: Re: Regular Expression to Replace UPPER Case Text with lower case text
Message-Id: <878x121666.fsf@zeekat.nl>
"Peter J. Holzer" <hjp-usenet2@hjp.at> writes:
> On 2008-02-19 04:59, Charlton Wilbur <cwilbur@chromatico.net> wrote:
>>>>>>> "p" == penny <kevin.penny@gmail.com> writes:
>
> [Solution: Use \L]
>
>> p> The reason I didn't go the CFusion newsgroup first is that CF
>> p> runs with a limited perl engine and I figured you guys would
>> p> know more about regex than the cf group (eek)
>>
>> Er, no, while CF may have a PCRE library glommed into it somewhere, it
>> has no Perl content at all; and "Perl compatible" regular expressions
>> aren't.
>
> Maybe not, but \L works in perl also (reading this thread (yes, I'm a
> bit behind ...) I was astounded that nobody came up with this).
That's because you don't need to use a regex if you're going to use \L.
In fact \L isn't valid in a regex (only in the replacement string of a
regex replace operator) so the "regex" would be
$var =~ s/.*/\L$1/g;
while you could more easily use $var = "\L$var" or even better: $var =
lc($var);
--
Joost Diepenmaat | blog: http://joost.zeekat.nl/ | work: http://zeekat.nl/
------------------------------
Date: Sat, 01 Mar 2008 22:39:13 +0100
From: Joost Diepenmaat <joost@zeekat.nl>
Subject: Re: Regular Expression to Replace UPPER Case Text with lower case text
Message-Id: <874pbq15vi.fsf@zeekat.nl>
Joost Diepenmaat <joost@zeekat.nl> writes:
> $var =~ s/.*/\L$1/g;
I meant:
$var =~ s/(.*)/\L$1/g;
Which is even more ugly.
--
Joost Diepenmaat | blog: http://joost.zeekat.nl/ | work: http://zeekat.nl/
------------------------------
Date: Sat, 01 Mar 2008 23:01:51 GMT
From: "John W. Krahn" <someone@example.com>
Subject: Re: Regular Expression to Replace UPPER Case Text with lower case text
Message-Id: <zflyj.48224$FO1.42337@edtnps82>
Joost Diepenmaat wrote:
> "Peter J. Holzer" <hjp-usenet2@hjp.at> writes:
>
>> Maybe not, but \L works in perl also (reading this thread (yes, I'm a
>> bit behind ...) I was astounded that nobody came up with this).
>
> That's because you don't need to use a regex if you're going to use \L.
> In fact \L isn't valid in a regex (only in the replacement string of a
> regex replace operator)
While technically true, \L is not a valid regular expression escape
sequence, it is valid in that the m// and s/// operators which use
regular expressions are evaluated as double quoted strings so \L is
valid in that context.
> so the "regex" would be
>
> $var =~ s/.*/\L$1/g;
$ perl -le'print $& if " q " =~ /\LQ/'
q
$ perl -le'print $& if " Q " =~ /\Uq/'
Q
$ perl -le'my $x = "Q"; print $& if " q " =~ /\L$x/'
q
$ perl -le'my $x = "q"; print $& if " Q " =~ /\U$x/'
Q
John
--
Perl isn't a toolbox, but a small machine shop where you
can special-order certain sorts of tools at low cost and
in short order. -- Larry Wall
------------------------------
Date: 6 Apr 2001 21:33:47 GMT (Last modified)
From: Perl-Users-Request@ruby.oce.orst.edu (Perl-Users-Digest Admin)
Subject: Digest Administrivia (Last modified: 6 Apr 01)
Message-Id: <null>
Administrivia:
#The Perl-Users Digest is a retransmission of the USENET newsgroup
#comp.lang.perl.misc. For subscription or unsubscription requests, send
#the single line:
#
# subscribe perl-users
#or:
# unsubscribe perl-users
#
#to almanac@ruby.oce.orst.edu.
NOTE: due to the current flood of worm email banging on ruby, the smtp
server on ruby has been shut off until further notice.
To submit articles to comp.lang.perl.announce, send your article to
clpa@perl.com.
#To request back copies (available for a week or so), send your request
#to almanac@ruby.oce.orst.edu with the command "send perl-users x.y",
#where x is the volume number and y is the issue number.
#For other requests pertaining to the digest, send mail to
#perl-users-request@ruby.oce.orst.edu. Do not waste your time or mine
#sending perl questions to the -request address, I don't have time to
#answer them even if I did know the answer.
------------------------------
End of Perl-Users Digest V11 Issue 1324
***************************************
