[31428] in Perl-Users-Digest
Perl-Users Digest, Issue: 2680 Volume: 11
daemon@ATHENA.MIT.EDU (Perl-Users Digest)
Tue Nov 17 18:09:46 2009
Date: Tue, 17 Nov 2009 15:09:08 -0800 (PST)
From: Perl-Users Digest <Perl-Users-Request@ruby.OCE.ORST.EDU>
To: Perl-Users@ruby.OCE.ORST.EDU (Perl-Users Digest)
Perl-Users Digest Tue, 17 Nov 2009 Volume: 11 Number: 2680
Today's topics:
Re: Converting images without Image::Magick <jwcarlton@gmail.com>
Re: Converting images without Image::Magick <ben@morrow.me.uk>
Failed connection to sftp server via Net::SFTP <se5u0o702@sneakemail.com>
Re: Failed connection to sftp server via Net::SFTP <glex_no-spam@qwest-spam-no.invalid>
Re: Failed connection to sftp server via Net::SFTP <se5u0o702@sneakemail.com>
Re: How to upper case letters in a pair of parenthesis <tadmc@seesig.invalid>
Re: How to upper case letters in a pair of parenthesis <OJZGSRPBZVCX@spammotel.com>
Re: How to upper case letters in a pair of parenthesis <willem@stack.nl>
Re: How to upper case letters in a pair of parenthesis (Randal L. Schwartz)
Re: How to upper case letters in a pair of parenthesis sln@netherlands.com
Re: mingw Perl: no long doubles? (Seymour J.)
Re: My perl script to run arbitrary tasks in parallel <ignoramus28865@NOSPAM.28865.invalid>
Re: My perl script to run arbitrary tasks in parallel despen@verizon.net
Re: My perl script to run arbitrary tasks in parallel <ignoramus28865@NOSPAM.28865.invalid>
Re: My perl script to run arbitrary tasks in parallel <ignoramus28865@NOSPAM.28865.invalid>
Re: My perl script to run arbitrary tasks in parallel despen@verizon.net
Re: My perl script to run arbitrary tasks in parallel <ignoramus28865@NOSPAM.28865.invalid>
parse the logs and search for matching string <cyrusgreats@gmail.com>
Re: parse the logs and search for matching string <cyrusgreats@gmail.com>
Re: parse the logs and search for matching string sln@netherlands.com
Re: parse the logs and search for matching string sln@netherlands.com
Digest Administrivia (Last modified: 6 Apr 01) (Perl-Users-Digest Admin)
----------------------------------------------------------------------
Date: Tue, 17 Nov 2009 09:56:13 -0800 (PST)
From: Jason Carlton <jwcarlton@gmail.com>
Subject: Re: Converting images without Image::Magick
Message-Id: <1cfd5d07-9362-4afc-a603-13766cfbcadf@k17g2000yqh.googlegroups.com>
On Nov 17, 4:15=A0am, Oliver 'ojo' Bedford <news...@web.de> wrote:
> Am Tue, 17 Nov 2009 00:39:01 -0800 schrieb Jason Carlton:
>
> > All that I'm using it for is to convert BMP, GIF, and PNG files to JPG.
> > Is there another way to do this? I've been using Imager; I don't suppos=
e
> > that you can convert an image with that module, can you?
>
> =A0 From the man page Imager::Files:
> =A0 =A0 =A0 The "type" parameter is a lowercase representation of the fil=
e type,
> =A0 =A0 =A0 =A0and can be any of the following:
>
> =A0 =A0 =A0 =A0 =A0bmp =A0 Windows BitMaP (BMP)
> =A0 =A0 =A0 =A0 =A0gif =A0 Graphics Interchange Format (GIF)
> =A0 =A0 =A0 =A0 =A0jpeg =A0JPEG/JFIF
> =A0 =A0 =A0 =A0 =A0png =A0 Portable Network Graphics (PNG)
> =A0 =A0 =A0 =A0 =A0pnm =A0 Portable aNyMap (PNM)
> =A0 =A0 =A0 =A0 =A0raw =A0 Raw
> =A0 =A0 =A0 =A0 =A0sgi =A0 SGI .rgb files
> =A0 =A0 =A0 =A0 =A0tga =A0 TARGA
> =A0 =A0 =A0 =A0 =A0tiff =A0Tagged Image File Format (TIFF)
>
> =A0 Oliver
I had read that, but just thought it meant that these were the file
types that it would recognize. I didn't see it say anything about
converting.
Can I simply do this:
use Imager;
$path =3D "/home/mydomain/www/images"
$pic =3D "leapord.bmp";
my $image =3D Imager->new;
# Read original $pic
$image->read(file =3D> "$path/$pic");
# Resize to width=3D300, height proportionate
$resize_pic =3D $image->scale(xpixels=3D>300, qtype =3D> 'mixing');
# Write as .jpg
$resize_pic->write(file =3D> "$path/$filename.jpg");
# I'm assuming that writing it with a different name will leave the
original
# image on the server, so it needs to be deleted
if (lc($ext) ne "jpg") { unlink("$path/$filename.$ext"); }
------------------------------
Date: Tue, 17 Nov 2009 18:47:04 +0000
From: Ben Morrow <ben@morrow.me.uk>
Subject: Re: Converting images without Image::Magick
Message-Id: <8m7bt6-j2u1.ln1@osiris.mauzo.dyndns.org>
Quoth Jason Carlton <jwcarlton@gmail.com>:
> On Nov 17, 4:15 am, Oliver 'ojo' Bedford <news...@web.de> wrote:
> >
> > From the man page Imager::Files:
> > The "type" parameter is a lowercase representation of the file type,
> > and can be any of the following:
> >
> > bmp Windows BitMaP (BMP)
> > gif Graphics Interchange Format (GIF)
> > jpeg JPEG/JFIF
> > png Portable Network Graphics (PNG)
> > pnm Portable aNyMap (PNM)
> > raw Raw
> > sgi SGI .rgb files
> > tga TARGA
> > tiff Tagged Image File Format (TIFF)
>
> I had read that, but just thought it meant that these were the file
> types that it would recognize. I didn't see it say anything about
> converting.
>
> Can I simply do this:
<snip>
What happened when you tried it?
(Or indeed, what does the first example in perldoc Imager look like?)
Ben
------------------------------
Date: Tue, 17 Nov 2009 10:34:19 -0800 (PST)
From: Kevin13 <se5u0o702@sneakemail.com>
Subject: Failed connection to sftp server via Net::SFTP
Message-Id: <dc2287a1-e481-4e0f-8334-1fa00864ecf7@u16g2000pru.googlegroups.com>
Hi. Following perl code fails to connect to SFTP server (user-id/
password authentication only supported on the server). Has
consistently worked for last year, so likely a change on far end.
Looking for some clue to pass along to techs that support the sftp
server. Diagnostic messages follow code.
Any thoughts or insights would be much appreciated!
#!/opt/perl/bin/perl
#
# Syntax:
#
# perl sftp_dbg.pl [user-id] [password] [file_name] [D]
#
#
use diagnostics;
use Net::SFTP;
use strict;
# Constants:
my $host = "xxx.xxx.gov.uk";
my $path = "/report_upload";
# Read command line for rest:
my $userid = shift @ARGV;
my $passwd = shift @ARGV;
my $trsfile = shift @ARGV;
my $debug = shift @ARGV;
# Debug mode?
($debug =~ "[Dd]") ? ($debug = "true") : ($debug = "false");
# Set up sftp params:
my %args = (
user => $userid,
password => $passwd,
debug => $debug,
ssh_args => [port => 22] );
my $localfile = $trsfile;
my $remotefile = $path . "/" . $trsfile;
# Main:
print "\nUploading ", $localfile, " to ", $remotefile, "...\n\n";
if ($debug eq "true") {
print "Contents of '\%args' hash:\n";
foreach my $key (keys %args) {
print "At $key we have $args{$key}\n";
}
print "\n";
}
my $sftp = Net::SFTP->new($host, %args);
print "Session opened...\n\n";
# Contrary to documentation, returns zero (false) on success; "true"
on error.
if (!($sftp->put($localfile, $remotefile))) {
print "\nFile uploaded successfully...\n";
### print "\nTRUE status received...\n";
exit;
} else {
# Note: Depending on error, script may just choke here. But
still sets $status flag.
print "\nERROR: File NOT uploaded!\n";
### print "\nFALSE status received...\n";
exit 1;
}
Output:
[54]/home/jones: /opt/perl/bin/perl -w ./sftpTRSdbg.pl user-id
password testfile.txt D
Uploading testfile.txt to /report_upload/testfile.txt...
Contents of '%args' hash:
At password we have password
At debug we have true
At user we have user-id
At ssh_args we have ARRAY(0x14ba7c)
bobcat: Reading configuration data /home/jones/.ssh/config
bobcat: Reading configuration data /etc/ssh_config
bobcat: Connecting to xxx.xxx.gov.uk, port 22.
bobcat: Remote protocol version 2.0, remote software version
ReflectionForSecureIT_7.1.0.246
bobcat: Net::SSH::Perl Version 1.24, protocol version 2.0.
.obcat: No compat match: ReflectionForSecureIT_7.1.0.246
bobcat: Connection established.
bobcat: Sent key-exchange init (KEXINIT), wait response.
bobcat: Algorithms, c->s: 3des-cbc hmac-sha1 none
bobcat: Algorithms, s->c: 3des-cbc hmac-sha1 none
bobcat: Entering Diffie-Hellman Group 1 key exchange.
bobcat: Sent DH public key, waiting for reply.
bobcat: Received host key, type 'ssh-dss'.
bobcat: Host 'xxx.xxx.gov.uk' is known and matches the host key.
bobcat: Computing shared secret key.
bobcat: Verifying server signature.
bobcat: Waiting for NEWKEYS message.
bobcat: Enabling incoming encryption/MAC/compression.
bobcat: Send NEWKEYS, enable outgoing encryption/MAC/compression.
bobcat: Sending request for user-authentication service.
bobcat: Service accepted: ssh-userauth.
bobcat: Trying empty user-authentication request.
bobcat: Authentication methods that can continue: publickey,keyboard-
interactive,password.
bobcat: Next method to try is publickey.
bobcat: Next method to try is password.
bobcat: Trying password authentication.
bobcat: Login completed, opening dummy shell channel.
bobcat: channel 0: new [client-session]
bobcat: Requesting channel_open for channel 0.
bobcat: channel 0: open confirm rwindow 0 rmax 32768
bobcat: Got channel open confirmation, requesting shell.
bobcat: Requesting service shell on channel 0.
bobcat: channel 1: new [client-session]
bobcat: Requesting channel_open for channel 1.
bobcat: channel 0: rcvd eof
bobcat: channel 0: output open -> drain
bobcat: channel 0: rcvd close
bobcat: channel 0: input open -> closed
bobcat: channel 0: close_read
bobcat: channel 0: obuf empty
bobcat: channel 0: output drain -> closed
bobcat: channel 0: close_write
bobcat: channel 0: send close
bobcat: channel 0: full closed
bobcat: sftp: Sending SSH2_FXP_INIT
Use of uninitialized value in numeric gt (>) at
/opt/perl/lib/site_perl/5.8.2/Net/SSH/Perl/Channel.pm line 85
(#1)
(W uninitialized) An undefined value was used as if it were
already
defined. It was interpreted as a "" or a 0, but maybe it was a
mistake.
To suppress this warning assign a defined value to your variables.
To help you figure out what was undefined, perl tells you what
operation
you used the undefined value in. Note, however, that perl
optimizes your
program and the operation displayed in the warning may not
necessarily
appear literally in your program. For example, "that $foo" is
usually optimized into "that " . $foo, and the warning will refer
to
the concatenation (.) operator, even though there is no . in your
program.
Use of uninitialized value in pack at
/opt/perl/lib/site_perl/5.8.2/Net/SSH/Perl/Buffer.pm line 105
(#1)
Uncaught exception from user code:
Connection closed at ./sftpTRSdbg.pl line 54
Net::SFTP::get_msg('Net::SFTP=HASH(0x180074)') called at /opt/
perl/lib/site_perl/5.8.2/Net/SFTP.pm line 101
Net::SFTP::do_init('Net::SFTP=HASH(0x180074)') called at /opt/
perl/lib/site_perl/5.8.2/Net/SFTP.pm line 43
Net::SFTP::init('Net::SFTP=HASH
(0x180074)','password','password','debug','true','user','user-
id','ssh_args','ARRAY(0x14ba7c)',...) called at /opt/perl/lib/
site_perl/5.8.2/Net/SFTP.pm line 24
Net::SFTP::new
('Net::SFTP','xxx.xxx.gov.uk','password','password','debug','true','user','user-
id','ssh_args',...) called at ./sftpTRSdbg.pl line 54
[55]/home/jones/:
------------------------------
Date: Tue, 17 Nov 2009 14:19:52 -0600
From: "J. Gleixner" <glex_no-spam@qwest-spam-no.invalid>
Subject: Re: Failed connection to sftp server via Net::SFTP
Message-Id: <4b030568$0$89869$815e3792@news.qwest.net>
Kevin13 wrote:
> Hi. Following perl code fails to connect to SFTP server (user-id/
> password authentication only supported on the server). Has
> consistently worked for last year, so likely a change on far end.
> Looking for some clue to pass along to techs that support the sftp
> server. Diagnostic messages follow code.
>
> Any thoughts or insights would be much appreciated!
Since you say your program hasn't changed, then why do you
think there's a problem there?
Start by testing if you can get to the server and do everything
via the command line.
------------------------------
Date: Tue, 17 Nov 2009 13:30:53 -0800 (PST)
From: Kevin13 <se5u0o702@sneakemail.com>
Subject: Re: Failed connection to sftp server via Net::SFTP
Message-Id: <e0ef4b97-f321-44fb-a833-06857178f784@m33g2000pri.googlegroups.com>
On Nov 17, 12:19=A0pm, "J. Gleixner" <glex_no-s...@qwest-spam-
no.invalid> wrote:
> Kevin13 wrote:
> > Hi. =A0Following perl code fails to connect to SFTP server (user-id/
> > password authentication only supported on the server). =A0Has
> > consistently worked for last year, so likely a change on far end.
> > Looking for some clue to pass along to techs that support the sftp
> > server. =A0Diagnostic messages follow code.
>
> > Any thoughts or insights would be much appreciated!
>
> Since you say your program hasn't changed, then why do you
> think there's a problem there?
>
> Start by testing if you can get to the server and do everything
> via the command line.
"Looking for some clue to pass along to techs that support the sftp
server."
Yes, I am able to run sftp from the command line and successfully
connect to the server, transfer file, etc. Sorry, thought I mentioned
that. Again, I do *not* think there is a problem with my program.
But, regardless I am being looked to to solve the issue and
unfortunately that likely means I have to "convince" the support staff
at the other end that there really is an issue.
------------------------------
Date: Tue, 17 Nov 2009 06:53:26 -0600
From: Tad McClellan <tadmc@seesig.invalid>
Subject: Re: How to upper case letters in a pair of parenthesis by PERL?
Message-Id: <slrnhg570p.8hs.tadmc@tadbox.sbcglobal.net>
sofin <ken.sofin@gmail.com> wrote:
> Input:
> "Modeltest: testing the model of (dna)-substitution"
>
> Output:
> "Modeltest: testing the model of DNA-substitution"
>
> Anyone knows how to do it by PERL?
Sure. That is pretty easy to do in Perl.
Show us the code you have so far, and we will help you fix it.
--
Tad McClellan
email: perl -le "print scalar reverse qq/moc.noitatibaher\100cmdat/"
------------------------------
Date: Tue, 17 Nov 2009 14:00:14 +0100
From: "Jochen Lehmeier" <OJZGSRPBZVCX@spammotel.com>
Subject: Re: How to upper case letters in a pair of parenthesis by PERL?
Message-Id: <op.u3jieovumk9oye@frodo>
On Tue, 17 Nov 2009 13:53:26 +0100, Tad McClellan <tadmc@seesig.invalid>
wrote:
> sofin <ken.sofin@gmail.com> wrote:
>> Input:
>> "Modeltest: testing the model of (dna)-substitution"
>>
>> Output:
>> "Modeltest: testing the model of DNA-substitution"
>>
>> Anyone knows how to do it by PERL?
> Sure. That is pretty easy to do in Perl.
s/\((\w+)\)/uc($1)/eg
------------------------------
Date: Tue, 17 Nov 2009 13:31:31 +0000 (UTC)
From: Willem <willem@stack.nl>
Subject: Re: How to upper case letters in a pair of parenthesis by PERL?
Message-Id: <slrnhg59dj.1vm6.willem@turtle.stack.nl>
Jochen Lehmeier wrote:
) On Tue, 17 Nov 2009 13:53:26 +0100, Tad McClellan <tadmc@seesig.invalid>
) wrote:
)
)> sofin <ken.sofin@gmail.com> wrote:
)>> Input:
)>> "Modeltest: testing the model of (dna)-substitution"
)>>
)>> Output:
)>> "Modeltest: testing the model of DNA-substitution"
)>>
)>> Anyone knows how to do it by PERL?
)
)> Sure. That is pretty easy to do in Perl.
)
) s/\((\w+)\)/uc($1)/eg
I think \U is preferrable to /e.
SaSW, Willem
--
Disclaimer: I am in no way responsible for any of the statements
made in the above text. For all I know I might be
drugged or something..
No I'm not paranoid. You all think I'm paranoid, don't you !
#EOT
------------------------------
Date: Tue, 17 Nov 2009 07:32:52 -0800
From: merlyn@stonehenge.com (Randal L. Schwartz)
Subject: Re: How to upper case letters in a pair of parenthesis by PERL?
Message-Id: <86zl6ljhej.fsf@blue.stonehenge.com>
>>>>> "sofin" == sofin <ken.sofin@gmail.com> writes:
sofin> Anyone knows how to do it by PERL? Thanks.
I'm pretty sure the Protecting Escarpment Rural Lands organization
(perlofburlington.org) isn't really commissioned to help you with your
homework. And I'm not even sure that your Pupils are Equal, Reactive to Light
at this point either.
When you have a non-homework question about the Perl programming language,
feel free to post again. :)
print "Just another Perl hacker,"; # the original
--
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<merlyn@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
Smalltalk/Perl/Unix consulting, Technical writing, Comedy, etc. etc.
See http://methodsandmessages.vox.com/ for Smalltalk and Seaside discussion
------------------------------
Date: Tue, 17 Nov 2009 14:31:48 -0800
From: sln@netherlands.com
Subject: Re: How to upper case letters in a pair of parenthesis by PERL?
Message-Id: <ks86g5tnlb7o34jb643nek82vpmkikeho4@4ax.com>
On Tue, 17 Nov 2009 07:32:52 -0800, merlyn@stonehenge.com (Randal L. Schwartz) wrote:
>>>>>> "sofin" == sofin <ken.sofin@gmail.com> writes:
>
>sofin> Anyone knows how to do it by PERL? Thanks.
>
>I'm pretty sure the Protecting Escarpment Rural Lands organization
>(perlofburlington.org) isn't really commissioned to help you with your
>homework. And I'm not even sure that your Pupils are Equal, Reactive to Light
>at this point either.
>
>When you have a non-homework question about the Perl programming language,
>feel free to post again. :)
>
>print "Just another Perl hacker,"; # the original
You mean Perl is actually offered as a HS or college course and with homework?
I find that hard to believe. It serves no purpose and has no reward.
-sln
------------------------------
Date: Tue, 17 Nov 2009 08:05:04 -0500
From: Shmuel (Seymour J.) Metz <spamtrap@library.lspace.org.invalid>
Subject: Re: mingw Perl: no long doubles?
Message-Id: <4b029f80$10$fuzhry+tra$mr2ice@news.patriot.net>
In <slrnhg3f8b.feh.hjp-usenet2@hrunkner.hjp.at>, on 11/16/2009
at 09:58 PM, "Peter J. Holzer" <hjp-usenet2@hjp.at> said:
>Normally a double is 64 bits[1], so a true long double type (i.e., one
>which isn't identical to double) has to have more than 64 bits.
The predecessor to the S/390, the S/370, already had 128 bit floating
point. I vaguely recall that it came in on the tail end of the S/360 line,
4 decades ago.
--
Shmuel (Seymour J.) Metz, SysProg and JOAT <http://patriot.net/~shmuel>
Unsolicited bulk E-mail subject to legal action. I reserve the
right to publicly post or ridicule any abusive E-mail. Reply to
domain Patriot dot net user shmuel+news to contact me. Do not
reply to spamtrap@library.lspace.org
------------------------------
Date: Tue, 17 Nov 2009 07:38:00 -0600
From: Ignoramus28865 <ignoramus28865@NOSPAM.28865.invalid>
Subject: Re: My perl script to run arbitrary tasks in parallel
Message-Id: <fJidnYHvTvClOp_WnZ2dnUVZ_qJi4p2d@giganews.com>
Updated version, fixes a bug
#!/usr/bin/perl
#
# This script reads its standard input, or from files given on
# command line (<>).
#
# It executes every line as a separate shell command. If --parallel
# argument is given, it executes as many jobs in parallel as
# possible, but no more tham "--parallel" at any given time.
#
# This can be helpful to speed up some tasks.
#
# You need module Parallel::ForkManager. It is available as
# an ubuntu package.
#
# Copyright(C) Igor Chudov, 2009. All rights reserved.
# This script is made available to the public under the latest
# GPL Version found at http://www.gnu.org/licenses/gpl.html
#
# No warranty is given or implied. Refunds will not be provided.
#
# Igor Chudov, http://igor.chudov.com/
#
use strict;
use warnings;
use Getopt::Long;
use Parallel::ForkManager;
my $parallel = 0;
GetOptions(
"parallel=i" => \$parallel,
);
my $pm = new Parallel::ForkManager( $parallel );
while( <> ) {
chomp;
my $pid = $pm->start and next;
system( $_ );
$pm->finish;
}
$pm->wait_all_children;
------------------------------
Date: Tue, 17 Nov 2009 11:41:18 -0500
From: despen@verizon.net
Subject: Re: My perl script to run arbitrary tasks in parallel
Message-Id: <icskcdw1ch.fsf@verizon.net>
Ignoramus30118 <ignoramus30118@NOSPAM.30118.invalid> writes:
> #!/usr/bin/perl
>
> #
> # This script reads its standard input, or from files given on
> # command line (<>).
> #
> # It executes every line as a separate shell command. If --parallel
> # argument is given, it executes as many jobs in parallel, as
> # possible, but no more tham "--parallel" at any given time.
> #
> # This can be helpful to speed up some tasks.
Make with the -j option.
-j = as many as possible
-j nnn up to nnn at a time.
In addition job interdedpence can be handled.
Ie. don't start these 3 jobs until these other 2 run to completion
successfully.
------------------------------
Date: Tue, 17 Nov 2009 11:26:15 -0600
From: Ignoramus28865 <ignoramus28865@NOSPAM.28865.invalid>
Subject: Re: My perl script to run arbitrary tasks in parallel
Message-Id: <nN-dncL3FcsqQZ_WnZ2dnUVZ_oFi4p2d@giganews.com>
On 2009-11-17, despen@verizon.net <despen@verizon.net> wrote:
> Ignoramus30118 <ignoramus30118@NOSPAM.30118.invalid> writes:
>
>> #!/usr/bin/perl
>>
>> #
>> # This script reads its standard input, or from files given on
>> # command line (<>).
>> #
>> # It executes every line as a separate shell command. If --parallel
>> # argument is given, it executes as many jobs in parallel, as
>> # possible, but no more tham "--parallel" at any given time.
>> #
>> # This can be helpful to speed up some tasks.
>
> Make with the -j option.
>
> -j = as many as possible
> -j nnn up to nnn at a time.
>
> In addition job interdedpence can be handled.
>
> Ie. don't start these 3 jobs until these other 2 run to completion
> successfully.
>
This is similar to what my script does, except that it does not
require a makefile.
My immediate use of this would be to speed up image conversion with
imagemagick. I have a script that would download pictures from my
camera, and then "convert" them to make it 50%x50% of original size.
I want to speed up "convert" by parallelizing it.
i
------------------------------
Date: Tue, 17 Nov 2009 11:32:36 -0600
From: Ignoramus28865 <ignoramus28865@NOSPAM.28865.invalid>
Subject: Re: My perl script to run arbitrary tasks in parallel
Message-Id: <QpqdnfxxsKOpQ5_WnZ2dnUVZ_oqlnZ2d@giganews.com>
Another update. This one tries to calculate the default parallel level
based on the number of CPUs on the system.
#!/usr/bin/perl
#
# This script reads its standard input, or from files given on
# command line (<>).
#
# It executes every line as a separate shell command. If --parallel
# argument is given, it executes as many jobs in parallel as
# possible, but no more tham "--parallel" at any given time.
#
# This can be helpful to speed up some tasks.
#
# The default level of parallel is based on the number of CPUs.
#
# You need module Parallel::ForkManager. It is available as
# an ubuntu package.
#
# Copyright(C) Igor Chudov, 2009. All rights reserved.
# This script is made available to the public under the latest
# GPL Version found at http://www.gnu.org/licenses/gpl.html
#
# No warranty is given or implied. Refunds will not be provided.
#
# Igor Chudov, http://igor.chudov.com/
#
use strict;
use warnings;
use Getopt::Long;
use Parallel::ForkManager;
my $parallel = undef;
GetOptions(
"parallel=i" => \$parallel,
);
unless( defined $parallel ) {
if( open( CPU, "/proc/cpuinfo" ) ) {
$parallel = 0;
while( <CPU> ) {
$parallel++ if /^processor\b/;
}
close( CPU );
} else {
$parallel = 0;
}
}
my $pm = new Parallel::ForkManager( $parallel );
while( <> ) {
chomp;
my $pid = $pm->start and next;
system( $_ );
$pm->finish;
}
$pm->wait_all_children;
------------------------------
Date: Tue, 17 Nov 2009 14:44:51 -0500
From: despen@verizon.net
Subject: Re: My perl script to run arbitrary tasks in parallel
Message-Id: <icocn1vsuk.fsf@verizon.net>
Ignoramus28865 <ignoramus28865@NOSPAM.28865.invalid> writes:
> On 2009-11-17, despen@verizon.net <despen@verizon.net> wrote:
>> Ignoramus30118 <ignoramus30118@NOSPAM.30118.invalid> writes:
>>
>>> #!/usr/bin/perl
>>>
>>> #
>>> # This script reads its standard input, or from files given on
>>> # command line (<>).
>>> #
>>> # It executes every line as a separate shell command. If --parallel
>>> # argument is given, it executes as many jobs in parallel, as
>>> # possible, but no more tham "--parallel" at any given time.
>>> #
>>> # This can be helpful to speed up some tasks.
>>
>> Make with the -j option.
>>
>> -j = as many as possible
>> -j nnn up to nnn at a time.
>>
>> In addition job interdedpence can be handled.
>>
>> Ie. don't start these 3 jobs until these other 2 run to completion
>> successfully.
>>
>
> This is similar to what my script does, except that it does not
> require a makefile.
>
> My immediate use of this would be to speed up image conversion with
> imagemagick. I have a script that would download pictures from my
> camera, and then "convert" them to make it 50%x50% of original size.
>
> I want to speed up "convert" by parallelizing it.
That's exactly the kind of thing I would use make for.
In fact, here's part of a makefile to create thumbnails for
jpgs and gifs:
small_%.jpg: %.jpg
convert -geometry 320x240 $< $@
chmod a+r $@
small_%.gif: %.gif
convert -geometry 320x240 -interlace plane $< $@
chmod a+r $@
Thumbnails only get created when the image is updated.
The same Makefile, uploads everything that's changed to the web server.
------------------------------
Date: Tue, 17 Nov 2009 16:15:44 -0600
From: Ignoramus28865 <ignoramus28865@NOSPAM.28865.invalid>
Subject: Re: My perl script to run arbitrary tasks in parallel
Message-Id: <6tGdnaeDZu4NvZ7WnZ2dnUVZ_t1i4p2d@giganews.com>
On 2009-11-17, despen@verizon.net <despen@verizon.net> wrote:
> Ignoramus28865 <ignoramus28865@NOSPAM.28865.invalid> writes:
>
>> On 2009-11-17, despen@verizon.net <despen@verizon.net> wrote:
>>> Ignoramus30118 <ignoramus30118@NOSPAM.30118.invalid> writes:
>>>
>>>> #!/usr/bin/perl
>>>>
>>>> #
>>>> # This script reads its standard input, or from files given on
>>>> # command line (<>).
>>>> #
>>>> # It executes every line as a separate shell command. If --parallel
>>>> # argument is given, it executes as many jobs in parallel, as
>>>> # possible, but no more tham "--parallel" at any given time.
>>>> #
>>>> # This can be helpful to speed up some tasks.
>>>
>>> Make with the -j option.
>>>
>>> -j = as many as possible
>>> -j nnn up to nnn at a time.
>>>
>>> In addition job interdedpence can be handled.
>>>
>>> Ie. don't start these 3 jobs until these other 2 run to completion
>>> successfully.
>>>
>>
>> This is similar to what my script does, except that it does not
>> require a makefile.
>>
>> My immediate use of this would be to speed up image conversion with
>> imagemagick. I have a script that would download pictures from my
>> camera, and then "convert" them to make it 50%x50% of original size.
>>
>> I want to speed up "convert" by parallelizing it.
>
> That's exactly the kind of thing I would use make for.
> In fact, here's part of a makefile to create thumbnails for
> jpgs and gifs:
>
> small_%.jpg: %.jpg
> convert -geometry 320x240 $< $@
> chmod a+r $@
> small_%.gif: %.gif
> convert -geometry 320x240 -interlace plane $< $@
> chmod a+r $@
>
> Thumbnails only get created when the image is updated.
> The same Makefile, uploads everything that's changed to the web server.
I think that you have a great approach, but in my case, I do not want
to keep the originals. (which may be addressed by adding rm in the
makefile, I am not sure)
Also, would this makefile make small_small_small_mypic.jpg if you
invoke it a few times, starting with mypic.jpg? Am I missing anything?
i
------------------------------
Date: Tue, 17 Nov 2009 13:39:47 -0800 (PST)
From: Obama <cyrusgreats@gmail.com>
Subject: parse the logs and search for matching string
Message-Id: <ad96d04c-5fed-4e35-9786-63c0bb8efd5e@m7g2000prd.googlegroups.com>
Hello my good people out there,
I need to write script to parse a log file and does the following:
- search for name of server (in this case network-1:Test1)
- search for start time (in this case Fri May 25 00:13:20)
- search for end time (in this case May 25 00:13:49)
- search for amount (in this case 2048 KB)
Note that these lines will not typically be sequential. There may be
numerous between the start and end lines, and many servers. So I
assume I can place them matching name into array and then print the
result into excel-sheet!
log file: The below lines are examples.
src Fri May 25 00:13:20 EDT myserver1:Test1 network-1:Test1 Start
src Fri May 25 00:13:49 EDT myserver1:Test1 network-1:Test1 End (2048
KB)
...
..
src Sat May 26 00:15:20 EDT myserver2:Test2 network-2:Test2 Start
src Sat May 26 00:15:49 EDT myserver2:Test2 network-2:Test2 End
(212048 KB)
my code..
----------------------
open (FILE, $file) || die "Can't open datafile: $file";
my @lines = reverse <FILE>;
my $count = 1;
foreach my $line ( @lines ) {
next if $line =~ /^#/; # skip comments
next if $line =~ /^\s*$/; # skip empty lines
next unless $line =~ /$match/;
#here I need your advice
($key, $val) = $line =~ /([^src]*) ........../x or die "bad data:
invalid field '$_' in chunk $.";
if ($key) {
$worksheet->write($row, $column , $val);
$row++;
$count++;
last;
}
------------------------------
Date: Tue, 17 Nov 2009 13:57:33 -0800 (PST)
From: Obama <cyrusgreats@gmail.com>
Subject: Re: parse the logs and search for matching string
Message-Id: <05d659ed-d220-4c9b-bbc9-8c377abdbd4e@g22g2000prf.googlegroups.com>
On Nov 17, 1:39=A0pm, Obama <cyrusgre...@gmail.com> wrote:
> Hello my good people out there,
> I need to write script to =A0parse a log file and does the following:
>
> - search for name of server (in this case network-1:Test1)
> - search for start time (in this case Fri May 25 00:13:20)
> - search for end time (in this case May 25 00:13:49)
> - search for amount (in this case 2048 KB)
>
> Note that these lines will not typically be sequential. There may be
> numerous between the start and end lines, and many servers. So I
> assume I can place them matching name into array and then print the
> result into excel-sheet!
>
> log file: The below lines are examples.
> src Fri May 25 00:13:20 EDT myserver1:Test1 network-1:Test1 Start
> src Fri May 25 00:13:49 EDT myserver1:Test1 network-1:Test1 End (2048
> KB)
> ...
>
> ..
> src Sat May 26 00:15:20 EDT myserver2:Test2 network-2:Test2 Start
> src Sat May 26 00:15:49 EDT myserver2:Test2 network-2:Test2 End
> (212048 KB)
>
> my code..
> ----------------------
> open (FILE, $file) || die "Can't open datafile: $file";
> =A0my @lines =3D reverse <FILE>;
> =A0my $count =3D 1;
>
> =A0foreach my $line ( @lines ) {
> =A0 =A0next if $line =3D~ /^#/; =A0 =A0 # skip comments
> =A0 =A0next if $line =3D~ /^\s*$/; =A0# skip empty lines
> =A0 =A0next unless $line =3D~ /$match/;
>
> #here I need your advice
> =A0 =A0($key, $val) =3D $line =3D~ /([^src]*) ........../x or die "bad da=
ta:
> invalid field '$_' in chunk $.";
> =A0 =A0if ($key) {
> =A0 =A0 =A0$worksheet->write($row, $column , $val);
>
> =A0 =A0 =A0$row++;
> =A0 =A0 =A0$count++;
> =A0 =A0 =A0last;
> =A0 =A0 }
Correction: the line in log could start with src or dst
------------------------------
Date: Tue, 17 Nov 2009 14:22:38 -0800
From: sln@netherlands.com
Subject: Re: parse the logs and search for matching string
Message-Id: <uv76g51qv06u5g3ieef6il56hhdedaiide@4ax.com>
On Tue, 17 Nov 2009 13:57:33 -0800 (PST), Obama <cyrusgreats@gmail.com> wrote:
>On Nov 17, 1:39 pm, Obama <cyrusgre...@gmail.com> wrote:
>> Hello my good people out there,
>> I need to write script to parse a log file and does the following:
>>
>> - search for name of server (in this case network-1:Test1)
>> - search for start time (in this case Fri May 25 00:13:20)
>> - search for end time (in this case May 25 00:13:49)
>> - search for amount (in this case 2048 KB)
>>
>> Note that these lines will not typically be sequential. There may be
>> numerous between the start and end lines, and many servers. So I
>> assume I can place them matching name into array and then print the
>> result into excel-sheet!
>>
>> log file: The below lines are examples.
>> src Fri May 25 00:13:20 EDT myserver1:Test1 network-1:Test1 Start
>> src Fri May 25 00:13:49 EDT myserver1:Test1 network-1:Test1 End (2048
>> KB)
>> ...
>>
>> ..
>> src Sat May 26 00:15:20 EDT myserver2:Test2 network-2:Test2 Start
>> src Sat May 26 00:15:49 EDT myserver2:Test2 network-2:Test2 End
>> (212048 KB)
>>
>> my code..
>> ----------------------
>> open (FILE, $file) || die "Can't open datafile: $file";
>> my @lines = reverse <FILE>;
>> my $count = 1;
>>
>> foreach my $line ( @lines ) {
>> next if $line =~ /^#/; # skip comments
>> next if $line =~ /^\s*$/; # skip empty lines
>> next unless $line =~ /$match/;
>>
>> #here I need your advice
>> ($key, $val) = $line =~ /([^src]*) ........../x or die "bad data:
>> invalid field '$_' in chunk $.";
>> if ($key) {
>> $worksheet->write($row, $column , $val);
>>
>> $row++;
>> $count++;
>> last;
>> }
>
>Correction: the line in log could start with src or dst
I am going to try to help you but you leave out so many
really important details.
Are these adjacent lines?
Is there a Start and End?
Are the fields of fixed width?
Is the size KB after End on the same line?
Does it wrap because of your news client?
What happens when either Start/End is invalid?
Throw the pair away?
src Fri May 25 00:13:20 EDT myserver1:Test1 network-1:Test1 Start
src Fri May 25 00:13:49 EDT myserver1:Test1 network-1:Test1 End (2048 KB)
~vs:
src Sat May 26 00:15:20 EDT myserver2:Test2 network-2:Test2 Start
src Sat May 26 00:15:49 EDT myserver2:Test2 network-2:Test2 End
(212048 KB)
These are some serious questions.
Do you have acess to the code that generates these logs?
If so, what is the intended data format?
Are there natural delimeters, and is it formatted to parse
at a later time?
If this is just free text, you are going to have problems.
Its not impossible, just unreliable. I can make/force hard
assertions out of thin air, infact I will have to.
How suseptable to change is it?
-sln
------------------------------
Date: Tue, 17 Nov 2009 14:46:42 -0800
From: sln@netherlands.com
Subject: Re: parse the logs and search for matching string
Message-Id: <sl96g5hjijukdopf7r3h0vh3f4qsaas0h5@4ax.com>
On Tue, 17 Nov 2009 14:22:38 -0800, sln@netherlands.com wrote:
>On Tue, 17 Nov 2009 13:57:33 -0800 (PST), Obama <cyrusgreats@gmail.com> wrote:
>
>>On Nov 17, 1:39 pm, Obama <cyrusgre...@gmail.com> wrote:
>>> Hello my good people out there,
>>> I need to write script to parse a log file and does the following:
>>>
>>> - search for name of server (in this case network-1:Test1)
>>> - search for start time (in this case Fri May 25 00:13:20)
>>> - search for end time (in this case May 25 00:13:49)
>>> - search for amount (in this case 2048 KB)
>>>
>>> Note that these lines will not typically be sequential. There may be
>>> numerous between the start and end lines, and many servers. So I
>>> assume I can place them matching name into array and then print the
>>> result into excel-sheet!
^^^^
I didn't read this part too well. You say it but it doesen't make sense.
The best you can do is to upload a huge sample because you make it sound
random and nobody can translate ambigous data through the mind of a novice.
>>> my code..
>>> ----------------------
>>> #here I need your advice
>>> ($key, $val) = $line =~ /([^src]*) ........../x or die "bad data:
^^^^^^^
What do you think this means?
>>
>>Correction: the line in log could start with src or dst
Why am I not suprised.
Canceled.
-sln
------------------------------
Date: 6 Apr 2001 21:33:47 GMT (Last modified)
From: Perl-Users-Request@ruby.oce.orst.edu (Perl-Users-Digest Admin)
Subject: Digest Administrivia (Last modified: 6 Apr 01)
Message-Id: <null>
Administrivia:
To submit articles to comp.lang.perl.announce, send your article to
clpa@perl.com.
Back issues are available via anonymous ftp from
ftp://cil-www.oce.orst.edu/pub/perl/old-digests.
#For other requests pertaining to the digest, send mail to
#perl-users-request@ruby.oce.orst.edu. Do not waste your time or mine
#sending perl questions to the -request address, I don't have time to
#answer them even if I did know the answer.
------------------------------
End of Perl-Users Digest V11 Issue 2680
***************************************
