[32566] in Perl-Users-Digest
Perl-Users Digest, Issue: 3832 Volume: 11
daemon@ATHENA.MIT.EDU (Perl-Users Digest)
Thu Dec 6 00:09:22 2012
Date: Wed, 5 Dec 2012 21:09:07 -0800 (PST)
From: Perl-Users Digest <Perl-Users-Request@ruby.OCE.ORST.EDU>
To: Perl-Users@ruby.OCE.ORST.EDU (Perl-Users Digest)
Perl-Users Digest Wed, 5 Dec 2012 Volume: 11 Number: 3832
Today's topics:
Re: a survey of templates <hjp-usenet2@hjp.at>
Re: a survey of templates <uri@stemsystems.com>
Re: a survey of templates <cal@example.invalid>
Re: a survey of templates <cal@example.invalid>
Re: a survey of templates <ben@morrow.me.uk>
Re: a survey of templates <cal@example.invalid>
Re: a survey of templates <uri@stemsystems.com>
Re: Backtick command with long output super slow <uri@stemsystems.com>
Re: Backtick command with long output super slow <rweikusat@mssgmbh.com>
Re: Backtick command with long output super slow <rweikusat@mssgmbh.com>
Re: Backtick command with long output super slow <willem@turtle.stack.nl>
Re: Backtick command with long output super slow <ben@morrow.me.uk>
How to print function with the code in the function bod <pengyu.ut@gmail.com>
Re: How to print function with the code in the function <rweikusat@mssgmbh.com>
Digest Administrivia (Last modified: 6 Apr 01) (Perl-Users-Digest Admin)
----------------------------------------------------------------------
Date: Wed, 5 Dec 2012 01:21:13 +0100
From: "Peter J. Holzer" <hjp-usenet2@hjp.at>
Subject: Re: a survey of templates
Message-Id: <slrnkbt4rp.upv.hjp-usenet2@hrunkner.hjp.at>
On 2012-12-04 11:27, Hans Mulder <hansmu@xs4all.nl> wrote:
> On 4/12/12 07:59:28, Cal Dershowitz wrote:
>> print $fh "<!DOCTYPE html>\n";
>> print $fh '<html lang="en">' . "\n";
>> print $fh "<head>\n";
>> print $fh '<meta charset="utf-8">' . "\n";
>> print $fh "<title>Operation Lasagna Chariot</title>\n";
>> print $fh "</head>\n";
>> print $fh "<body>\n";
>> print $fh "<h1>one stays; one goes.</h1>\n";
>> print $fh "<h2>take your pick!</h2>\n";
>
> It would be more perlish to use a "here document":
Here documents cannot be indented and are therefore too ugly to use in
about 99% of all possible use cases.
hp
--
_ | Peter J. Holzer | Fluch der elektronischen Textverarbeitung:
|_|_) | Sysadmin WSR | Man feilt solange an seinen Text um, bis
| | | hjp@hjp.at | die Satzbestandteile des Satzes nicht mehr
__/ | http://www.hjp.at/ | zusammenpaßt. -- Ralph Babel
------------------------------
Date: Wed, 05 Dec 2012 02:25:46 -0500
From: Uri Guttman <uri@stemsystems.com>
Subject: Re: a survey of templates
Message-Id: <87k3sxj6yt.fsf@stemsystems.com>
>>>>> "PJH" == Peter J Holzer <hjp-usenet2@hjp.at> writes:
PJH> On 2012-12-04 11:27, Hans Mulder <hansmu@xs4all.nl> wrote:
>> On 4/12/12 07:59:28, Cal Dershowitz wrote:
>>> print $fh "<!DOCTYPE html>\n";
>>> print $fh '<html lang="en">' . "\n";
>>> print $fh "<head>\n";
>>> print $fh '<meta charset="utf-8">' . "\n";
>>> print $fh "<title>Operation Lasagna Chariot</title>\n";
>>> print $fh "</head>\n";
>>> print $fh "<body>\n";
>>> print $fh "<h1>one stays; one goes.</h1>\n";
>>> print $fh "<h2>take your pick!</h2>\n";
>>
>> It would be more perlish to use a "here document":
PJH> Here documents cannot be indented and are therefore too ugly to use in
PJH> about 99% of all possible use cases.
not in my book. i like to see how the text will look on the left
margin. and in what way is that mess of prints, noisy quotes and such
cleaner than one here doc? and if you insist on indenting the text of
here docs there are several short tricks to removing the indent cleanly.
uri
------------------------------
Date: Wed, 05 Dec 2012 02:07:26 -0700
From: Cal Dershowitz <cal@example.invalid>
Subject: Re: a survey of templates
Message-Id: <FZOdnc3mAptTkyLNnZ2dnUVZ_vKdnZ2d@supernews.com>
On 12/04/2012 05:21 PM, Peter J. Holzer wrote:
> On 2012-12-04 11:27, Hans Mulder <hansmu@xs4all.nl> wrote:
>> On 4/12/12 07:59:28, Cal Dershowitz wrote:
>>> print $fh "<!DOCTYPE html>\n";
>>> print $fh '<html lang="en">' . "\n";
>>> print $fh "<head>\n";
>>> print $fh '<meta charset="utf-8">' . "\n";
>>> print $fh "<title>Operation Lasagna Chariot</title>\n";
>>> print $fh "</head>\n";
>>> print $fh "<body>\n";
>>> print $fh "<h1>one stays; one goes.</h1>\n";
>>> print $fh "<h2>take your pick!</h2>\n";
>>
>> It would be more perlish to use a "here document":
>
> Here documents cannot be indented and are therefore too ugly to use in
> about 99% of all possible use cases.
>
> hp
>
>
So I thought I would write the header and footer files directly to the
html doc I was creating. Also put the new images on a corresponding
directory in images/.
I would call this the equivalent of a 15-car pileup:
$ cat ftp_bash2.pl
#!/usr/bin/perl -w
use strict;
use 5.010;
use Net::FTP;
use HTML::Entities;
use File::Basename;
# diamond bracket in paragraph mode declared at top scope
$/ = "";
## usage needs 2 args from argv
#identity and config
my $ident = 'my_ftp.txt';
my ( $config, $domain );
$config = do($ident);
unless ($config) {
die("read error: $!") if $!;
die("parse error: $@") if $@;
}
$domain = $config->{ $ARGV[0] };
die("unknown domain: $ARGV[0]") unless $domain;
#preliminaries at top scope
my $word = "qlefw";
# get files
my $path = $ARGV[1];
my @files = <$path*>;
#dial up the server
my $ftp = Net::FTP->new( $domain->{domain}, Debug => 1, Passive => 1 )
or die "Can't connect: $@\n";
$ftp->login( $domain->{username}, $domain->{password} )
or die "Couldn't login\n";
$ftp->binary();
# get files from remote root that end in html:
my @remote_files = $ftp->ls();
# make unique .html choice
my @matching = map /${word}_(\d+)\.html/, @remote_files;
print "matching is @matching\n";
push( @matching, 0 );
@matching = sort { $a <=> $b } @matching;
my $winner = pop @matching;
my $newnum1 = $winner + 1;
my $word2 = "${word}_$newnum1";
print " new word is $word2\n";
my $html_file = "${word2}.html";
print "html file is $html_file\n";
#make directory in images to correspond
$ftp->cwd("images/")
or die "Cannot change working directory ", $ftp->message;
$ftp->mkdir("${word2}/")
or die "Cannot create directory ", $ftp->message;
$ftp->cdup();
# create file for html stubouts
open( my $fh, '>', $html_file )
or die("Can't open $html_file for writing: $!");
my $header = "header1.txt";
# write in the header
open( my $gh, '<', $header ) or die("Can't open: $!");
while (<$gh>) {
print $fh;
}
my $h1 = 'one stays; one goes.';
my $h2 = 'take your pick!';
print $fh "<h1>{$h1}</h1>\n";
print $fh "<h2>{$h2}</h2>\n";
#create template outside loop
my $template = <<'TEMPLATE';
<img src="/images/%s"/>
<p>%s</p>
TEMPLATE
open my $CAPTIONS, "<", "lasagna1.txt" or die "file not there\n";
$ftp->cwd("/images/${word2}/") or die "cwd failed $@\n";
# main control
for my $name (@files) {
print "name is $name\n";
my $remote_file = basename($name);
$ftp->put( $name, $remote_file ) or die "put failed $!\n";
# captions
my $caption = <$CAPTIONS>;
printf $fh $template, $domain->{domain} . '.' . "${word2}/" .
$remote_file,
$caption;
}
# write in the footer
my $footer = "footer1.txt";
open( my $hh, '<', $footer )
or die("Can't open: $!");
while (<$hh>) {
print $fh;
}
close $fh;
$ftp->cdup() or die "cdup failed $@\n";
$ftp->put($html_file) or die "put failed $@\n";
$
Output looks something like this.
<h1>{one stays; one goes.}</h1>
<h2>{take your pick!}</h2>
<img src="/images/www.merrillpjensen.com.qlefw_1/03-img_0034.jpg"/>
<p>I like this brand and eat either of these with a certain frequency.
</p>
<img src="/images/www.merrillpjensen.com.qlefw_1/03-img_0035.jpg"/>
<p>Dishes be gone!
...
and I just keep digging myself in farther and farther, so I have to call
a mercy kill and start asking some questions.
Q1)
$ cat header1.txt
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
$ cat footer1.txt
</head>
</body>
$ ls
002 013 ftp_bash1.pl lasagna_2.html qlefw_1.html
003 014 ftp_bash1.pl~ lasagna_3.html qlew_1.html
004 015 ftp_bash2.pl lasagna_4.html qltew_1.html
005 016 ftp_bash2.pl~ lasagna_5.html secret2.pl
006 017 ftp_bash2.pl.bak lasagna_6.html secret2.pl~
007 018 header1.txt lasagna_7.html secret2.tmpl
008 03 las_1.html lasagna_8.html secret2.tmpl~
009 badger1.pl lasa_1.html lasw_1.html secret4.pl
01 bash2.sh lasag_1.html ltew_1.html secret.pl
010 bash3.sh lasagna_1.html ltrw_1.html secret.pl~
011 bash3.sh~ lasagna1.txt ltsw_1.html secret.tmpl
012 footer1.txt lasagna1.txt~ my_ftp.txt secret.tmpl~
$
These files exist in the same directory as the script calling for them.
Why are they not showing up in the output?
Q2) When you have an html page that is running in root, and images that
are a directory on your server of the form images/some_word/ , how on
god's green earth do you code for that without making a huge mess like this?
Q3) I've talked about this before and gotten good feedback, but I
wouldn't really know the context anymore, so I'm gonna have to ask it
again. Since mkdir and checking for the existence of a new directory
sets up a race condition, how does one do that best?
Q4) Could my file reading problems have anything to do twith <> being
in paragraph mode? I guess I thought that would mean that it just did
it in fewer chunks.
Any tips appreciated.
--
Cal
------------------------------
Date: Wed, 05 Dec 2012 02:20:50 -0700
From: Cal Dershowitz <cal@example.invalid>
Subject: Re: a survey of templates
Message-Id: <obqdnaGWCZFvjCLNnZ2dnUVZ_rSdnZ2d@supernews.com>
On 12/05/2012 12:25 AM, Uri Guttman wrote:
>>>>>> "PJH" == Peter J Holzer <hjp-usenet2@hjp.at> writes:
>
> PJH> On 2012-12-04 11:27, Hans Mulder <hansmu@xs4all.nl> wrote:
> >> On 4/12/12 07:59:28, Cal Dershowitz wrote:
> >>> print $fh "<!DOCTYPE html>\n";
> >>> print $fh '<html lang="en">' . "\n";
> >>> print $fh "<head>\n";
> >>> print $fh '<meta charset="utf-8">' . "\n";
> >>> print $fh "<title>Operation Lasagna Chariot</title>\n";
> >>> print $fh "</head>\n";
> >>> print $fh "<body>\n";
> >>> print $fh "<h1>one stays; one goes.</h1>\n";
> >>> print $fh "<h2>take your pick!</h2>\n";
> >>
> >> It would be more perlish to use a "here document":
>
> PJH> Here documents cannot be indented and are therefore too ugly to use in
> PJH> about 99% of all possible use cases.
>
> not in my book. i like to see how the text will look on the left
> margin. and in what way is that mess of prints, noisy quotes and such
> cleaner than one here doc? and if you insist on indenting the text of
> here docs there are several short tricks to removing the indent cleanly.
>
> uri
>
yeah, but I think the direction we're going is that there's this
hierarchy of files on the local machine that get thrown together to form
an html page, and then the page and other supporting components, e.g.
images, are loaded up to a hierarchy on the server that perl can make
sense of.
Still fishing for what others have done with templating.
--
Cal
------------------------------
Date: Wed, 5 Dec 2012 11:58:42 +0000
From: Ben Morrow <ben@morrow.me.uk>
Subject: Re: a survey of templates
Message-Id: <igp3p9-fc4.ln1@anubis.morrow.me.uk>
Quoth Cal Dershowitz <cal@example.invalid>:
>
> Q1)
<snip: this sort of random dump of mostly irrelevant information really
doesn't help>
>
> These files exist in the same directory as the script calling for them.
> Why are they not showing up in the output?
Because this:
> while (<$gh>) {
> print $fh;
> }
does not do what you think it does. Reread perldoc -f print.
> Q2) When you have an html page that is running in root, and images
> that are a directory on your server of the form images/some_word/ ,
> how on god's green earth do you code for that without making a huge
> mess like this?
I don't understand this question. Which part of the mess are you worried
about?
In general the whole program could do with being broken up into subs.
Take logical chunks of code, things like 'read the config file', 'set up
the FTP connection', 'work out which local files we're dealing with' and
put each one in a sub. And *START GIVING YOUR VARIABLES BETTER NAMES*.
> Q3) I've talked about this before and gotten good feedback, but I
> wouldn't really know the context anymore, so I'm gonna have to ask it
> again. Since mkdir and checking for the existence of a new directory
> sets up a race condition, how does one do that best?
mkdir(2) fails with EEXIST if the directory already exists. FTP's MKD
command ought to fail with 521 if the directory exists, but given the
general flakiness of FTP I'd want to check that a particular server
actually got that right before relying on it. Distinguishing 'directory
already exists' from other forms of access denied error is likely to be
difficult with FTP.
> Q4) Could my file reading problems have anything to do twith <> being
> in paragraph mode? I guess I thought that would mean that it just did
> it in fewer chunks.
No. You don't have a file reading problem, you have a printing problem.
Ben
------------------------------
Date: Wed, 05 Dec 2012 14:33:28 -0700
From: Cal Dershowitz <cal@example.invalid>
Subject: Re: a survey of templates
Message-Id: <2fidnUdSwfU7ICLNnZ2dnUVZ_t-dnZ2d@supernews.com>
On 12/05/2012 04:58 AM, Ben Morrow wrote:
>
> Quoth Cal Dershowitz <cal@example.invalid>:
>>
>> Q1)
> <snip: this sort of random dump of mostly irrelevant information really
> doesn't help>
>>
>> These files exist in the same directory as the script calling for them.
>> Why are they not showing up in the output?
>
> Because this:
>
>> while (<$gh>) {
>> print $fh;
>> }
>
> does not do what you think it does. Reread perldoc -f print.
Ok.
>
>> Q2) When you have an html page that is running in root, and images
>> that are a directory on your server of the form images/some_word/ ,
>> how on god's green earth do you code for that without making a huge
>> mess like this?
>
> I don't understand this question. Which part of the mess are you worried
> about?
When there's so many things going wrong in a script, I start to lose my
moorings on file hierarchies and how to traverse and refer to them. I
think it's tricky business, and I'm hoping to sort it out in a
systematic way through the use of templating.
>
> In general the whole program could do with being broken up into subs.
> Take logical chunks of code, things like 'read the config file', 'set up
> the FTP connection', 'work out which local files we're dealing with' and
> put each one in a sub. And *START GIVING YOUR VARIABLES BETTER NAMES*.
Ben, I sit there as I'm defining new things on the fly, and I say to
myself "Ben is going to shake his head at that choice of this variable
name," but that kind of indicates the problem in that I really don't
know what I'm doing here and am making it up as I go.
I think I did alright on $h1, $h2, $footer, $header. I could go with
something more flamboyant for $word2, but it wouldn't have any more
meaning without dangerously acquiring underscores: $frobnitz, $lemons,
$particular_that_regex_can_cue_on.
$ cat ftp_bash2.pl
#!/usr/bin/perl -w
use strict;
use 5.010;
use Net::FTP;
use HTML::Entities;
use File::Basename;
# diamond bracket in paragraph mode declared at top scope
$/ = "";
## usage needs 2 args from argv
#identity and config
my $ident = 'my_ftp.txt';
my ( $config, $domain );
$config = do($ident);
unless ($config) {
die("read error: $!") if $!;
die("parse error: $@") if $@;
}
$domain = $config->{ $ARGV[0] };
die("unknown domain: $ARGV[0]") unless $domain;
#preliminaries at top scope
my $word = "ghjklm";
# get files
my $path = $ARGV[1];
my @files = <$path*>;
#dial up the server
my $ftp = Net::FTP->new( $domain->{domain}, Debug => 1, Passive => 1 )
or die "Can't connect: $@\n";
$ftp->login( $domain->{username}, $domain->{password} )
or die "Couldn't login\n";
$ftp->binary();
# get files from remote root that end in html:
my @remote_files = $ftp->ls();
# make unique .html choice
my @matching = map /${word}_(\d+)\.html/, @remote_files;
print "matching is @matching\n";
push( @matching, 0 );
@matching = sort { $a <=> $b } @matching;
my $winner = pop @matching;
my $newnum1 = $winner + 1;
my $word2 = "${word}_$newnum1";
print " new word is $word2\n";
my $html_file = "${word2}.html";
print "html file is $html_file\n";
#make directory in images to correspond
$ftp->cwd("images/")
or die "Cannot change working directory ", $ftp->message;
$ftp->mkdir("${word2}/")
or die "Cannot create directory ", $ftp->message;
$ftp->cdup();
# create file for html stubouts
open( my $fh, '>', $html_file )
or die("Can't open $html_file for writing: $!");
my $header = "header1.txt";
# write in the header
open( my $gh, '<', $header ) or die("Can't open: $!");
while (<$gh>) {
print $fh $_;
}
my $h1 = 'one stays; one goes.';
my $h2 = 'take your pick!';
print $fh "<h1>$h1</h1>\n";
print $fh "<h2>$h2</h2>\n";
#create template outside loop
my $template = <<'TEMPLATE';
<img src="/images/%s"/>
<p>%s</p>
TEMPLATE
open my $CAPTIONS, "<", "lasagna1.txt" or die "file not there\n";
$ftp->cwd("/images/${word2}/") or die "cwd failed $@\n";
# main control
for my $name (@files) {
print "name is $name\n";
my $remote_file = basename($name);
$ftp->put( $name, $remote_file ) or die "put failed $!\n";
# captions
my $caption = <$CAPTIONS>;
printf $fh $template, "${word2}/" . $remote_file, $caption;
}
# write in the footer
my $footer = "footer1.txt";
open( my $hh, '<', $footer )
or die("Can't open: $!");
while (<$hh>) {
print $fh $_;
}
close $fh;
$ftp->cdup() or die "cdup failed $@\n";
$ftp->cdup() or die "cdup failed $@\n";
$ftp->put($html_file) or die "put failed $@\n";
$
>
>> Q3) I've talked about this before and gotten good feedback, but I
>> wouldn't really know the context anymore, so I'm gonna have to ask it
>> again. Since mkdir and checking for the existence of a new directory
>> sets up a race condition, how does one do that best?
>
> mkdir(2) fails with EEXIST if the directory already exists. FTP's MKD
> command ought to fail with 521 if the directory exists, but given the
> general flakiness of FTP I'd want to check that a particular server
> actually got that right before relying on it. Distinguishing 'directory
> already exists' from other forms of access denied error is likely to be
> difficult with FTP.
Your reply gave me exact-enough search terms to find what we did before:
sub mk_new_dir {
my $counter2 = 1;
while (1) {
my $word = "site";
my $name = $word . '_' . $counter2++;
if ( mkdir $name, 0755 ) {
return $name; # success, return new dir name
}
else {
next if $!{EEXIST}; # mkdir failed because file exists
die sprintf "(%d) %s", $!, $!; # other failure; bail out!
}
}
}
I'll hook this up and drop the writing of the header and footer files to
subs as well. Is it advisable to drop the creation of the ftp
connection to a sub as well?
>
>> Q4) Could my file reading problems have anything to do twith <> being
>> in paragraph mode? I guess I thought that would mean that it just did
>> it in fewer chunks.
>
> No. You don't have a file reading problem, you have a printing problem.
Right. Thanks for your help.
--
Cal
------------------------------
Date: Wed, 05 Dec 2012 21:30:35 -0500
From: Uri Guttman <uri@stemsystems.com>
Subject: Re: a survey of templates
Message-Id: <87boe7kj3o.fsf@stemsystems.com>
>>>>> "CD" == Cal Dershowitz <cal@example.invalid> writes:
CD> On 12/05/2012 12:25 AM, Uri Guttman wrote:
PJH> Here documents cannot be indented and are therefore too ugly to use in
PJH> about 99% of all possible use cases.
>>
>> not in my book. i like to see how the text will look on the left
>> margin. and in what way is that mess of prints, noisy quotes and such
>> cleaner than one here doc? and if you insist on indenting the text of
>> here docs there are several short tricks to removing the indent cleanly.
CD> Still fishing for what others have done with templating.
then checkout Template::Simple. fastest templater around and it is very
easy to use. only 4 markups and it can do most templating tasks.
uri
------------------------------
Date: Wed, 05 Dec 2012 02:29:58 -0500
From: Uri Guttman <uri@stemsystems.com>
Subject: Re: Backtick command with long output super slow
Message-Id: <87fw3lj6rt.fsf@stemsystems.com>
>>>>> "BM" == Ben Morrow <ben@morrow.me.uk> writes:
BM> I don't know of any specific problem with backticks, but there was a
BM> problem at some point with perl's memory allocation strategy on Win32.
BM> When growing a long string perl ended up doing the growing in far more
BM> far smaller increments than was helpful; since the memory allocation
BM> calls on Win32 are rather slow this could cause a significant slowdown.
BM> I don't remember exactly when this was fixed; I thought it was before
BM> 5.12, but I could be wrong.
then wouldn't preallocating the buffer in the string help? not that i
will play with winblows but it is an easy fix.
OP: assign a long string first to the var and then call backticks:
my $buffer = ' ' x 65_000_000 ;
$buffer = `command` ;
see if that speeds it up. if it does, it should be faster than using a
temp file. also if you still want the temp file, use File::Slurp to read
it is as it should be faster than perl I/O.
uri
------------------------------
Date: Wed, 05 Dec 2012 08:37:24 +0000
From: Rainer Weikusat <rweikusat@mssgmbh.com>
Subject: Re: Backtick command with long output super slow
Message-Id: <87fw3k99ob.fsf@sapphire.mobileactivedefense.com>
Ben Morrow <ben@morrow.me.uk> writes:
> Quoth Willem <willem@turtle.stack.nl>:
>> jl_post@hotmail.com wrote:
[...]
>> ) my $text = `$command`;
>> )
>> ) Now, the command isn't trivial; it generates about 65 MB of output.
>> ) When I ran my script, my script was essentially hanging on that line
>> ) (I later realized that it wasn't hanging; it just took over 20 minutes
>> ) to run that line of code).
[...]
>> It sounds like a string-growing issue.
>> Have you tried reading it line by line?
>>
>> my @text = `$command`;
>
> That would almost certainly be worse: array-growing uses the same
> allocation strategy as string-growing, plus you've introduced the
> additional overhead of looking for newlines.
I don't have the time to test this now but assuming the issue is what
was hinted at, namely, O(n*n) realloc because of 'small', constant
increments combined with an allocator which hasn't been tuned to work
well despite some people's hell-bentness to do it the wrong way,
ie, one which does a copying realloc rather often, expecting the 'read
into array' code to perform better isn't completely unreasonable:
Unless the lines are very short, an array storing 65M lines of text is
going to be a lot smaller than 65M (roughly, a pointer per line plus
some constant 'management space', if the average line lenght is n,
that would be 65M/(8n) [64bit]) so, extending it by copying the old
contents to a new location will be faster and 'scanning for newlines'
amounts to 0.5 copies (read every byte and compare it with some value)
of the complete input data (actually splitting it into lines is
another complete copy).
------------------------------
Date: Wed, 05 Dec 2012 08:59:47 +0000
From: Rainer Weikusat <rweikusat@mssgmbh.com>
Subject: Re: Backtick command with long output super slow
Message-Id: <87mwxs3md8.fsf@sapphire.mobileactivedefense.com>
Rainer Weikusat <rweikusat@mssgmbh.com> writes:
[...]
> Unless the lines are very short, an array storing 65M lines of text is
> going to be a lot smaller than 65M (roughly, a pointer per line plus
> some constant 'management space', if the average line lenght is n,
> that would be 65M/(8n) [64bit])
This should have been (8*65M)/n.
------------------------------
Date: Wed, 5 Dec 2012 22:41:53 +0000 (UTC)
From: Willem <willem@turtle.stack.nl>
Subject: Re: Backtick command with long output super slow
Message-Id: <slrnkbvjdh.jen.willem@turtle.stack.nl>
Ben Morrow wrote:
)
) Quoth Willem <willem@turtle.stack.nl>:
)> It sounds like a string-growing issue.
)> Have you tried reading it line by line?
)>
)> my @text = `$command`;
)
) That would almost certainly be worse: array-growing uses the same
) allocation strategy as string-growing, plus you've introduced the
) additional overhead of looking for newlines.
But in the case of array-growing there won't be a gazillion copy operations
on the whole string buffer, but only on the array, which is a lot smaller.
SaSW, Willem
--
Disclaimer: I am in no way responsible for any of the statements
made in the above text. For all I know I might be
drugged or something..
No I'm not paranoid. You all think I'm paranoid, don't you !
#EOT
------------------------------
Date: Wed, 5 Dec 2012 23:18:41 +0000
From: Ben Morrow <ben@morrow.me.uk>
Subject: Re: Backtick command with long output super slow
Message-Id: <hb15p9-fv9.ln1@anubis.morrow.me.uk>
Quoth Willem <willem@turtle.stack.nl>:
> Ben Morrow wrote:
> )
> ) Quoth Willem <willem@turtle.stack.nl>:
> )> It sounds like a string-growing issue.
> )> Have you tried reading it line by line?
> )>
> )> my @text = `$command`;
> )
> ) That would almost certainly be worse: array-growing uses the same
> ) allocation strategy as string-growing, plus you've introduced the
> ) additional overhead of looking for newlines.
>
> But in the case of array-growing there won't be a gazillion copy operations
> on the whole string buffer, but only on the array, which is a lot smaller.
You're right; I wasn't thinking straight. Probably the best answer
(assuming you don't know the final size in advance) is reading
block-mode into an array, followed by a join if you want the string all
in one piece. (Join will preallocate the whole result string in one
step, so you'll have to copy the whole thing once only.)
Ben
------------------------------
Date: Wed, 5 Dec 2012 07:36:37 -0800 (PST)
From: Peng Yu <pengyu.ut@gmail.com>
Subject: How to print function with the code in the function body?
Message-Id: <14b90ebc-fea2-43f9-9325-0df167ec9f48@px4g2000pbc.googlegroups.com>
Hi,
I try to use Dumper to print a hash which has a function ref as one of
its value. But I only can get "sub { "DUMMY" }", which is not helpful
for me to know what this function is. Is there a way to somehow print
the function body?
~/linux/test/perl/library/Data/Dumper/Dumper$ cat main_fun.pl
#!/usr/bin/perl
use strict;
use warnings;
use Data::Dumper;
sub f {
return 0;
}
my %hash_of_arrays = (
a => \&f,
);
print Dumper(\%hash_of_arrays);
~/linux/test/perl/library/Data/Dumper/Dumper$ ./main_fun.pl
$VAR1 = {
'a' => sub { "DUMMY" }
};
Regards,
Peng
------------------------------
Date: Wed, 05 Dec 2012 16:10:15 +0000
From: Rainer Weikusat <rweikusat@mssgmbh.com>
Subject: Re: How to print function with the code in the function body?
Message-Id: <87vccgh448.fsf@sapphire.mobileactivedefense.com>
Peng Yu <pengyu.ut@gmail.com> writes:
> I try to use Dumper to print a hash which has a function ref as one of
> its value. But I only can get "sub { "DUMMY" }", which is not helpful
> for me to know what this function is. Is there a way to somehow print
> the function body?
In many cases, consulting the fine manual can be helpful:
o $Data::Dumper::Deparse or $OBJ->Deparse([NEWVAL])
Can be set to a boolean value to control whether code
references are turned into perl source code. If set to a
true value, "B::Deparse" will be used to get
the source of the code reference. Using this option will
force using the Perl implementation of the dumper, since
the fast XSUB implementation doesn't
support it.
Caution : use this option only if you know that your
coderefs will be properly reconstructed by "B::Deparse".
--------------
#!/usr/bin/perl
use strict;
use warnings;
use Data::Dumper;
sub f {
return 0;
}
my %hash_of_arrays = (
a => \&f,
);
$Data::Dumper::Deparse = 1;
print Dumper(\%hash_of_arrays);
------------------------------
Date: 6 Apr 2001 21:33:47 GMT (Last modified)
From: Perl-Users-Request@ruby.oce.orst.edu (Perl-Users-Digest Admin)
Subject: Digest Administrivia (Last modified: 6 Apr 01)
Message-Id: <null>
Administrivia:
To submit articles to comp.lang.perl.announce, send your article to
clpa@perl.com.
Back issues are available via anonymous ftp from
ftp://cil-www.oce.orst.edu/pub/perl/old-digests.
#For other requests pertaining to the digest, send mail to
#perl-users-request@ruby.oce.orst.edu. Do not waste your time or mine
#sending perl questions to the -request address, I don't have time to
#answer them even if I did know the answer.
------------------------------
End of Perl-Users Digest V11 Issue 3832
***************************************
