[32116] in Perl-Users-Digest
Perl-Users Digest, Issue: 3381 Volume: 11
daemon@ATHENA.MIT.EDU (Perl-Users Digest)
Wed May 11 14:14:24 2011
Date: Wed, 11 May 2011 11:14:11 -0700 (PDT)
From: Perl-Users Digest <Perl-Users-Request@ruby.OCE.ORST.EDU>
To: Perl-Users@ruby.OCE.ORST.EDU (Perl-Users Digest)
Perl-Users Digest Wed, 11 May 2011 Volume: 11 Number: 3381
Today's topics:
Re: Moving data from one machine to another <john@castleamber.com>
Re: Moving data from one machine to another <justin.1104@purestblue.com>
Re: Moving data from one machine to another <RedGrittyBrick@spamweary.invalid>
Re: Moving data from one machine to another <john@castleamber.com>
Re: Moving data from one machine to another <justin.1104@purestblue.com>
Re: Moving data from one machine to another <justin.1104@purestblue.com>
Re: Moving data from one machine to another <rweikusat@mssgmbh.com>
Re: Moving data from one machine to another <justin.1104@purestblue.com>
Re: Moving data from one machine to another <rweikusat@mssgmbh.com>
Re: Moving data from one machine to another <justin.1104@purestblue.com>
Re: Moving data from one machine to another <john@castleamber.com>
Re: Sandboxing re sln@netherlands.com
Digest Administrivia (Last modified: 6 Apr 01) (Perl-Users-Digest Admin)
----------------------------------------------------------------------
Date: Tue, 10 May 2011 08:11:44 -0500
From: John Bokma <john@castleamber.com>
Subject: Re: Moving data from one machine to another
Message-Id: <87ipti8zu7.fsf@castleamber.com>
Justin C <justin.1104@purestblue.com> writes:
> On 2011-05-05, John Bokma <john@castleamber.com> wrote:
[..]
>> It's a text file... I would say:
>> gzip the data and email that, and then do the post-processing on the
>> other machine.
>
> If I send over just the text I can't do
> use Storable;
> my $hashref = retrieve('somefile');
>
> Which means I've got to take much more care over getting the data into
> the program on the receiving machine.
OK, clear. In that case I strongly recommend JSON or YAML, see Peter's
reply. I used Storable in the past until I ran into the fun of
version incompatibility :-(.
>> If you want to "massage" the data on-site, why not store
>> it using JSON or YAML? It sounds like most of it is text data. You can
>> always compress the JSON/YAML output using gzip.
>
> Aren't those two just another way to do the same thing, but requiring
> more code/effort on my part?
>
> I don't see a way that using either would be quicker/easier than:
> use Storable;
> store \%parts_info, 'parts_info';
use YAML;
DumpFile( 'parts_info', \%parts_info );
;-)
See the YAML documentation.
--
John Bokma j3b
Blog: http://johnbokma.com/ Facebook: http://www.facebook.com/j.j.j.bokma
Freelance Perl & Python Development: http://castleamber.com/
------------------------------
Date: Tue, 10 May 2011 16:18:09 +0100
From: Justin C <justin.1104@purestblue.com>
Subject: Re: Moving data from one machine to another
Message-Id: <hi1o98-h9v.ln1@zem.masonsmusic.co.uk>
On 2011-05-10, John Bokma <john@castleamber.com> wrote:
> Justin C <justin.1104@purestblue.com> writes:
>
>> On 2011-05-05, John Bokma <john@castleamber.com> wrote:
>
> [..]
>
>>> It's a text file... I would say:
>>> gzip the data and email that, and then do the post-processing on the
>>> other machine.
>>
>> If I send over just the text I can't do
>> use Storable;
>> my $hashref = retrieve('somefile');
>>
>> Which means I've got to take much more care over getting the data into
>> the program on the receiving machine.
>
> OK, clear. In that case I strongly recommend JSON or YAML, see Peter's
> reply. I used Storable in the past until I ran into the fun of
> version incompatibility :-(.
>
>>> If you want to "massage" the data on-site, why not store
>>> it using JSON or YAML? It sounds like most of it is text data. You can
>>> always compress the JSON/YAML output using gzip.
>>
>> Aren't those two just another way to do the same thing, but requiring
>> more code/effort on my part?
>>
>> I don't see a way that using either would be quicker/easier than:
>> use Storable;
>> store \%parts_info, 'parts_info';
>
> use YAML;
> DumpFile( 'parts_info', \%parts_info );
I'm banging my head against the wall WRT gziping the file, using
MIME::Lite to attach it to an email, and then using MIME::Parser to
extract the attachment the other end.
I'll read the YAML docs, I probably should anyway.
And I didn't know about the problems with Storable, that's a PITA, I've
only just started using it following advice in the FAQ (How do I keep
persistent data across program calls?)... though, admittedly, I do tend
to use it for it's purpose, and not to pass data between machines
(though that is handy).
Justin.
--
Justin C, by the sea.
------------------------------
Date: Tue, 10 May 2011 17:18:49 +0100
From: RedGrittyBrick <RedGrittyBrick@spamweary.invalid>
Subject: Re: Moving data from one machine to another
Message-Id: <4dc96569$0$12166$fa0fcedb@news.zen.co.uk>
On 10/05/2011 16:18, Justin C wrote:
>
> I'm banging my head against the wall WRT gziping the file, using
> MIME::Lite to attach it to an email, and then using MIME::Parser to
> extract the attachment the other end.
>
Stop banging your head against a wall.
I'd only use email to transfer data if connectivity between the two
systems is intermittent.
Use something else - CGI, SOAP, Sockets, FTP.
--
RGB
------------------------------
Date: Tue, 10 May 2011 12:01:06 -0500
From: John Bokma <john@castleamber.com>
Subject: Re: Moving data from one machine to another
Message-Id: <87wrhy4hil.fsf@castleamber.com>
Justin C <justin.1104@purestblue.com> writes:
> On 2011-05-10, John Bokma <john@castleamber.com> wrote:
>> Justin C <justin.1104@purestblue.com> writes:
>>
>>> On 2011-05-05, John Bokma <john@castleamber.com> wrote:
>>
>> [..]
>>
>>>> It's a text file... I would say:
>>>> gzip the data and email that, and then do the post-processing on the
>>>> other machine.
>>>
>>> If I send over just the text I can't do
>>> use Storable;
>>> my $hashref = retrieve('somefile');
>>>
>>> Which means I've got to take much more care over getting the data into
>>> the program on the receiving machine.
>>
>> OK, clear. In that case I strongly recommend JSON or YAML, see Peter's
>> reply. I used Storable in the past until I ran into the fun of
>> version incompatibility :-(.
>>
>>>> If you want to "massage" the data on-site, why not store
>>>> it using JSON or YAML? It sounds like most of it is text data. You can
>>>> always compress the JSON/YAML output using gzip.
>>>
>>> Aren't those two just another way to do the same thing, but requiring
>>> more code/effort on my part?
>>>
>>> I don't see a way that using either would be quicker/easier than:
>>> use Storable;
>>> store \%parts_info, 'parts_info';
>>
>> use YAML;
>> DumpFile( 'parts_info', \%parts_info );
>
>
> I'm banging my head against the wall WRT gziping the file, using
If it's not too much data (less than a few MB) I wouldn't worry too much
about it at this stage.
Otherwise: use YAML to dump the data to a string and use Compress::Gzip.
Something like:
use IO::Compress::Gzip qw(gzip $GzipError);
gzip $yaml_buffer, $output,
or die "gzip failed: $GzipError\n";
$output can be a buffer, which you can email as an attachment using
MIME::Lite
> MIME::Lite to attach it to an email, and then using MIME::Parser to
> extract the attachment the other end.
>
> I'll read the YAML docs, I probably should anyway.
Yup, that's for sure: Always read the documentation, a lot comes with
handy examples.
> And I didn't know about the problems with Storable, that's a PITA, I've
> only just started using it following advice in the FAQ (How do I keep
> persistent data across program calls?)... though, admittedly, I do tend
> to use it for it's purpose, and not to pass data between machines
> (though that is handy).
YAML and more so JSON is portable between other programming
languages. YAML is (somewhat?) a superset of JSON. And if you have a lot
of textual data, quite readable.
--
John Bokma j3b
Blog: http://johnbokma.com/ Facebook: http://www.facebook.com/j.j.j.bokma
Freelance Perl & Python Development: http://castleamber.com/
------------------------------
Date: Wed, 11 May 2011 11:13:57 +0100
From: Justin C <justin.1104@purestblue.com>
Subject: Re: Moving data from one machine to another
Message-Id: <544q98-87r.ln1@zem.masonsmusic.co.uk>
On 2011-05-10, RedGrittyBrick <RedGrittyBrick@spamweary.invalid> wrote:
> On 10/05/2011 16:18, Justin C wrote:
>>
>> I'm banging my head against the wall WRT gziping the file, using
>> MIME::Lite to attach it to an email, and then using MIME::Parser to
>> extract the attachment the other end.
>>
>
> Stop banging your head against a wall.
>
> I'd only use email to transfer data if connectivity between the two
> systems is intermittent.
>
> Use something else - CGI, SOAP, Sockets, FTP.
Actually, the main reason for email is the instant action on the
received data that procmail can initiate. Otherwise I must have a
process watching for the arrival of a file, spending most of it's time
asleep, but waking once a minute/five minutes/hour/whatever depending on
how urgently we want the data acted upon. I dislike leaving programs
just running because you then find you need another program to check
once in a while that they haven't failed for some reason.
The procmail solution would be good if I could make it work... I did get
close, and I am close, a little more reading of docs is required and a
little more intelligence (I'll have that when I return from vacation),
luckily this isn't urgent enough that it must be sorted before I go
away.
Justin.
--
Justin C, by the sea.
------------------------------
Date: Wed, 11 May 2011 12:04:02 +0100
From: Justin C <justin.1104@purestblue.com>
Subject: Re: Moving data from one machine to another
Message-Id: <227q98-fus.ln1@zem.masonsmusic.co.uk>
On 2011-05-10, John Bokma <john@castleamber.com> wrote:
> Justin C <justin.1104@purestblue.com> writes:
>
>> On 2011-05-10, John Bokma <john@castleamber.com> wrote:
>>> Justin C <justin.1104@purestblue.com> writes:
>>>
>>>> On 2011-05-05, John Bokma <john@castleamber.com> wrote:
>>>
>>> [..]
>>>
>>>>> It's a text file... I would say:
>>>>> gzip the data and email that, and then do the post-processing on the
>>>>> other machine.
>>>>
>>>> If I send over just the text I can't do
>>>> use Storable;
>>>> my $hashref = retrieve('somefile');
>>>>
>>>> Which means I've got to take much more care over getting the data into
>>>> the program on the receiving machine.
>>>
>>> OK, clear. In that case I strongly recommend JSON or YAML, see Peter's
>>> reply. I used Storable in the past until I ran into the fun of
>>> version incompatibility :-(.
>>>
>>>>> If you want to "massage" the data on-site, why not store
>>>>> it using JSON or YAML? It sounds like most of it is text data. You can
>>>>> always compress the JSON/YAML output using gzip.
>>>>
>>>> Aren't those two just another way to do the same thing, but requiring
>>>> more code/effort on my part?
>>>>
>>>> I don't see a way that using either would be quicker/easier than:
>>>> use Storable;
>>>> store \%parts_info, 'parts_info';
>>>
>>> use YAML;
>>> DumpFile( 'parts_info', \%parts_info );
>>
>>
>> I'm banging my head against the wall WRT gziping the file, using
>
> If it's not too much data (less than a few MB) I wouldn't worry too much
> about it at this stage.
>
> Otherwise: use YAML to dump the data to a string and use Compress::Gzip.
>
> Something like:
>
> use IO::Compress::Gzip qw(gzip $GzipError);
>
> gzip $yaml_buffer, $output,
> or die "gzip failed: $GzipError\n";
>
> $output can be a buffer, which you can email as an attachment using
> MIME::Lite
>
>> MIME::Lite to attach it to an email, and then using MIME::Parser to
>> extract the attachment the other end.
>>
>> I'll read the YAML docs, I probably should anyway.
>
> Yup, that's for sure: Always read the documentation, a lot comes with
> handy examples.
I meant that I should read up about YAML as I don't (didn't) really know
what it is, of course I'll read the docs before I try and use it,
whaddya take me for a n00b?! ;-)
>> And I didn't know about the problems with Storable, that's a PITA, I've
>> only just started using it following advice in the FAQ (How do I keep
>> persistent data across program calls?)... though, admittedly, I do tend
>> to use it for it's purpose, and not to pass data between machines
>> (though that is handy).
>
> YAML and more so JSON is portable between other programming
> languages. YAML is (somewhat?) a superset of JSON. And if you have a lot
> of textual data, quite readable.
Am investigating, thanks for the suggestion.
Justin.
--
Justin C, by the sea.
------------------------------
Date: Wed, 11 May 2011 12:48:41 +0100
From: Rainer Weikusat <rweikusat@mssgmbh.com>
Subject: Re: Moving data from one machine to another
Message-Id: <874o51tq3q.fsf@sapphire.mobileactivedefense.com>
Justin C <justin.1104@purestblue.com> writes:
> On 2011-05-10, RedGrittyBrick <RedGrittyBrick@spamweary.invalid> wrote:
>> On 10/05/2011 16:18, Justin C wrote:
>>>
>>> I'm banging my head against the wall WRT gziping the file, using
>>> MIME::Lite to attach it to an email, and then using MIME::Parser to
>>> extract the attachment the other end.
>>>
>>
>> Stop banging your head against a wall.
>>
>> I'd only use email to transfer data if connectivity between the two
>> systems is intermittent.
>>
>> Use something else - CGI, SOAP, Sockets, FTP.
>
> Actually, the main reason for email is the instant action on the
> received data that procmail can initiate. Otherwise I must have a
> process watching for the arrival of a file, spending most of it's time
> asleep, but waking once a minute/five minutes/hour/whatever depending on
> how urgently we want the data acted upon.
Not necessarily. Provided you have ssh-access, you could upload the
data with scp/sftp and start the processing on the target machine
afterwards by executing some command there via ssh.
------------------------------
Date: Wed, 11 May 2011 13:23:05 +0100
From: Justin C <justin.1104@purestblue.com>
Subject: Re: Moving data from one machine to another
Message-Id: <9mbq98-iru.ln1@zem.masonsmusic.co.uk>
On 2011-05-11, Rainer Weikusat <rweikusat@mssgmbh.com> wrote:
> Justin C <justin.1104@purestblue.com> writes:
>> On 2011-05-10, RedGrittyBrick <RedGrittyBrick@spamweary.invalid> wrote:
>>> On 10/05/2011 16:18, Justin C wrote:
>>>>
>>>> I'm banging my head against the wall WRT gziping the file, using
>>>> MIME::Lite to attach it to an email, and then using MIME::Parser to
>>>> extract the attachment the other end.
>>>>
>>>
>>> Stop banging your head against a wall.
>>>
>>> I'd only use email to transfer data if connectivity between the two
>>> systems is intermittent.
>>>
>>> Use something else - CGI, SOAP, Sockets, FTP.
>>
>> Actually, the main reason for email is the instant action on the
>> received data that procmail can initiate. Otherwise I must have a
>> process watching for the arrival of a file, spending most of it's time
>> asleep, but waking once a minute/five minutes/hour/whatever depending on
>> how urgently we want the data acted upon.
>
> Not necessarily. Provided you have ssh-access, you could upload the
> data with scp/sftp and start the processing on the target machine
> afterwards by executing some command there via ssh.
Different users on each machine. I'm not sure it's possible to do
authentication with pre-shared key when the users are different.
Justin.
--
Justin C, by the sea.
------------------------------
Date: Wed, 11 May 2011 15:52:18 +0100
From: Rainer Weikusat <rweikusat@mssgmbh.com>
Subject: Re: Moving data from one machine to another
Message-Id: <87vcxhs319.fsf@sapphire.mobileactivedefense.com>
Justin C <justin.1104@purestblue.com> writes:
[...]
>>> Actually, the main reason for email is the instant action on the
>>> received data that procmail can initiate. Otherwise I must have a
>>> process watching for the arrival of a file,
[...]
>> you could upload the data with scp/sftp and start the processing on
>> the target machine afterwards by executing some command there via
>> ssh.
>
> Different users on each machine. I'm not sure it's possible to do
> authentication with pre-shared key when the users are different.
It is.
------------------------------
Date: Wed, 11 May 2011 16:12:18 +0100
From: Justin C <justin.1104@purestblue.com>
Subject: Re: Moving data from one machine to another
Message-Id: <ijlq98-7f4.ln1@zem.masonsmusic.co.uk>
On 2011-05-11, Rainer Weikusat <rweikusat@mssgmbh.com> wrote:
> Justin C <justin.1104@purestblue.com> writes:
>
> [...]
>
>>>> Actually, the main reason for email is the instant action on the
>>>> received data that procmail can initiate. Otherwise I must have a
>>>> process watching for the arrival of a file,
>
> [...]
>
>>> you could upload the data with scp/sftp and start the processing on
>>> the target machine afterwards by executing some command there via
>>> ssh.
>>
>> Different users on each machine. I'm not sure it's possible to do
>> authentication with pre-shared key when the users are different.
>
> It is.
It is? OK, I'll investigate that, that's interesting to know, thank you.
Justin.
--
Justin C, by the sea.
------------------------------
Date: Wed, 11 May 2011 12:01:54 -0500
From: John Bokma <john@castleamber.com>
Subject: Re: Moving data from one machine to another
Message-Id: <87pqnp5fy5.fsf@castleamber.com>
Justin C <justin.1104@purestblue.com> writes:
> On 2011-05-10, John Bokma <john@castleamber.com> wrote:
>> Yup, that's for sure: Always read the documentation, a lot comes with
>> handy examples.
>
> I meant that I should read up about YAML as I don't (didn't) really know
> what it is, of course I'll read the docs before I try and use it,
> whaddya take me for a n00b?! ;-)
No, no, I didn't want to imply such a thing. I often write general since
I hope more people read it ;-).
Both YAML and JSON are worth a look at even if you decide not to use
it. I have been bitten once hard enough by Storable to never use it
again (incompatible between versions).
And I've replaced Data::Dumper with YAML::Syck for printing / debugging
data structures (which is maybe not the best YAML implementation out
there, see the documentation). At this time YAML::XS seems to be your
best bet.
--
John Bokma j3b
Blog: http://johnbokma.com/ Facebook: http://www.facebook.com/j.j.j.bokma
Freelance Perl & Python Development: http://castleamber.com/
------------------------------
Date: Wed, 11 May 2011 10:21:08 -0700
From: sln@netherlands.com
Subject: Re: Sandboxing re
Message-Id: <aahls6lbrck9oaiuhfq7c6g100nh0fqv7b@4ax.com>
On Mon, 09 May 2011 13:45:56 -0700, sln@netherlands.com wrote:
>On Wed, 04 May 2011 15:41:27 -0500, John Bokma <john@castleamber.com> wrote:
>
[...]
>will trap runtime syntax errors as well as code
>execution in either (?{}) or (?{{}})
^^^^^^
This should be (??{})
------------------------------
Date: 6 Apr 2001 21:33:47 GMT (Last modified)
From: Perl-Users-Request@ruby.oce.orst.edu (Perl-Users-Digest Admin)
Subject: Digest Administrivia (Last modified: 6 Apr 01)
Message-Id: <null>
Administrivia:
To submit articles to comp.lang.perl.announce, send your article to
clpa@perl.com.
Back issues are available via anonymous ftp from
ftp://cil-www.oce.orst.edu/pub/perl/old-digests.
#For other requests pertaining to the digest, send mail to
#perl-users-request@ruby.oce.orst.edu. Do not waste your time or mine
#sending perl questions to the -request address, I don't have time to
#answer them even if I did know the answer.
------------------------------
End of Perl-Users Digest V11 Issue 3381
***************************************
