[32117] in Perl-Users-Digest
Perl-Users Digest, Issue: 3382 Volume: 11
daemon@ATHENA.MIT.EDU (Perl-Users Digest)
Fri May 13 03:09:33 2011
Date: Fri, 13 May 2011 00:09:09 -0700 (PDT)
From: Perl-Users Digest <Perl-Users-Request@ruby.OCE.ORST.EDU>
To: Perl-Users@ruby.OCE.ORST.EDU (Perl-Users Digest)
Perl-Users Digest Fri, 13 May 2011 Volume: 11 Number: 3382
Today's topics:
Re: alternatives for external data storage and manipula <cartercc@gmail.com>
Re: alternatives for external data storage and manipula <john@castleamber.com>
Re: Moving data from one machine to another <m@rtij.nl.invlalid>
Re: Moving data from one machine to another <rweikusat@mssgmbh.com>
Re: Moving data from one machine to another <RedGrittyBrick@spamweary.invalid>
Perl + Skype4com <blues.lead@gmail.com>
piping a vraiable content through a filter <hhr-m@web.de>
Re: piping a vraiable content through a filter <hhr-m@web.de>
Re: piping a vraiable content through a filter <rweikusat@mssgmbh.com>
Digest Administrivia (Last modified: 6 Apr 01) (Perl-Users-Digest Admin)
----------------------------------------------------------------------
Date: Wed, 11 May 2011 12:28:11 -0700 (PDT)
From: ccc31807 <cartercc@gmail.com>
Subject: Re: alternatives for external data storage and manipulation
Message-Id: <42e4f7be-e651-4bc0-b3d7-247d977339fa@bl1g2000vbb.googlegroups.com>
On May 11, 1:52=A0pm, John Bokma <j...@castleamber.com> wrote:
> You have to validate the data that you read, always. You have to check
> for entries missing and to check if entries have values that are legal.
There's a difference between validating the DATA and the correctness
of the program. I want OUT of the data validation business -- if the
user types the wrong username, or password, or filename, I don't want
to have to fix it, I want to give the user the tools to fix it
himself. I have config files that have worked for years, and I do mean
years, without me talking to the user or touching the file or the
code.
It's not my business to validate the data values, since I don't know
them or set them. It IS my business that to write code that will run
regardless of whether or not the user supplies valid or invalid data
values, with the understanding that 'running' and 'returning the
values the user expects' are not necessarily the same thing. A message
of 'Unrecognized User Name' isn't a bug in the code -- it's a bug in
the user, or as we say, the problem lies between the keyboard and the
chair. ;-)
> If you use YAML/JSON you have to still check the values you get
> (missing? within range) inside your program.
Well, yes. I still have to check whether values are missing whatever
the source. For example, users may not have a middle name or a cell
number, and I have to anticipate that these values may be null, but I
have no way of knowing if the middle name or cell number supplied by
the user is correct.
> OK, XML in this case is harder since it's more verbose and hence more
> prone to accidental errors.
> Sure, it's more verbose, and hence more error prone. But you are /totally=
/
> in control of the validation.
> What happens if your INI has key+value or kye=3Dvalue?
The program fails. BUT ... I don't have to change the code to get the
program to work, the user has to change the config file. It's a lot
easier on both of us if the user can do this directly without
interrupting what I'm doing or tracking me down or waiting for me to
wake up from my nap.
> What happens if your CSV is missing a , or has one too many (assuming
> notepad)?
This actually is quite common. If I can anticipate extra columns in a
particular app, I know that they are always in the same place, for
example, in one app it's common for the first two columns to be
missing, so I do this:
my @line =3D split(/,/, $_);
unshift(@line, 'NULL', 'NULL') if @line !=3D $correct_num_of_cols;
Otherwise, I do this:
my @line =3D split(/,/, $_);
if (@line !=3D $correct_num_of_cols) #can't process line
{
print ERROR_LOG "$_\n";
next; #go to next record
}
I want something (1) that I can use to directly print multi-level Perl
data structures and (2) that users can read and modify without having
to know anything about multi-level Perl data structures.
Thanks for your input, CC.
------------------------------
Date: Wed, 11 May 2011 15:49:55 -0500
From: John Bokma <john@castleamber.com>
Subject: Re: alternatives for external data storage and manipulation
Message-Id: <8762phvu6k.fsf@castleamber.com>
ccc31807 <cartercc@gmail.com> writes:
> On May 11, 1:52 pm, John Bokma <j...@castleamber.com> wrote:
>> If you use YAML/JSON you have to still check the values you get
>> (missing? within range) inside your program.
>
> Well, yes. I still have to check whether values are missing whatever
> the source. For example, users may not have a middle name or a cell
> number, and I have to anticipate that these values may be null, but I
> have no way of knowing if the middle name or cell number supplied by
> the user is correct.
Clear. With XML you can do that checking via validating using something
like RELAX NG.
>> What happens if your CSV is missing a , or has one too many (assuming
>> notepad)?
>
> This actually is quite common. If I can anticipate extra columns in a
> particular app, I know that they are always in the same place, for
> example, in one app it's common for the first two columns to be
> missing, so I do this:
> my @line = split(/,/, $_);
> unshift(@line, 'NULL', 'NULL') if @line != $correct_num_of_cols;
>
> Otherwise, I do this:
> my @line = split(/,/, $_);
> if (@line != $correct_num_of_cols) #can't process line
> {
> print ERROR_LOG "$_\n";
> next; #go to next record
> }
Clear, I do somewhat the same thing (but in this case to support an
older format with less columns).
> I want something (1) that I can use to directly print multi-level Perl
> data structures and (2) that users can read and modify without having
> to know anything about multi-level Perl data structures.
The problem is multi-level: this makes it harder on your
users. YAML/JSON/XML all can be accidentally corrupted in such a way
that what you get is not what you expect. Either it will be accepted by
your program or your program can't read the data.
While XML is more prone to such accidents (it's more text for the same
amount of data) there are also better tools (as far as I know) for your
users to verify if their file is OK and dedicated editors that make it
easier for them to enter the data. Some of those editors can just give
your users a form or spreadsheet to enter the data and always generate a
valid XML file. So impossible to mess up the structure unless they
modify the file afterward in Notepad. See for an example:
http://www.altova.com/authentic/wysiwyg-xml.html
If too much, maybe roll your own? On the left a tree structure, on the
right input fields which accept data according to some rules? This works
of course no matter of what file format you use. Only with XML it's very
likely that it already exists.
--
John Bokma j3b
Blog: http://johnbokma.com/ Facebook: http://www.facebook.com/j.j.j.bokma
Freelance Perl & Python Development: http://castleamber.com/
------------------------------
Date: Thu, 12 May 2011 07:35:12 +0200
From: Martijn Lievaart <m@rtij.nl.invlalid>
Subject: Re: Moving data from one machine to another
Message-Id: <g58s98-tpf.ln1@news.rtij.nl>
On Wed, 11 May 2011 12:48:41 +0100, Rainer Weikusat wrote:
> Not necessarily. Provided you have ssh-access, you could upload the data
> with scp/sftp and start the processing on the target machine afterwards
> by executing some command there via ssh.
or (assuming $target command takes data on stdin):
cat $file | ssh $target "$targetcommand"
or write a small wrapper that stores stdin to a tempfile; executes the
targetcommand on that tempfile and deletes the tempfile.
Obviously you have to think about errorhandling. It may be an advantage
that this solution sends the exitcode, stdout and stderr back to the
initiator, the initiator can check.
M4
------------------------------
Date: Thu, 12 May 2011 11:38:59 +0100
From: Rainer Weikusat <rweikusat@mssgmbh.com>
Subject: Re: Moving data from one machine to another
Message-Id: <87wrhww6d8.fsf@sapphire.mobileactivedefense.com>
Martijn Lievaart <m@rtij.nl.invlalid> writes:
> On Wed, 11 May 2011 12:48:41 +0100, Rainer Weikusat wrote:
>> Not necessarily. Provided you have ssh-access, you could upload the data
>> with scp/sftp and start the processing on the target machine afterwards
>> by executing some command there via ssh.
>
> or (assuming $target command takes data on stdin):
> cat $file | ssh $target "$targetcommand"
>
> or write a small wrapper that stores stdin to a tempfile; executes the
> targetcommand on that tempfile and deletes the tempfile.
The transmission and processing steps should be separate because they
could fail separately: Transmission could be interrupted 'midway'
which would cause the processing $whatever to process a 'corrupted'/
truncated file. Assuming processing fails for some reason, there is no
way to restart it without transmitting all the data again in the
example above.
Lastly,
[rw@sapphire]~/work/linux-2-6/include/asm-generic $whatis cat
cat (1) - concatenate files and print on the standard output
[rw@sapphire]~/work/linux-2-6/include/asm-generic $ssh splittermine head </etc/group
setterm: $TERM is not defined.
root:x:0:
daemon:x:1:
bin:x:2:
sys:x:3:
adm:x:4:rw
tty:x:5:
disk:x:6:
lp:x:7:
mail:x:8:
news:x:9:
------------------------------
Date: Thu, 12 May 2011 11:41:40 +0100
From: RedGrittyBrick <RedGrittyBrick@spamweary.invalid>
Subject: Re: Moving data from one machine to another
Message-Id: <4dcbb965$0$2505$db0fefd9@news.zen.co.uk>
On 11/05/2011 11:13, Justin C wrote:
> On 2011-05-10, RedGrittyBrick<RedGrittyBrick@spamweary.invalid> wrote:
>> On 10/05/2011 16:18, Justin C wrote:
>>>
>>> I'm banging my head against the wall WRT gziping the file, using
>>> MIME::Lite to attach it to an email, and then using MIME::Parser to
>>> extract the attachment the other end.
>>>
>>
>> Stop banging your head against a wall.
>>
>> I'd only use email to transfer data if connectivity between the two
>> systems is intermittent.
>>
>> Use something else - CGI, SOAP, Sockets, FTP.
>
> Actually, the main reason for email is the instant action on the
> received data that procmail can initiate. Otherwise I must have a
> process watching for the arrival of a file, spending most of it's time
> asleep, but waking once a minute/five minutes/hour/whatever depending on
> how urgently we want the data acted upon.
You already have such a program, in your proposed solution it is a Mail
Transfer Agent such as sendmail.
In the case of, say SOAP (using say SOAP::Lite) you also very likely
have the program already running - Apache. No other constantly running
process is needed.
> I dislike leaving programs
> just running because you then find you need another program to check
> once in a while that they haven't failed for some reason.
Whatever program you have that checks your SMTP server hasn't failed
could as easily check that your HTTP server hasn't failed. If you are in
fact not checking the former then you don't need to check the latter.
Here's the guts of the client program:
use SOAP::Lite +autodispatch =>
uri => 'http://www.JustinC.com/project',
proxy => 'http://datacruncher.JustinC.com/dispatcher.cgi';
act_on_data(get_data_from_database())
where get_data_from_database() is a locally defined subroutine and
act_on_data() is defined in a Perl module on the server and is
autodispatched there by SOAP::Lite.
Here's the guts of the server program
sub act_on_data {
my $data_ref = shift;
...
}
--
RGB
------------------------------
Date: Thu, 12 May 2011 22:19:08 -0700 (PDT)
From: Anton Kondratjev <blues.lead@gmail.com>
Subject: Perl + Skype4com
Message-Id: <20299fcb-9249-4d19-bd38-18f521a8c2d7@h9g2000yqk.googlegroups.com>
Please tell me anybody about Skype4com. I try sample code and can make
a call to friends from script. But I can`t capture events from skype
with Skype4com and Win32::OLE. Code, with I try to capture events is:
use Win32::OLE qw(EVENTS);
$sk=Win32::OLE->new("Skype4COM.Skype") || die "Coudn`t connect $!";
Win32::OLE->WithEvents($sk,\&Eve,"_ISkypeEvents");
$sk->Attach();
$user=$sk->CurrentUser();
$sts=$sk->Convert->OnlineStatusToText($user->OnlineStatus);
print "Current user: " . $user->{FullName} . " is " . "$sts\n";
Win32::OLE->MessageLoop();
sub Eve
{
my($msg,$stat,@args)=@_;
print "Event triggered\n";
exit 0;
}
exit 0;
I not found it problem at skype forum and i read about Win32::Skype
and Win32::OLE.
------------------------------
Date: Thu, 12 May 2011 19:36:13 +0200
From: Helmut Richter <hhr-m@web.de>
Subject: piping a vraiable content through a filter
Message-Id: <alpine.LNX.2.00.1105121922320.5086@lxhri02.ws.lrz.de>
It looks like a FAQ but I did not find it.
I have a filter program "filter" which reads from STDIN and writes to
STDOUT. Now I have in a Perl script a variable $filter_in with data that
have to be transformed to $filter_out. An obvious ways to do that would be
use File::Temp qw/tempfile/;
($fh, $fpath) = tempfile();
binmode($fh, ":utf8" );
print ($fh $filter_in);
close ($fh);
open (FILTER, "filter <$fpath |");
read (FILTER [...]
Can it be done without creating a temporary file? It is tempting to try
something like
open (FILTER, "echo '$filter_in' | filter <$fpath |");
but that places restrictions on the allowable contents of $filter_in.
If a temporary file is needed, can one do with the filehandle alone? After
all, the file has to be closed before that filter can open it.
I'd like to avoid solutions that are clumsy, unsafe, or both.
--
Helmut Richter
------------------------------
Date: Thu, 12 May 2011 19:38:34 +0200
From: Helmut Richter <hhr-m@web.de>
Subject: Re: piping a vraiable content through a filter
Message-Id: <alpine.LNX.2.00.1105121937450.5086@lxhri02.ws.lrz.de>
On Thu, 12 May 2011, Helmut Richter wrote:
> Can it be done without creating a temporary file? It is tempting to try
> something like
>
> open (FILTER, "echo '$filter_in' | filter <$fpath |");
Corr.: pen (FILTER, "echo '$filter_in' | filter |");
Of course.
--
Helmut Richter
------------------------------
Date: Thu, 12 May 2011 19:04:48 +0100
From: Rainer Weikusat <rweikusat@mssgmbh.com>
Subject: Re: piping a vraiable content through a filter
Message-Id: <87boz7x0an.fsf@sapphire.mobileactivedefense.com>
Helmut Richter <hhr-m@web.de> writes:
> I have a filter program "filter" which reads from STDIN and writes to
> STDOUT. Now I have in a Perl script a variable $filter_in with data
> that have to be transformed to $filter_out. An obvious ways to do that
> would be
>
> use File::Temp qw/tempfile/;
> ($fh, $fpath) = tempfile();
> binmode($fh, ":utf8" );
> print ($fh $filter_in);
> close ($fh);
> open (FILTER, "filter <$fpath |");
> read (FILTER [...]
>
> Can it be done without creating a temporary file?
Have a look at IPC::Open3(3pm).
------------------------------
Date: 6 Apr 2001 21:33:47 GMT (Last modified)
From: Perl-Users-Request@ruby.oce.orst.edu (Perl-Users-Digest Admin)
Subject: Digest Administrivia (Last modified: 6 Apr 01)
Message-Id: <null>
Administrivia:
To submit articles to comp.lang.perl.announce, send your article to
clpa@perl.com.
Back issues are available via anonymous ftp from
ftp://cil-www.oce.orst.edu/pub/perl/old-digests.
#For other requests pertaining to the digest, send mail to
#perl-users-request@ruby.oce.orst.edu. Do not waste your time or mine
#sending perl questions to the -request address, I don't have time to
#answer them even if I did know the answer.
------------------------------
End of Perl-Users Digest V11 Issue 3382
***************************************
