[30022] in Perl-Users-Digest
Perl-Users Digest, Issue: 1265 Volume: 11
daemon@ATHENA.MIT.EDU (Perl-Users Digest)
Fri Feb 8 16:09:45 2008
Date: Fri, 8 Feb 2008 13:09:09 -0800 (PST)
From: Perl-Users Digest <Perl-Users-Request@ruby.OCE.ORST.EDU>
To: Perl-Users@ruby.OCE.ORST.EDU (Perl-Users Digest)
Perl-Users Digest Fri, 8 Feb 2008 Volume: 11 Number: 1265
Today's topics:
manually installing a ppm <nospam@nospam.com>
Re: moving binary data from one RDBMS to Other <a@a.com>
Re: moving binary data from one RDBMS to Other <hjp-usenet2@hjp.at>
Re: moving binary data from one RDBMS to Other <a@a.com>
Re: moving binary data from one RDBMS to Other <john@castleamber.com>
Re: moving binary data from one RDBMS to Other <hjp-usenet2@hjp.at>
Re: moving binary data from one RDBMS to Other <a@a.com>
Re: moving binary data from one RDBMS to Other <john@castleamber.com>
Re: Need regexp to parse newsgroups <me@privacy.invalid>
Re: problem with '>' character <stoupa@practisoft.cz>
Using END and the die function <pgodfrin@gmail.com>
Re: Using END and the die function <smallpond@juno.com>
Re: Using END and the die function <simon.chao@fmr.com>
Re: Using END and the die function <pgodfrin@gmail.com>
Re: XS code <ben@morrow.me.uk>
Digest Administrivia (Last modified: 6 Apr 01) (Perl-Users-Digest Admin)
----------------------------------------------------------------------
Date: Fri, 8 Feb 2008 13:15:27 -0000
From: "Nospam" <nospam@nospam.com>
Subject: manually installing a ppm
Message-Id: <fohklv$8gd$1@north.jnrs.ja.net>
For some strange reason when unable to install a ppm with activeperl, what
are the step by step instructions on installing this manually?
------------------------------
Date: 8 Feb 2008 06:23:58 -0800
From: Perl Lover <a@a.com>
Subject: Re: moving binary data from one RDBMS to Other
Message-Id: <foholu0dui@drn.newsguy.com>
In article <pan.2008.02.07.17.14.32@rtij.nl.invlalid>, Martijn Lievaart says...
>First try to od -x the file and see if it contains exactly what it
>should. If it doesn't there's a problem getting the data from the
>database.
>
>Now read the data back and dump that (Data::Dumper will probably do,
>otherwise convert to hex with a small loop). Still the same? Then your
>problem is in storing the data.
>
>A possible problem with reading from the database is character set. A
>VARCHAR is not ment to hold binary data and you may run into characterset
>conversions here, or something else.
I am looking at the possibility of perl incorrectly reporting two
identical binary strings as different.
Can two variables storing binary data be comapred
by standard eq operator as in
if ( $a eq $b )
Does it do the equivalent C function memcmp.
Is there a fool proof way of comparing these two variables.
Thanks.
------------------------------
Date: Fri, 8 Feb 2008 18:05:16 +0100
From: "Peter J. Holzer" <hjp-usenet2@hjp.at>
Subject: Re: moving binary data from one RDBMS to Other
Message-Id: <slrnfqp2ud.ag4.hjp-usenet2@hrunkner.hjp.at>
On 2008-02-08 14:23, Perl Lover <a@a.com> wrote:
> In article <pan.2008.02.07.17.14.32@rtij.nl.invlalid>, Martijn Lievaart says...
[...]
>>A possible problem with reading from the database is character set. A
>>VARCHAR is not ment to hold binary data and you may run into characterset
>>conversions here, or something else.
>
> I am looking at the possibility of perl incorrectly reporting two
> identical binary strings as different.
>
> Can two variables storing binary data be comapred
> by standard eq operator as in
> if ( $a eq $b )
Yes.
> Does it do the equivalent C function memcmp.
Not quite. memcmp compares bytes irrespective of type. The eq operator
makes sure that both operands are of the same type before comparing
them. If one of the operands isn't a string it is converted to a
string. If one of the strings is a byte string, and the other is a
character string, the byte string is converted to a character string.
So "eq" tests if the operands have the same string value, not the same
representation.
hp
------------------------------
Date: 8 Feb 2008 09:43:17 -0800
From: Perl Lover <a@a.com>
Subject: Re: moving binary data from one RDBMS to Other
Message-Id: <foi4bl01aq1@drn.newsguy.com>
>Not quite. memcmp compares bytes irrespective of type. The eq operator
>makes sure that both operands are of the same type before comparing
>them. If one of the operands isn't a string it is converted to a
>string. If one of the strings is a byte string, and the other is a
>character string, the byte string is converted to a character string.
>
>So "eq" tests if the operands have the same string value, not the same
>representation.
Hmm, this might be a clue to the problem we are facing.
I am reading data from a DB2 database where the field type is char(128)
and it contains binary data (db2 allows it). The reason why binary
data is used is because the data is copied via memcpy to an array
of integers and then processed. Apparently this has given
a huge performance boost. It works great. Oracle is the database
where it is written to and it does not allow binary data in char field.
So we are using RAW, LONG- anything that allows us to store binary data
as it is.
I have a script in perl to compare them after it is written to Oracle.
So I read both DB2 and Oracle row and use eq operator
and it says "not equal". When we use BLOB data type to store the
value in Oracle, it shows them as equal. I know Oracle stores
BLOB data as it is with no characterset conversion.
So to me, it proves that the problem is
in Oracle storing the data in raw/long mode
but others are skeptical about perl. Hence my post.
If you can recommend a better way of comparing this data, that would
be great.
thanks all.
------------------------------
Date: 8 Feb 2008 18:35:12 GMT
From: John Bokma <john@castleamber.com>
Subject: Re: moving binary data from one RDBMS to Other
Message-Id: <Xns9A3E8009F24B3castleamber@130.133.1.4>
Perl Lover <a@a.com> wrote:
>>Not quite. memcmp compares bytes irrespective of type. The eq operator
>>makes sure that both operands are of the same type before comparing
>>them. If one of the operands isn't a string it is converted to a
>>string. If one of the strings is a byte string, and the other is a
>>character string, the byte string is converted to a character string.
>>
>>So "eq" tests if the operands have the same string value, not the same
>>representation.
>
> Hmm, this might be a clue to the problem we are facing.
>
> I am reading data from a DB2 database where the field type is char(128)
> and it contains binary data (db2 allows it). The reason why binary
> data is used is because the data is copied via memcpy to an array
> of integers and then processed. Apparently this has given
> a huge performance boost. It works great. Oracle is the database
> where it is written to and it does not allow binary data in char field.
> So we are using RAW, LONG- anything that allows us to store binary data
> as it is.
>
> I have a script in perl to compare them after it is written to Oracle.
> So I read both DB2 and Oracle row and use eq operator
> and it says "not equal". When we use BLOB data type to store the
> value in Oracle, it shows them as equal. I know Oracle stores
> BLOB data as it is with no characterset conversion.
> So to me, it proves that the problem is
> in Oracle storing the data in raw/long mode
> but others are skeptical about perl. Hence my post.
>
> If you can recommend a better way of comparing this data, that would
> be great.
Have you already considered to actually print the data (in hex format for
example) and visually compare them, or use a loop and compare byte by byte
and warn when a difference occur.
Did you check the length of each string?
--
John
http://johnbokma.com/perl/
------------------------------
Date: Fri, 8 Feb 2008 20:02:16 +0100
From: "Peter J. Holzer" <hjp-usenet2@hjp.at>
Subject: Re: moving binary data from one RDBMS to Other
Message-Id: <slrnfqp9pp.b27.hjp-usenet2@hrunkner.hjp.at>
On 2008-02-08 17:43, Perl Lover <a@a.com> wrote:
>>Not quite. memcmp compares bytes irrespective of type. The eq operator
>>makes sure that both operands are of the same type before comparing
>>them. If one of the operands isn't a string it is converted to a
>>string. If one of the strings is a byte string, and the other is a
>>character string, the byte string is converted to a character string.
>>
>>So "eq" tests if the operands have the same string value, not the same
>>representation.
>
> Hmm, this might be a clue to the problem we are facing.
>
> I am reading data from a DB2 database where the field type is char(128)
> and it contains binary data (db2 allows it). The reason why binary
> data is used is because the data is copied via memcpy to an array
> of integers and then processed. Apparently this has given
> a huge performance boost. It works great. Oracle is the database
> where it is written to and it does not allow binary data in char field.
> So we are using RAW, LONG- anything that allows us to store binary data
> as it is.
LONG is a character type and doesn't allow binary data (or rather, it
allows it in the same way as VARCHAR2 and CLOB: You may get away with it
if you are lucky). RAW and LONG RAW are intended for storing binary
data, as is BLOB.
The LONG types are deprecated: Instead of LONG you should use CLOB, and
instead of LONG RAW you should use BLOB.
Since you only need to store 128 bytes, RAW is probably the best type.
But I've never used RAW and it is possible that DBD::Oracle doesn't
handle it correctly in some cases.
> I have a script in perl to compare them after it is written to Oracle.
> So I read both DB2 and Oracle row and use eq operator
> and it says "not equal".
You should also print the differences between the strings. Often the
problem is apparent when you see what the difference is.
The DBI package contains some useful utility functions.
> When we use BLOB data type to store the
> value in Oracle, it shows them as equal.
That's new. I understood your previous posting that it doesn't work with
blobs if the data is written to a file and read back - which suggested
some problem related to file handling.
> If you can recommend a better way of comparing this data, that would
> be great.
eq is the correct way to compare strings for equality in perl. There is
no better way. However, you may also want to know whether the strings
are character or byte strings: See the utf8::is_utf8 function. And you
almost always want to know *how* the strings are different.
hp
------------------------------
Date: 8 Feb 2008 11:30:50 -0800
From: Perl Lover <a@a.com>
Subject: Re: moving binary data from one RDBMS to Other
Message-Id: <foiala01mrp@drn.newsguy.com>
In article <slrnfqp9pp.b27.hjp-usenet2@hrunkner.hjp.at>, Peter J. Holzer says...
>> When we use BLOB data type to store the
>> value in Oracle, it shows them as equal.
>
>That's new. I understood your previous posting that it doesn't work with
>blobs if the data is written to a file and read back - which suggested
>some problem related to file handling.
This is true. It works only with direct feed and not via file.
This is what I wrote originally:
"I am currently writing a perl dbi script to copy data from one rdbms to
another.
One of the columns contain binary data. When I copy the data directly (reading
it from source, feeding it to target), it works fine. However when I save the
source data first in a file and then use that file to load it to the target it
messes up the binary data."
I use print FILEHANDLE to write it to the file.
------------------------------
Date: 8 Feb 2008 20:02:10 GMT
From: John Bokma <john@castleamber.com>
Subject: Re: moving binary data from one RDBMS to Other
Message-Id: <Xns9A3E8EC8AFDF5castleamber@130.133.1.4>
Perl Lover <a@a.com> wrote:
[ binary data ]
> I use print FILEHANDLE to write it to the file.
perldoc -f binmode (note the 3rd paragraph)
If that's not the issue, use od -x file (assuming *nix) to get a hex dump
of the contents. Compare it with a hex dump of the string before you print
it to file.
--
John
Arachnids near Coyolillo - part 1
http://johnbokma.com/mexit/2006/05/04/arachnids-coyolillo-1.html
------------------------------
Date: Fri, 08 Feb 2008 11:59:10 GMT
From: "M.L." <me@privacy.invalid>
Subject: Re: Need regexp to parse newsgroups
Message-Id: <iuXqj.7616$0o7.5841@newssvr13.news.prodigy.net>
>> {} In short, I want the following newsgroup examples to pass the
>> Xref
>> {} filter so they can be dropped:
>> {} general.soc
>> {} soc.general
>> {} soc.general.sci
>> {} francom.chatting.generale
>> {}
>> {} I want the following two to fail so they will be retained:
>> {} chi.general
>> {} microsoft.public.windowsxp.general
>> {}
>> {} Any assistance would be greatly appreciated. Thanks in advance.
>
>> Untested:
>
>> /(?!\Qchi.general\E|\Qmicrosoft.public.windowsxp.general\E)(?<!\S)\S*general\S
>> */
>>
>
>> @groups = grep {$_ eq 'chi.general' ||
>> $_ eq 'microsoft.public.windowsxp.general' ||
>> !/general/} @groups;
>
>
> As Abigail does, forget about the exceptions in the regex. Pass those
> through first then look at the rest. Note that I added \b to regex so
> you don't filter out alt.generalissimo.franco. :)
>
> #!/usr/bin/perl
>
> my $string = "prodigy.net general.soc:449517 chi.general:641065
> soc.general.sci:329682 francom.chatting.generale:152591";
>
> %Exceptions = map { $_, 1 } qw( chi.general
> microsoft.public.windowsxp.general );
>
> print
> join " ",
> grep {
> my( $group, $number ) = split /:/;
> exists $Exceptions{$group} or $group !~ /\bgenerale?\b/
> }
> split /\s+/, $string;
Sorry to be replying so late. Thanks for the advice, I'll try to follow
what I can. I need to mention that the regexp is not going to be used
within a Perl script. It's just an entry to a Windows program that uses
Perl regexps for filtering. So grep and map are out. I'll check to see
if Abigail's will work though. Thanks again.
------------------------------
Date: Fri, 8 Feb 2008 15:14:00 +0100
From: "Petr Vileta" <stoupa@practisoft.cz>
Subject: Re: problem with '>' character
Message-Id: <fohoc6$1mbs$1@ns.felk.cvut.cz>
Abigail wrote:
> _
> Petr Vileta (stoupa@practisoft.cz) wrote on VCCLXXIV September
> MCMXCIII in <URL:news:fogg9n$13pf$2@ns.felk.cvut.cz>:
> && JD wrote:
> && > On 2008-02-07, Abigail <abigail@abigail.be> wrote:
> && >> _
> && >>
> && >> That seems like a bug.
> && >>
> && >
> && > Found it: It was a a file from a MS-Windows system, with CRLF
> line && > ends. chomp removed the LF, but the CR was still there :(
> && >
> && >
> && > dos2unix solved it.
> && Or you can use regexp intead of chomp
> &&
> && s/[\r\n]//;
>
>
> That just deletes the first CR *or* NL from the string. This has the
> problem that instead of leaving the CR, it leaves the NL.
>
> If you want to use a regexp, I'd use:
>
> s/\R//;
>
> or, if you're working with an old perl:
>
> s/\r?\n|\r//
>
Yes, I'm working with old Perl and regex should be
s/[\r\n]*$//g;
--
Petr Vileta, Czech republic
(My server rejects all messages from Yahoo and Hotmail. Send me your
mail from another non-spammer site please.)
Please reply to <petr AT practisoft DOT cz>
------------------------------
Date: Fri, 8 Feb 2008 11:07:04 -0800 (PST)
From: pgodfrin <pgodfrin@gmail.com>
Subject: Using END and the die function
Message-Id: <45aee011-6469-4080-8a90-e29632df7a5e@e10g2000prf.googlegroups.com>
Greetings,
I'd like to die() with a return code (no pun intended) - I'm not sure
how to do it without wrapping some extra code around die. Any
thoughts?
Here's a snippet:
if($?) { $return_code=$?; die "Error generating temp file names\n" ;}
...
END
{
if (defined WL) { close WL; }
print "Script ended. RC=$return_code \n";
$?=$return_code;
}
This works fine if I explicitly code both steps - first setting the
$return_code and then executing die. But, this kind of statement won't
permit that 'cause of the nature of the or operator:
mkdir($tgt_dir) or die "Mkdir ($tgt_dir) command failed.\n";
Is there an elegant way to this or will I need to use if statements
for anything I want to test for errors?
pg
------------------------------
Date: Fri, 8 Feb 2008 11:23:36 -0800 (PST)
From: smallpond <smallpond@juno.com>
Subject: Re: Using END and the die function
Message-Id: <df256f92-264c-4cc2-aad3-0bc37dec7021@c23g2000hsa.googlegroups.com>
On Feb 8, 2:07 pm, pgodfrin <pgodf...@gmail.com> wrote:
> Greetings,
> I'd like to die() with a return code (no pun intended) - I'm not sure
> how to do it without wrapping some extra code around die. Any
> thoughts?
>
> Here's a snippet:
> if($?) { $return_code=$?; die "Error generating temp file names\n" ;}
> ...
> END
> {
> if (defined WL) { close WL; }
> print "Script ended. RC=$return_code \n";
> $?=$return_code;
>
> }
>
> This works fine if I explicitly code both steps - first setting the
> $return_code and then executing die. But, this kind of statement won't
> permit that 'cause of the nature of the or operator:
>
> mkdir($tgt_dir) or die "Mkdir ($tgt_dir) command failed.\n";
>
> Is there an elegant way to this or will I need to use if statements
> for anything I want to test for errors?
> pg
how about:
sub mydie {
$!= shift;
die @_;
}
mkdir($tgt_dir) or mydie 3,"Mkdir ($tgt_dir) command failed.\n";
------------------------------
Date: Fri, 8 Feb 2008 11:28:47 -0800 (PST)
From: nolo contendere <simon.chao@fmr.com>
Subject: Re: Using END and the die function
Message-Id: <91d88c4b-7bee-4053-9699-5b052943e8ee@i29g2000prf.googlegroups.com>
On Feb 8, 2:07=A0pm, pgodfrin <pgodf...@gmail.com> wrote:
> Greetings,
> I'd like to die() with a return code (no pun intended) - I'm not sure
> how to do it without wrapping some extra code around die. Any
> thoughts?
>
> Here's a snippet:
> if($?) { $return_code=3D$?; die "Error generating temp file names\n" ;}
> ...
> END
> {
> if (defined WL) { close WL; }
> print "Script ended. RC=3D$return_code \n";
> $?=3D$return_code;
>
> }
>
> This works fine if I explicitly code both steps - first setting the
> $return_code and then executing die. But, this kind of statement won't
> permit that 'cause of the nature of the or operator:
>
> mkdir($tgt_dir) or die "Mkdir ($tgt_dir) command failed.\n";
>
> Is there an elegant way to this or will I need to use if statements
> for anything I want to test for errors?
Would 'exit' suit your needs? Just 'warn' whatever message you want,
then call 'exit' with the return code you want.
------------------------------
Date: Fri, 8 Feb 2008 11:35:16 -0800 (PST)
From: pgodfrin <pgodfrin@gmail.com>
Subject: Re: Using END and the die function
Message-Id: <a4935cfe-b0cd-4bbc-8967-4a8877bc3be4@b2g2000hsg.googlegroups.com>
On Feb 8, 1:28 pm, nolo contendere <simon.c...@fmr.com> wrote:
> On Feb 8, 2:07 pm, pgodfrin <pgodf...@gmail.com> wrote:
>
>
>
> > Greetings,
> > I'd like to die() with a return code (no pun intended) - I'm not sure
> > how to do it without wrapping some extra code around die. Any
> > thoughts?
>
> > Here's a snippet:
> > if($?)
> > ...
> > END
> > {
> > if (defined WL) { close WL; }
> > print "Script ended. RC=$return_code \n";
> > $?=$return_code;
>
> > }
>
> > This works fine if I explicitly code both steps - first setting the
> > $return_code and then executing die. But, this kind of statement won't
> > permit that 'cause of the nature of the or operator:
>
> > mkdir($tgt_dir) or die "Mkdir ($tgt_dir) command failed.\n";
>
> > Is there an elegant way to this or will I need to use if statements
> > for anything I want to test for errors?
>
> Would 'exit' suit your needs? Just 'warn' whatever message you want,
> then call 'exit' with the return code you want.
That would work, but it's still a two step operation, so I couldn't
use the or operator. using a "mydie" function is just a wrapper and
while that would work too, I'm seeking a more 'stylish' or elegant way
to do it. Frankly I could just do: if (!mkdir($tgt_dir))
{ $return_code=$?; die "Error generating temp file names\n" ;}.
But I'm feeling a bit whiney and thought there might be a 'cleaner
way'.
pg
------------------------------
Date: Fri, 8 Feb 2008 13:55:50 +0000
From: Ben Morrow <ben@morrow.me.uk>
Subject: Re: XS code
Message-Id: <6k3u75-g82.ln1@osiris.mauzo.dyndns.org>
Quoth Robert Henniger <robert.henniger@googlemail.com>:
>
[HTML::Template::Pro]
>
> I checked my code, and of course i am using the query methode. And yes
> this methode is not implemented yet.
> Do you know if it is possible to implement this methode in the perl
> part or do i have to do that in the xs code?
I'v really no idea. I would suspect that it's Hard to implement in HTP
without losing the speed, as otherwise it would be there already.
> I am very sorry that i wasted your time with my stupid questions.
That's OK: I don't have to answer them :).
Ben
------------------------------
Date: 6 Apr 2001 21:33:47 GMT (Last modified)
From: Perl-Users-Request@ruby.oce.orst.edu (Perl-Users-Digest Admin)
Subject: Digest Administrivia (Last modified: 6 Apr 01)
Message-Id: <null>
Administrivia:
#The Perl-Users Digest is a retransmission of the USENET newsgroup
#comp.lang.perl.misc. For subscription or unsubscription requests, send
#the single line:
#
# subscribe perl-users
#or:
# unsubscribe perl-users
#
#to almanac@ruby.oce.orst.edu.
NOTE: due to the current flood of worm email banging on ruby, the smtp
server on ruby has been shut off until further notice.
To submit articles to comp.lang.perl.announce, send your article to
clpa@perl.com.
#To request back copies (available for a week or so), send your request
#to almanac@ruby.oce.orst.edu with the command "send perl-users x.y",
#where x is the volume number and y is the issue number.
#For other requests pertaining to the digest, send mail to
#perl-users-request@ruby.oce.orst.edu. Do not waste your time or mine
#sending perl questions to the -request address, I don't have time to
#answer them even if I did know the answer.
------------------------------
End of Perl-Users Digest V11 Issue 1265
***************************************
