[27138] in Perl-Users-Digest
Perl-Users Digest, Issue: 8992 Volume: 10
daemon@ATHENA.MIT.EDU (Perl-Users Digest)
Fri Feb 24 21:05:37 2006
Date: Fri, 24 Feb 2006 18:05:03 -0800 (PST)
From: Perl-Users Digest <Perl-Users-Request@ruby.OCE.ORST.EDU>
To: Perl-Users@ruby.OCE.ORST.EDU (Perl-Users Digest)
Perl-Users Digest Fri, 24 Feb 2006 Volume: 10 Number: 8992
Today's topics:
Re: Decimal equality question <jurgenex@hotmail.com>
Re: Implementing a "pull" (?) interface in perl robic0
Re: Implementing a "pull" (?) interface in perl robic0
Re: sharing variables-data perl-asp <jay@cutmeukjay.com>
Re: sharing variables-data perl-asp <matthew.garrish@sympatico.ca>
Digest Administrivia (Last modified: 6 Apr 01) (Perl-Users-Digest Admin)
----------------------------------------------------------------------
Date: Sat, 25 Feb 2006 02:04:38 GMT
From: "Jürgen Exner" <jurgenex@hotmail.com>
Subject: Re: Decimal equality question
Message-Id: <WWOLf.10239$fU6.10137@trnddc08>
a wrote:
> this is probably a dumb question, but i'm not seeing why perl is
> treating these numbers as unequal. i've tried on a few different
> machines (intel and sparc) and perl versions (5.8.7, 5.8.3) and get
> the same result.
>
> perl -e 'if (361.35 == (72.27*5)) { print "EQUAL"; } else {print "NOT
> EQUAL";}'
First commandment of computer numerics:
"Thou shalt not use equality on floating point numbers."
For details see "perldoc -q 999" and/or the numerous previous articles on
this topic.
jue
------------------------------
Date: Fri, 24 Feb 2006 17:23:03 -0800
From: robic0
Subject: Re: Implementing a "pull" (?) interface in perl
Message-Id: <cv8vv19s843svnr3po3h142ednjktgglmb@4ax.com>
On Tue, 21 Feb 2006 09:28:28 -0800, Arvin Portlock <nomail@sorry.com> wrote:
>I'm writing a module to parse an XML file of records. It
>will be used by a variety of different applications, e.g.,
>loading into a relational database, etc. I'll be using
>a SAX based approach, ExpatXS, as the XML files can be
>very large.
>
>In the past I've written such modules by assembling a huge
>data structure in memory then returning it to the calling
>application as, say, a reference to an array of hashes.
>This was tremendously convenient yet very very slow. Some
>applications would take hours to execute. This time around
>I'd like to learn something new and approach it differently.
>
>Is there some way to design this, module plus application,
>so that as a record is read the application can process it
>immediately? Is this what is know as a pull-based architecture?
>How does the application "know" when a new record is available?
>Does it listen for something that the module emits? I'm
>thinking maybe it can be done with a callback. The callback
>subroutine is written in the calling application and when
>the end of the record is parsed, that subroutine is called.
>
>I'm sure this is a basic question but it's new to me. Is my
>callback idea worth exploring? Are there any design patterns
>people can point me to? Example programs? Articles online?
>
>Thanks!
>
>Arvin
I guess i'm coming late to this question but will give it a shot
for you. I see some code thrown around so i'll throw some too.
First and formost, if your processing a xtra large xml file
you want to use a "stream" processor where you get start/end tag/content
notification. The stream processor, if passed a file handle should do something
like this:
"p" is a parsing object instantiated from your program.
$p->parse(*DATA);
-------------
here's what happens:
"module"
============
sub parse {
my ($self, $data) = @_;
throwX ('30') unless (!$self->{'InParse'});
throwX ('31') unless (defined $data);
$self->{'InParse'} = 1;
# call processor
if (ref($data) eq 'SCALAR') {
print "SCALAR ref\n" if ($self->{'debug'});
eval {Processor($self, 1, $data);};
if ($@) {
Cleanup($self); die $@;
}
}
elsif (ref(\$data) eq 'SCALAR') {
print "SCALAR string\n" if ($self->{'debug'});
eval {Processor($self, 1, \$data);};
if ($@) {
Cleanup($self); die $@;
}
} else {
if (ref($data) ne 'GLOB' && ref(\$data) ne 'GLOB') {
$self->{'InParse'} = 0;
die "rp_error_parse, data source not a string or filehandle nor reference to one\n";
}
print "GLOB ref or filehandle\n" if ($self->{'debug'});
eval {Processor($self, 0, $data);};
if ($@) {
Cleanup($self); die $@;
}
}
$self->{'InParse'} = 0;
}
sub Processor
{
my ($obj, $BUFFERED, $rpl_mk) = @_;
my ($markup_file);
my $parse_ln = '';
my $dyna_ln = '';
my $ref_parse_ln = \$parse_ln;
my $ref_dyna_ln = \$dyna_ln;
if ($BUFFERED) {
$ref_parse_ln = $rpl_mk;
$ref_dyna_ln = \$dyna_ln;
} else {
# assume its a ref to a global or global itself
$markup_file = $rpl_mk;
$ref_dyna_ln = $ref_parse_ln;
}
my $ln_cnt = 0;
my $complete_comment = 0;
my $complete_cdata = 0;
my @Tags = ();
my $havroot = 0;
my $last_cpos = 0;
my $done = 0;
my $content = '';
my $altcontent = undef;
$obj->{'origcontent'} = \$content;
while (!$done)
{
$ln_cnt++;
# stream processing (if not buffered)
if (!$BUFFERED) {
if (!($_ = <$markup_file>)) {
# just parse what we have
$done = 1;
# boundry check for runnaway
if (($complete_comment+$complete_cdata) > 0) {
$ln_cnt--;
}
} else {
$$ref_parse_ln .= $_;
## buffer if needing comment/cdata closure
next if ($complete_comment && !/-->/);
next if ($complete_cdata && !/\]\]>/);
## reset comment/cdata flags
$complete_comment = 0;
$complete_cdata = 0;
## flag serialized comments/cdata buffering
if (/(<!--)|(<!\[CDATA\[)/)
{
if (defined $1) { # complete comment
if ($$ref_parse_ln !~ /<!--.*?-->/s) {
$complete_comment = 1;
next;
}
}
elsif (defined $2) { # complete cdata
if ($$ref_parse_ln !~ /<!\[CDATA\[.*?\]\]>/s) {
$complete_cdata = 1;
next;
}
}
}
## buffer until '>' or eof
next if (!/>/);
}
} else {
$ln_cnt = 1;
$done = 1;
}
## REGEX Parsing loop
while ($$ref_parse_ln =~ /$RxParse/g)
{
... the core ...
does callbacks here and alot of other shit, section totals about 3000 lines
}
}
}
=============
This just happens to handle eol oriented file io (also ram based io) and
a buffer passed reference (from a slurrped file? .. this is 20% faster).
The passed file handle technique above will read past as many eol's as
necessary to get a target processing character. The eol is *not* even a
factor. Beyond the *target*, this technique does not buffer file data,
so a very *large* file consumes almost no ram in the transaction.
In order to guarantee the structural integrity of your records you have
to use a shcema checker ahead of time. If you don't care and there's a problem,
well..... Shucks!
In a programming sense, you will want to validate that integrity,
otherwise all that is said below is invalid.
Use a schema checker (and have a schema file, this is mandatory) before you
parse. Then when you parse, you have safeguards.
If you want detailed instructions on how to install Xerces, let me know.
Xerces is a robo-lib generated interface to Apache's Xerces.
A note about schema. Schema is not a guarantee of structural integrity.
It becomes apparent if you have complex possibilities and schema is only
general, as in level oriented. I could give examples and solutions if requested.
When you parse, it is you who is parsing the data into records (structures), not
the parser. You should be in control of start and end of your records.
You have control in your program *when* the eor state is valid. That validity
will be reached by *you* in the context of *your* program.
Before you initiate parsing, you should validate the file (if you can), something
like this:
your program..
use XML::Xerces;
if (!ValidateSchema ($Xmlfilename,$SchemaFilename)) {not valid};
sub ValidateSchema {
my ($xfile, $SchemaFile) = @_;
# Docs: http://xml.apache.org/xerces-c/apiDocs/classAbstractDOMParser.html#z869_9
my $Xparser = XML::Xerces::XercesDOMParser->new();
$Xparser->setValidationScheme(1);
$Xparser->setDoNamespaces(1);
$Xparser->setDoSchema(1);
#$Xparser->setValidationSchemaFullChecking(1); # full constraint (if enabled, may be time-consuming)
$Xparser->setExternalNoNamespaceSchemaLocation($SchemaFile);
my $ERROR_HANDLER = XLoggingErrorHandler->new(\&LogX_warn, \&LogX_error, \&LogX_ferror, );
$Xparser->setErrorHandler($ERROR_HANDLER);
eval {$Xparser->parse (XML::Xerces::LocalFileInputSource->new($xfile));};
return 1;
}
Ok, i got a important phone call, so i'll continue this in a little later.
The important stuff comes on how to easily
pick off your records and process them in-stream. No problem whatsoever.
Brb..
------------------------------
Date: Fri, 24 Feb 2006 17:48:13 -0800
From: robic0
Subject: Re: Implementing a "pull" (?) interface in perl
Message-Id: <3pdvv1tffvlhb62f2hv9kg5mtrd1o99mge@4ax.com>
On Fri, 24 Feb 2006 17:23:03 -0800, robic0 wrote:
Ok, its gonna have to wait until tommorow, something has come up...
>On Tue, 21 Feb 2006 09:28:28 -0800, Arvin Portlock <nomail@sorry.com> wrote:
>
>>I'm writing a module to parse an XML file of records. It
>>will be used by a variety of different applications, e.g.,
>>loading into a relational database, etc. I'll be using
>>a SAX based approach, ExpatXS, as the XML files can be
>>very large.
>>
>>In the past I've written such modules by assembling a huge
>>data structure in memory then returning it to the calling
>>application as, say, a reference to an array of hashes.
>>This was tremendously convenient yet very very slow. Some
>>applications would take hours to execute. This time around
>>I'd like to learn something new and approach it differently.
>>
>>Is there some way to design this, module plus application,
>>so that as a record is read the application can process it
>>immediately? Is this what is know as a pull-based architecture?
>>How does the application "know" when a new record is available?
>>Does it listen for something that the module emits? I'm
>>thinking maybe it can be done with a callback. The callback
>>subroutine is written in the calling application and when
>>the end of the record is parsed, that subroutine is called.
>>
>>I'm sure this is a basic question but it's new to me. Is my
>>callback idea worth exploring? Are there any design patterns
>>people can point me to? Example programs? Articles online?
>>
>>Thanks!
>>
>>Arvin
>
>
>I guess i'm coming late to this question but will give it a shot
>for you. I see some code thrown around so i'll throw some too.
>
>First and formost, if your processing a xtra large xml file
>you want to use a "stream" processor where you get start/end tag/content
>notification. The stream processor, if passed a file handle should do something
>like this:
>
>"p" is a parsing object instantiated from your program.
>
>$p->parse(*DATA);
>
>-------------
>here's what happens:
>
>"module"
>============
>sub parse {
> my ($self, $data) = @_;
> throwX ('30') unless (!$self->{'InParse'});
> throwX ('31') unless (defined $data);
> $self->{'InParse'} = 1;
>
> # call processor
> if (ref($data) eq 'SCALAR') {
> print "SCALAR ref\n" if ($self->{'debug'});
> eval {Processor($self, 1, $data);};
> if ($@) {
> Cleanup($self); die $@;
> }
> }
> elsif (ref(\$data) eq 'SCALAR') {
> print "SCALAR string\n" if ($self->{'debug'});
> eval {Processor($self, 1, \$data);};
> if ($@) {
> Cleanup($self); die $@;
> }
> } else {
> if (ref($data) ne 'GLOB' && ref(\$data) ne 'GLOB') {
> $self->{'InParse'} = 0;
> die "rp_error_parse, data source not a string or filehandle nor reference to one\n";
> }
> print "GLOB ref or filehandle\n" if ($self->{'debug'});
> eval {Processor($self, 0, $data);};
> if ($@) {
> Cleanup($self); die $@;
> }
> }
> $self->{'InParse'} = 0;
>}
>
>sub Processor
>{
> my ($obj, $BUFFERED, $rpl_mk) = @_;
> my ($markup_file);
> my $parse_ln = '';
> my $dyna_ln = '';
> my $ref_parse_ln = \$parse_ln;
> my $ref_dyna_ln = \$dyna_ln;
> if ($BUFFERED) {
> $ref_parse_ln = $rpl_mk;
> $ref_dyna_ln = \$dyna_ln;
> } else {
> # assume its a ref to a global or global itself
> $markup_file = $rpl_mk;
> $ref_dyna_ln = $ref_parse_ln;
> }
> my $ln_cnt = 0;
> my $complete_comment = 0;
> my $complete_cdata = 0;
> my @Tags = ();
> my $havroot = 0;
> my $last_cpos = 0;
> my $done = 0;
> my $content = '';
> my $altcontent = undef;
>
> $obj->{'origcontent'} = \$content;
>
> while (!$done)
> {
> $ln_cnt++;
>
> # stream processing (if not buffered)
> if (!$BUFFERED) {
> if (!($_ = <$markup_file>)) {
> # just parse what we have
> $done = 1;
> # boundry check for runnaway
> if (($complete_comment+$complete_cdata) > 0) {
> $ln_cnt--;
> }
> } else {
> $$ref_parse_ln .= $_;
>
> ## buffer if needing comment/cdata closure
> next if ($complete_comment && !/-->/);
> next if ($complete_cdata && !/\]\]>/);
>
> ## reset comment/cdata flags
> $complete_comment = 0;
> $complete_cdata = 0;
>
> ## flag serialized comments/cdata buffering
> if (/(<!--)|(<!\[CDATA\[)/)
> {
> if (defined $1) { # complete comment
> if ($$ref_parse_ln !~ /<!--.*?-->/s) {
> $complete_comment = 1;
> next;
> }
> }
> elsif (defined $2) { # complete cdata
> if ($$ref_parse_ln !~ /<!\[CDATA\[.*?\]\]>/s) {
> $complete_cdata = 1;
> next;
> }
> }
> }
> ## buffer until '>' or eof
> next if (!/>/);
> }
> } else {
> $ln_cnt = 1;
> $done = 1;
> }
>
> ## REGEX Parsing loop
> while ($$ref_parse_ln =~ /$RxParse/g)
> {
> ... the core ...
> does callbacks here and alot of other shit, section totals about 3000 lines
> }
> }
>}
>=============
>
>This just happens to handle eol oriented file io (also ram based io) and
>a buffer passed reference (from a slurrped file? .. this is 20% faster).
>The passed file handle technique above will read past as many eol's as
>necessary to get a target processing character. The eol is *not* even a
>factor. Beyond the *target*, this technique does not buffer file data,
>so a very *large* file consumes almost no ram in the transaction.
>
>In order to guarantee the structural integrity of your records you have
>to use a shcema checker ahead of time. If you don't care and there's a problem,
>well..... Shucks!
>
>In a programming sense, you will want to validate that integrity,
>otherwise all that is said below is invalid.
>
>Use a schema checker (and have a schema file, this is mandatory) before you
>parse. Then when you parse, you have safeguards.
>If you want detailed instructions on how to install Xerces, let me know.
>Xerces is a robo-lib generated interface to Apache's Xerces.
>A note about schema. Schema is not a guarantee of structural integrity.
>It becomes apparent if you have complex possibilities and schema is only
>general, as in level oriented. I could give examples and solutions if requested.
>
>When you parse, it is you who is parsing the data into records (structures), not
>the parser. You should be in control of start and end of your records.
>
>You have control in your program *when* the eor state is valid. That validity
>will be reached by *you* in the context of *your* program.
>
>Before you initiate parsing, you should validate the file (if you can), something
>like this:
>
>your program..
>
>use XML::Xerces;
>
>if (!ValidateSchema ($Xmlfilename,$SchemaFilename)) {not valid};
>
>sub ValidateSchema {
> my ($xfile, $SchemaFile) = @_;
>
> # Docs: http://xml.apache.org/xerces-c/apiDocs/classAbstractDOMParser.html#z869_9
> my $Xparser = XML::Xerces::XercesDOMParser->new();
> $Xparser->setValidationScheme(1);
> $Xparser->setDoNamespaces(1);
> $Xparser->setDoSchema(1);
> #$Xparser->setValidationSchemaFullChecking(1); # full constraint (if enabled, may be time-consuming)
>
> $Xparser->setExternalNoNamespaceSchemaLocation($SchemaFile);
>
> my $ERROR_HANDLER = XLoggingErrorHandler->new(\&LogX_warn, \&LogX_error, \&LogX_ferror, );
> $Xparser->setErrorHandler($ERROR_HANDLER);
>
> eval {$Xparser->parse (XML::Xerces::LocalFileInputSource->new($xfile));};
> return 1;
>}
>
>Ok, i got a important phone call, so i'll continue this in a little later.
>The important stuff comes on how to easily
>pick off your records and process them in-stream. No problem whatsoever.
>Brb..
------------------------------
Date: Fri, 24 Feb 2006 23:11:37 -0000
From: "UkJay" <jay@cutmeukjay.com>
Subject: Re: sharing variables-data perl-asp
Message-Id: <dto3r4$7pg$1@newsg4.svr.pol.co.uk>
"Matt Garrish" <matthew.garrish@sympatico.ca> wrote in message
news:mYLLf.25636$%14.666814@news20.bellglobal.com...
>
> "UkJay" <jay@cutmeukjay.com> wrote in message
> news:dtlcu0$pbn$1@news6.svr.pol.co.uk...
>>
>> "Todd W" <toddrw69@excite.com> wrote in message
>> news:gHnLf.59889$PL5.1320@newssvr11.news.prodigy.com...
>>>
>>> "UkJay" <jay@cutmeukjay.com> wrote in message
>>> news:dtkhp3$4va$1@news6.svr.pol.co.uk...
>>>> Is it possible to pass data between variables when using active server
>>> pages
>>>> and inserting some perl script in asp?
>>>>
>>>> Also how do you invoke a perl script from an active server page?
>>>>
>>>> Yes I want the best of both worlds ;-)
>>>>
>>>
>>> The "pass data between variables" part I dont understand, but I think
>>> you
>>> are looking for PerlScript. ASP in Perl. Here is a quickstart guide:
>>>
>>> http://www.4guysfromrolla.com/webtech/021100-1.shtml
>>>
>>> trwww
>>>
>>>
>>
>> I'm not looking for perl script
>> I use it
>>
>> I want to use perl script in asp
>> and share data
>>
>
> <%@ Language="PerlScript" %>
>
> You're now using Perlscript in asp.
>
> What exactly is "share data" supposed to mean? You want to store it in
> session: $Session->setVariable('key', 'value') or just write it back to
> hidden input fields in the next page's form? You'll have to elaborate if
> you expect any meaningful help.
>
> Matt
I'll try but this is my prob I'm hopeless at explaining
Ok I use asp which has variables. I then somehow call a perl script from
asp. (not embed it in asp)
This perl script can grab the values from my asp variables, do some work and
then return updated data into my asp variable(s)
I did say I;m no good at explaining Matt
--
Best Regards,
James (ukjay)
http://www.ukjay.co.uk
Garden WebCam,Photography,Competitions,Weather (AWS)
>
------------------------------
Date: Fri, 24 Feb 2006 18:59:19 -0500
From: "Matt Garrish" <matthew.garrish@sympatico.ca>
Subject: Re: sharing variables-data perl-asp
Message-Id: <r5NLf.26070$%14.676402@news20.bellglobal.com>
"UkJay" <jay@cutmeukjay.com> wrote in message
news:dto3r4$7pg$1@newsg4.svr.pol.co.uk...
>
> "Matt Garrish" <matthew.garrish@sympatico.ca> wrote in message
> news:mYLLf.25636$%14.666814@news20.bellglobal.com...
>>
>> "UkJay" <jay@cutmeukjay.com> wrote in message
>> news:dtlcu0$pbn$1@news6.svr.pol.co.uk...
>>>
>>> "Todd W" <toddrw69@excite.com> wrote in message
>>> news:gHnLf.59889$PL5.1320@newssvr11.news.prodigy.com...
>>>>
>>>> "UkJay" <jay@cutmeukjay.com> wrote in message
>>>> news:dtkhp3$4va$1@news6.svr.pol.co.uk...
>>>>> Is it possible to pass data between variables when using active server
>>>> pages
>>>>> and inserting some perl script in asp?
>>>>>
>>>>> Also how do you invoke a perl script from an active server page?
>>>>>
>>>>> Yes I want the best of both worlds ;-)
>>>>>
>>>>
>>>> The "pass data between variables" part I dont understand, but I think
>>>> you
>>>> are looking for PerlScript. ASP in Perl. Here is a quickstart guide:
>>>>
>>>> http://www.4guysfromrolla.com/webtech/021100-1.shtml
>>>>
>>>> trwww
>>>>
>>>>
>>>
>>> I'm not looking for perl script
>>> I use it
>>>
>>> I want to use perl script in asp
>>> and share data
>>>
>>
>> <%@ Language="PerlScript" %>
>>
>> You're now using Perlscript in asp.
>>
>> What exactly is "share data" supposed to mean? You want to store it in
>> session: $Session->setVariable('key', 'value') or just write it back to
>> hidden input fields in the next page's form? You'll have to elaborate if
>> you expect any meaningful help.
>>
>
>
> I'll try but this is my prob I'm hopeless at explaining
>
> Ok I use asp which has variables. I then somehow call a perl script from
> asp. (not embed it in asp)
> This perl script can grab the values from my asp variables, do some work
> and then return updated data into my asp variable(s)
>
Why do you want to shell out to perl when you're already running perl?
PerlScript is Perl; just running within the asp environment. That said,
there's no reason you can't also shell out to another script using backticks
(but you're going to pay the overhead of starting another perl interpreter
each time):
my $returnValues = `perl somescript.pl`;
Note that I called perl explicitly. I never looked into this too deeply on
Windows, but have always just assumed that the Windows extension mappings
are not available in the shell that is created to execute the command. If
you don't call perl, don't expect your script to run.
The data sharing is certainly not impossible, but not terribly nice either.
Options that spring to mind are to print the variables you want back to
stdout as a string in some delimited fashion and then split that string from
your within your asp script. Or you could go the more powerful route and use
the Storable module to write the data structures to file and then send that
filename back to your script to read them back in.
You may find yourself limited in terms of what you can run and do from a
shell, though. You will be running with limited rights no matter what server
you're using.
In other words, reconsider why you want to write a separate script!
Matt
------------------------------
Date: 6 Apr 2001 21:33:47 GMT (Last modified)
From: Perl-Users-Request@ruby.oce.orst.edu (Perl-Users-Digest Admin)
Subject: Digest Administrivia (Last modified: 6 Apr 01)
Message-Id: <null>
Administrivia:
#The Perl-Users Digest is a retransmission of the USENET newsgroup
#comp.lang.perl.misc. For subscription or unsubscription requests, send
#the single line:
#
# subscribe perl-users
#or:
# unsubscribe perl-users
#
#to almanac@ruby.oce.orst.edu.
NOTE: due to the current flood of worm email banging on ruby, the smtp
server on ruby has been shut off until further notice.
To submit articles to comp.lang.perl.announce, send your article to
clpa@perl.com.
#To request back copies (available for a week or so), send your request
#to almanac@ruby.oce.orst.edu with the command "send perl-users x.y",
#where x is the volume number and y is the issue number.
#For other requests pertaining to the digest, send mail to
#perl-users-request@ruby.oce.orst.edu. Do not waste your time or mine
#sending perl questions to the -request address, I don't have time to
#answer them even if I did know the answer.
------------------------------
End of Perl-Users Digest V10 Issue 8992
***************************************
