[26602] in Perl-Users-Digest
Perl-Users Digest, Issue: 8726 Volume: 10
daemon@ATHENA.MIT.EDU (Perl-Users Digest)
Thu Dec 1 03:05:40 2005
Date: Thu, 1 Dec 2005 00:05:05 -0800 (PST)
From: Perl-Users Digest <Perl-Users-Request@ruby.OCE.ORST.EDU>
To: Perl-Users@ruby.OCE.ORST.EDU (Perl-Users Digest)
Perl-Users Digest Thu, 1 Dec 2005 Volume: 10 Number: 8726
Today's topics:
Re: How to tell if a method exists (Malcolm Dew-Jones)
My experience with XML::DOM VS XML::LibXML <jvs@india.ti.com>
Re: My experience with XML::DOM VS XML::LibXML robic0
Re: My experience with XML::DOM VS XML::LibXML robic0
Re: XML::Simple array size robic0
Re: XML::Simple array size robic0
Digest Administrivia (Last modified: 6 Apr 01) (Perl-Users-Digest Admin)
----------------------------------------------------------------------
Date: 30 Nov 2005 14:13:33 -0700
From: yf110@victoria.tc.ca (Malcolm Dew-Jones)
Subject: Re: How to tell if a method exists
Message-Id: <438e240d$1@news.victoria.tc.ca>
Brian Miller (brian.miller@alcatel.com) wrote:
: Hello All,
: I am attempting to find a way to see if a method exists within a
: specific package. I cannot use the PACKAGE->can(method) because
: it follows the class hierarchy and it is possible that the method
: exists somewhere else. What I was hopping to do is to have a base class
: that scans the class hierarchy of an object and executes (in a specific
: order) a method (let's say post_init). This would eliminate the need
: for each method to run through it's package @ISA array and execute the
: parents version of the method. If anyone has a suggestion I would
: greatly appreciate it.
: Thanks
: Brian Miller
: brian.miller@alcatel.com
Localize the appropriate @ISA and then do the lookup.
The syntax of the following should be made prettier, but it illustrates
the point. The output from running the following shows that when
"can_no_isa" is called with the correct package (TWO) then the object
can't. The other package names are just to confirm that the routine is
doing the right thing.
I guess you could also check if the routine was defined in the package
hash but that didn't work in my quick example (!) so I gave up and just
tried the following instead.
#!perl
use strict;
sub can_no_isa
{
my ($pack,$meth,$obj) = @_;
no strict;
local @{"${pack}::ISA"} = ();
return $obj->can($meth);
}
package ONE;
sub try {'ta-dah!'}
package TWO;
our @ISA = qw(ONE);
package main;
my $obj = bless \my $x , 'TWO';
print 'obj can try = ' , $obj->can('try') , "\n";
print 'obj can_no_is try in THR = ' , can_no_isa('THR','try',$obj) , "\n";
print 'obj can_no_is try in TWO = ' , can_no_isa('TWO','try',$obj) , "\n";
print 'obj can_no_is try in ONE = ' , can_no_isa('ONE','try',$obj) , "\n";
------------------------------
Date: Thu, 1 Dec 2005 10:49:38 +0530
From: Jahagirdar Vijayvithal S <jvs@india.ti.com>
Subject: My experience with XML::DOM VS XML::LibXML
Message-Id: <slrndot1va.q1s.jvs@bstclindt008.india.ti.com>
I have a script where I am
1> Opening a pipe to a program which reads in a binary file(400MB) and dumps out XML data(XXX GB's) (tethereal)
2> Grabing chunk's of data within tags <packet>.....</packet> (approx 4
to 20 K)
3> parsing the XML
4> post processing based on fields and attributes in the XML document.
Initially I used XML::DOM and found that my memory consumption
constantly increased filling up the entire RAM and SWAP space before
crashing(approx 32GB RAM and 160+ GB Swap consumed).
Switching to XML:LibXML and replacing the XML::DOM constructs with their
equivalent I find that my worst case Memory consumption remains below
2GB each (RAM and swap) and average is around 50 MB each.
While my problem is solved I am curious to know wether there is any
known Issues which caused the above problems?
code fragment used by me is as below
-----------------------------Code-----------------
use XML::DOM;
my $parser = XML::DOM::Parser->new();
open XML ,"$tethereal -r $pcapfile -T pdml|" or die "Cant open a simple pipe? go smoke one!";
while(<XML>){
#print;
if(my $range=/<packet>/.../<\/packet>/){
if($range==1){
$data="<JVS_PARSER>";
}
$data="$data $_";
if ($range=~/E0/){
$data="$data</JVS_PARSER>";
........ various calls to function getval and Other processingstuff
}
}
}
sub getval(){
my ($data,$name,$attribute)=@_;
return unless defined($data);
my $doc = $parser->parse($data);#Should be parsing this just once outside the function call
foreach my $element ($doc->getElementsByTagName('field')){
if ($element->getAttribute('name') eq $name){
return $element->getAttribute($attribute);
}
}
return -1; #Error
}
--------------------------End Code-------------------------
Regards
Jahagirdar Vijayvithal S
--
------------------------------
Date: Wed, 30 Nov 2005 22:08:17 -0800
From: robic0
Subject: Re: My experience with XML::DOM VS XML::LibXML
Message-Id: <0k3to1dsca7avfqvpo7hjh0ddas4n83acm@4ax.com>
On Thu, 1 Dec 2005 10:49:38 +0530, Jahagirdar Vijayvithal S
<jvs@india.ti.com> wrote:
>I have a script where I am
>1> Opening a pipe to a program which reads in a binary file(400MB) and dumps out XML data(XXX GB's) (tethereal)
>2> Grabing chunk's of data within tags <packet>.....</packet> (approx 4
>to 20 K)
So the data within tags contains the xml you want to parse?
>3> parsing the XML
within the packet tags
>4> post processing based on fields and attributes in the XML document.
which document?
>
>Initially I used XML::DOM and found that my memory consumption
>constantly increased filling up the entire RAM and SWAP space before
>crashing(approx 32GB RAM and 160+ GB Swap consumed).
>Switching to XML:LibXML and replacing the XML::DOM constructs with their
>equivalent I find that my worst case Memory consumption remains below
>2GB each (RAM and swap) and average is around 50 MB each.
>
use SAX, nodes will fall off the ends of the earth
>While my problem is solved I am curious to know wether there is any
>known Issues which caused the above problems?
>
>code fragment used by me is as below
>-----------------------------Code-----------------
>use XML::DOM;
>my $parser = XML::DOM::Parser->new();
>open XML ,"$tethereal -r $pcapfile -T pdml|" or die "Cant open a simple pipe? go smoke one!";
> while(<XML>){
> #print;
> if(my $range=/<packet>/.../<\/packet>/){
^ ^^^
bad Perl? (...) capture?
what would happen if:
<packet>adsfbasdfbabf</packet><packet>adsfbasdfbabf
</packet><packet>adsfbasdfbabf</packet>
<packet>adsfbasdfbabf
</packet>
Thats the funny thing about xml, you can edit it in notepad.
> if($range==1){
will it even get here if $range == 0?
> $data="<JVS_PARSER>";
> }
> $data="$data $_";
what are you grabbing here? i thought you just needed packet data?
> if ($range=~/E0/){
> $data="$data</JVS_PARSER>";
>........ various calls to function getval and Other processingstuff
> }
> }
>}
>
>sub getval(){
sub getval(???){ ## ?
> my ($data,$name,$attribute)=@_;
> return unless defined($data);
> my $doc = $parser->parse($data);#Should be parsing this just once outside the function call
> foreach my $element ($doc->getElementsByTagName('field')){
> if ($element->getAttribute('name') eq $name){
> return $element->getAttribute($attribute);
> }
> }
> return -1; #Error
>}
>
>--------------------------End Code-------------------------
>
>Regards
>Jahagirdar Vijayvithal S
Don't know how you got anything to work on this.
You would be better off if you use SAX to start with.
I know the whole XXX gigabyte attracts attention to you
question but you should give more details.
------------------------------
Date: Wed, 30 Nov 2005 22:23:00 -0800
From: robic0
Subject: Re: My experience with XML::DOM VS XML::LibXML
Message-Id: <2d5to1tcetck00q2mb9u02qljpdgvn93d9@4ax.com>
On Wed, 30 Nov 2005 22:08:17 -0800, robic0 wrote:
>>Initially I used XML::DOM and found that my memory consumption
>>constantly increased filling up the entire RAM and SWAP space before
>>crashing(approx 32GB RAM and 160+ GB Swap consumed).
>>Switching to XML:LibXML and replacing the XML::DOM constructs with their
>>equivalent I find that my worst case Memory consumption remains below
>>2GB each (RAM and swap) and average is around 50 MB each.
>>
>use SAX, nodes will fall off the ends of the earth
>>While my problem is solved I am curious to know wether there is any
>>known Issues which caused the above problems?
>>
These are a good general use, high performance strategy, node
alternative for xml:
use XML::Xerces;
use XML::Parser::Expat;
use XML::Simple;
$XML::Simple::PREFERRED_PARSER = 'XML::Parser';
Once you setup a template for SAX parsing processing,
you can use it on any xml no matter what the structure.
And I'm talking about quad-terrabyte xml file that
you can process with 512 megs of ram.
------------------------------
Date: Wed, 30 Nov 2005 21:36:37 -0800
From: robic0
Subject: Re: XML::Simple array size
Message-Id: <ii2to1p7pmlg6ig14r1ohhl19eq3m4msga@4ax.com>
On Tue, 29 Nov 2005 14:01:02 +0100, "Dave" <daveandniki@ntlworld.com>
wrote:
>
><robic0> wrote in message news:g5uno11r6901vv00tnpln9r35j3q4v3jfc@4ax.com...
>> On Sat, 14 Feb 2004 17:58:08 -0500, Mark J Fenbers
>> <Mark.Fenbers@noaa.gov> wrote:
>>
>>>In my sample XML file...
>>>
>>><main>
>>> <level1>
>>> <level2>whatever</level2>
>>> <level2>whatever</level2>
>>> <level2>whatever</level2>
>>> <level2>whatever</level2>
>>> <level2>whatever</level2>
>>> ...
>>> </level1>
>>></main>
>>>
>>>I have an variable number of level2 tags. Using XML::Simple, I can access
>>>the
>>>value of the 4th one by something like this:
>>>
>>> my $main = XMLin("myfile.xml");
>>> print $main->{level1}->{level2}->[3]->{content};
>>>
>>>But I really want to know how many level2 records there are. I tried
>>>variations
>>>
>> Hey isn't the "$main" name take already?
>> New rule, you have to give more details than "I want to count all
>> animals on Noah's ark". Noah didn't coun't them for a reason....
>> Unless you positively know ahead of time the structure of the xml
>> in question, you can't rely on xml::simple's output structure.
>> I guess thats the other question.....
>
>Answering the OP's question. If there are a variable number <level2>
>my $element_count = @{ $main->{level1}->{level2} };
>should do it (untested).
>If there is only one level2 this will give an error unless you force
>XML::Simple to make it an array.
that should do it
$main=
{
$level1 =>
{
$level2 => [whatever,whatever,whatever,whatever],
},
};
>
------------------------------
Date: Wed, 30 Nov 2005 22:38:40 -0800
From: robic0
Subject: Re: XML::Simple array size
Message-Id: <od6to1tnnorv64hdfpen750dl66oa3ocsm@4ax.com>
On Wed, 30 Nov 2005 21:36:37 -0800, robic0 wrote:
>On Tue, 29 Nov 2005 14:01:02 +0100, "Dave" <daveandniki@ntlworld.com>
>wrote:
>
>>
>><robic0> wrote in message news:g5uno11r6901vv00tnpln9r35j3q4v3jfc@4ax.com...
>>> On Sat, 14 Feb 2004 17:58:08 -0500, Mark J Fenbers
>>> <Mark.Fenbers@noaa.gov> wrote:
>>>
>>>>In my sample XML file...
>>>>
>>>><main>
>>>> <level1>
>>>> <level2>whatever</level2>
>>>> <level2>whatever</level2>
>>>> <level2>whatever</level2>
>>>> <level2>whatever</level2>
>>>> <level2>whatever</level2>
>>>> ...
>>>> </level1>
>>>></main>
>>>>
>>>>I have an variable number of level2 tags. Using XML::Simple, I can access
>>>>the
>>>>value of the 4th one by something like this:
>>>>
>>>> my $main = XMLin("myfile.xml");
>>>> print $main->{level1}->{level2}->[3]->{content};
>>>>
>>>>But I really want to know how many level2 records there are. I tried
>>>>variations
>>>>
>>> Hey isn't the "$main" name take already?
>>> New rule, you have to give more details than "I want to count all
>>> animals on Noah's ark". Noah didn't coun't them for a reason....
>>> Unless you positively know ahead of time the structure of the xml
>>> in question, you can't rely on xml::simple's output structure.
>>> I guess thats the other question.....
>>
>>Answering the OP's question. If there are a variable number <level2>
>>my $element_count = @{ $main->{level1}->{level2} };
>>should do it (untested).
>>If there is only one level2 this will give an error unless you force
>>XML::Simple to make it an array.
$ref = eval { XMLin( $_xml , SuppressEmpty => '') };
# $ref = eval { XMLin( $_xml, ForceArray => 1, SuppressEmpty => '' )
been a while, thought force array had a problem depending on the
usage.
>
>that should do it
>
>$main=
>{
> $level1 =>
> {
> $level2 => [whatever,whatever,whatever,whatever],
> },
>};
>>
------------------------------
Date: 6 Apr 2001 21:33:47 GMT (Last modified)
From: Perl-Users-Request@ruby.oce.orst.edu (Perl-Users-Digest Admin)
Subject: Digest Administrivia (Last modified: 6 Apr 01)
Message-Id: <null>
Administrivia:
#The Perl-Users Digest is a retransmission of the USENET newsgroup
#comp.lang.perl.misc. For subscription or unsubscription requests, send
#the single line:
#
# subscribe perl-users
#or:
# unsubscribe perl-users
#
#to almanac@ruby.oce.orst.edu.
NOTE: due to the current flood of worm email banging on ruby, the smtp
server on ruby has been shut off until further notice.
To submit articles to comp.lang.perl.announce, send your article to
clpa@perl.com.
#To request back copies (available for a week or so), send your request
#to almanac@ruby.oce.orst.edu with the command "send perl-users x.y",
#where x is the volume number and y is the issue number.
#For other requests pertaining to the digest, send mail to
#perl-users-request@ruby.oce.orst.edu. Do not waste your time or mine
#sending perl questions to the -request address, I don't have time to
#answer them even if I did know the answer.
------------------------------
End of Perl-Users Digest V10 Issue 8726
***************************************
