[26614] in Perl-Users-Digest
Perl-Users Digest, Issue: 8726 Volume: 10
daemon@ATHENA.MIT.EDU (Perl-Users Digest)
Sat Dec 3 03:05:16 2005
Date: Sat, 3 Dec 2005 00:05:08 -0800 (PST)
From: Perl-Users Digest <Perl-Users-Request@ruby.OCE.ORST.EDU>
To: Perl-Users@ruby.OCE.ORST.EDU (Perl-Users Digest)
Perl-Users Digest Sat, 3 Dec 2005 Volume: 10 Number: 8726
Today's topics:
Re: A question of hashes in a module. robic0
Re: FAQ 5.34 Why can't I use "C:\temp\foo" in DOS paths <sherm@dot-app.org>
Re: FAQ 5.34 Why can't I use "C:\temp\foo" in DOS paths <sherm@dot-app.org>
IO::Socket help <ves@ves.net>
Re: My experience with XML::DOM VS XML::LibXML robic0
Re: My experience with XML::DOM VS XML::LibXML robic0
Re: My experience with XML::DOM VS XML::LibXML robic0
Re: My experience with XML::DOM VS XML::LibXML robic0
Re: My experience with XML::DOM VS XML::LibXML robic0
Re: My experience with XML::DOM VS XML::LibXML robic0
Re: My experience with XML::DOM VS XML::LibXML robic0
Re: My experience with XML::DOM VS XML::LibXML robic0
Digest Administrivia (Last modified: 6 Apr 01) (Perl-Users-Digest Admin)
----------------------------------------------------------------------
Date: Fri, 02 Dec 2005 23:50:58 -0800
From: robic0
Subject: Re: A question of hashes in a module.
Message-Id: <ogj2p1hf5prbgee3mn9e9trrqejhnbt5e9@4ax.com>
On 3 Dec 2005 04:37:43 GMT, John Bokma <john@castleamber.com> wrote:
>Don Gatlin <invalid@nowhere.com> wrote:
>
>> A couple of questions on a hash inside of a module. Actually I have
>> just started to learn object perl.
>>
>> Here is the module (shortened for clarity). Not a real program, just
>> a test to try to understand it.
>>
>> ###################################### 1 package Initalise;
>> 2
>> 3 sub new {
>> 4 my $class = shift;
>> 5 my $ref = {
>> 6 Hammer => {
>> 7 price => 10,
>> 8 color => 'Blue',
>> 9 weight => 2,
>> 10 },
>> 11 Wrench => {
>> 12 price => 5,
>> 13 color => 'Silver',
>> 14 weight => 1,
>> 15 },
>> 16 };
>> 17 bless($ref, $class);
>> 18 return $ref;
>> 19 #---------------------------
>> 20 sub start_structures {
>> 21 my $self = shift;
>> 22 my $key;
>> 23 my $value;
>> 24 while ( ($key, $value) = each %$self) { 25
>> if ($self{$key}{color} eq 'Silver') { 26
>> #do something 27 }
>> 28 }
>> 29 return $z;
>> 30 }
>> 31 }
>> 32 1;
>> #####################################
>>
>> I have been playing with hashes and hashes of hashes in normal
>> procedural "Hello World" type of programming but this works somewhat
>> differently. In line 24, why is the hash called %$self rather than
>> the usual hash name of %self ?
>
>because $self is a scalar containing a reference to a hash, the % does a
>dereference.
>
>> And in the debugger looking at %self
>> gives just a hash id, but $self gives the whole hash value tree. In
>> an ordinary MAIN program, %hash gives the whole value tree and $hash
>> just gives the values for that one key.
>
>my $hash = ( foo => bar );
>
>my $ref = \%hash;
>
>print "$hash{ foo }\n";
>print "$ref->{ foo }\n";
>
>> What I have in the new{} part is what looks like to me to be a normal
>> hash of hashes. In fact that is what it looks like to my debugger.
>
>It's a *reference* to a hash with references of hashes as values for its
>keys. The name ref is a giveaway, although in order to keep your code
>more consistent I recommend to call that one self as well.
>
>> So, I am not understanding the differences between a hash outside or
>> inside of a module/object and my books so far aren't helping.
>>
>> Insights anyone?
>
>Stop with the OO course, and start with references:
>
>perldoc perlreftut
>
>(hmmm, I knew that one from the top of my head, now that's odd)
>
>
>BTW:
>
>$self{$key}{color} -> $self->{ $key }{ color }
^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^
don't rely on this syntax... what does it mean to you?
>
>while ( my ( $key, $value ) = each %$self ) {
> ^^
> \--- inside the while
>
>}
------------------------------
Date: Sat, 03 Dec 2005 01:03:43 -0500
From: Sherm Pendley <sherm@dot-app.org>
Subject: Re: FAQ 5.34 Why can't I use "C:\temp\foo" in DOS paths? Why doesn't `C:\temp\foo.exe` work?
Message-Id: <m27jamg1j4.fsf@Sherm-Pendleys-Computer.local>
PerlFAQ Server <comdog@pair.com> writes:
> 5.34: Why can't I use "C:\temp\foo" in DOS paths? Why doesn't `C:\temp\foo.exe` work?
>
>
> Whoops! You just put a tab and a formfeed into that filename! Remember
> that within double quoted strings ("like\this"), the backslash is an
> escape character. The full list of these is in "Quote and Quote-like
> Operators" in perlop. Unsurprisingly, you don't have a file called
> "c:(tab)emp(formfeed)oo" or "c:(tab)emp(formfeed)oo.exe" on your legacy
> DOS filesystem.
Isn't it time to update the "DOS paths" and "legacy DOS filesystem" bits?
Windows hasn't been built on top of MS-DOS, or used FAT16, for quite a long
time now.
I understand that this FAQ may have been written some time ago, when MS-DOS
and FAT16 were actually relevant, but now, in 2005, calling it "DOS" just comes
across as an immature jab, IMHO - it puts us in the same group as those who
think it's "kewl" to write "M$ Window$" and "Internet Exploder".
> Either single-quote your strings, or (preferably) use forward slashes.
> Since all DOS and Windows versions since something like MS-DOS 2.0 or so
> have treated "/" and "\" the same in a path, you might as well use the
> one that doesn't clash with Perl--or the POSIX shell, ANSI C and C++,
> awk, Tcl, Java, or Python, just to mention a few. POSIX paths are more
> portable, too.
This need to be clarified. Windows supports the use of "/" in function calls
like open(), stat(), etc. The standard Windows command shell uses "/"s as
argument delimiters, and requires "\"s in paths. Anything that will be
interpreted by the command shell, like single-argument system() or single-
argment piped open()s, must "\"s to avoid errors from the command shell.
sherm--
--
Cocoa programming in Perl: http://camelbones.sourceforge.net
Hire me! My resume: http://www.dot-app.org
------------------------------
Date: Sat, 03 Dec 2005 01:06:26 -0500
From: Sherm Pendley <sherm@dot-app.org>
Subject: Re: FAQ 5.34 Why can't I use "C:\temp\foo" in DOS paths? Why doesn't `C:\temp\foo.exe` work?
Message-Id: <m23blag1el.fsf@Sherm-Pendleys-Computer.local>
PerlFAQ Server <comdog@pair.com> writes:
> 5.34: Why can't I use "C:\temp\foo" in DOS paths? Why doesn't `C:\temp\foo.exe` work?
Oh, and one more kvetch - this question goes all the way out to column 86.
Shouldn't it be wrapped? With all the complaining about usenet conventions
going on here, the FAQs should be following them too... ;-/
sherm--
--
Cocoa programming in Perl: http://camelbones.sourceforge.net
Hire me! My resume: http://www.dot-app.org
------------------------------
Date: Sat, 03 Dec 2005 07:08:45 GMT
From: Vespasian <ves@ves.net>
Subject: IO::Socket help
Message-Id: <f1h2p19288npe26f545s17qs4g8nf3pbh8@4ax.com>
Why does the following code correctly sets up a server
my $port = 9850;
my $proto = getprotobyname('tcp');
socket(SERVER, PF_INET, SOCK_STREAM, $proto) or die "socket:
$!";
setsockopt(SERVER, SOL_SOCKET, SO_REUSEADDR, 1) or die
"setsock: $!";
my $paddr = sockaddr_in($port, INADDR_ANY);
bind(SERVER, $paddr) or die "bind: $!";
listen(SERVER, SOMAXCONN) or die "listen: $!";
print "SERVER started on port $port\n";
# accepting a connection
my $client_addr;
while ($client_addr = accept(CLIENT, SERVER))
while the following does not i.e. it dies at the die statement,
my $port = 9850;
my $proto = getprotobyname('tcp');
my $sock = new IO::Socket::INET(
LocalHost => 'localhost',
LocalPort => 9850,
Proto => 'tcp',
Listen => SOMAXCONN,
Type => SOCK_STREAM,
Reuse => 1);
$sock or die "no socket :$!";
print "SERVER started on port $port\n";
# accepting a connection
my $client_sock = $sock->accept();
TIA,
Ves
------------------------------
Date: Fri, 02 Dec 2005 21:52:18 -0800
From: robic0
Subject: Re: My experience with XML::DOM VS XML::LibXML
Message-Id: <shc2p1hdjconhuv9pg43h1ahp95ddr46b6@4ax.com>
On Thu, 01 Dec 2005 17:59:29 GMT, Ala Qumsieh <notvalid@email.com>
wrote:
>robic0 wrote:
>> On Thu, 1 Dec 2005 10:49:38 +0530, Jahagirdar Vijayvithal S
>> <jvs@india.ti.com> wrote:
>
>[snip]
>
>>> while(<XML>){
>>> #print;
>>> if(my $range=/<packet>/.../<\/packet>/){
>>
>> ^ ^^^
>> bad Perl? (...) capture?
>
>No, this is good Perl. This is not a regexp, but a range operator.
>
>--Ala
Oh you know what, I must be going blind. Let me look at this whole
thing again. Give me a few minutes.
------------------------------
Date: Fri, 02 Dec 2005 22:06:05 -0800
From: robic0
Subject: Re: My experience with XML::DOM VS XML::LibXML
Message-Id: <kbd2p156mc0gqg3pdp64k20eb5nc6uqq9e@4ax.com>
On Thu, 1 Dec 2005 15:58:17 +0530, Jahagirdar Vijayvithal S
<jvs@india.ti.com> wrote:
>* robic0 <robic0> wrote:
>> On Thu, 1 Dec 2005 10:49:38 +0530, Jahagirdar Vijayvithal S
>> <jvs@india.ti.com> wrote:
>>
>>>I have a script where I am
>>>1> Opening a pipe to a program which reads in a binary file(400MB) and dumps out XML data(XXX GB's) (tethereal)
>>>2> Grabing chunk's of data within tags <packet>.....</packet> (approx 4
>>>to 20 K)
>> So the data within tags contains the xml you want to parse?
>Right.
>>>3> parsing the XML
>> within the packet tags
>right again
>>>4> post processing based on fields and attributes in the XML document.
>> which document?
>Oops sorry for the confusion I meant based on elements and attributes
>within the <packet> </packet> tags.
>>>
>>>Initially I used XML::DOM and found that my memory consumption
>>>constantly increased filling up the entire RAM and SWAP space before
>>>crashing(approx 32GB RAM and 160+ GB Swap consumed).
>>>Switching to XML:LibXML and replacing the XML::DOM constructs with their
>>>equivalent I find that my worst case Memory consumption remains below
>>>2GB each (RAM and swap) and average is around 50 MB each.
>one additional observation is that the XML::DOM would crash after
>processing around 6K such packets where as XML::LibXML was able to
>handle more than .6G packets
>since the only thing that changed was the line
>use XML::DOM;
>my $parser = XML::DOM::Parser->new();
>and the function getval();
>
>and the changes were that of replacing these lines by their equivalent
>in XML::LibXML I was wondering what could be the problem. while my
>initial guess is that of some sort of memory leak......
>>>
>> use SAX, nodes will fall off the ends of the earth
>>>While my problem is solved I am curious to know wether there is any
>>>known Issues which caused the above problems?
>>>
>>>code fragment used by me is as below
>>>-----------------------------Code-----------------
>>>use XML::DOM;
>>>my $parser = XML::DOM::Parser->new();
>>>open XML ,"$tethereal -r $pcapfile -T pdml|" or die "Cant open a simple pipe? go smoke one!";
>>> while(<XML>){
>>> #print;
>>> if(my $range=/<packet>/.../<\/packet>/){
>> ^ ^^^
>> bad Perl? (...) capture?
>bad perl: why?
>... => range operator with evaluation of RHS pattern in the next cycle.
>and $range capturing the sequence number.
>> what would happen if:
>> <packet>adsfbasdfbabf</packet><packet>adsfbasdfbabf
>> </packet><packet>adsfbasdfbabf</packet>
>> <packet>adsfbasdfbabf
>> </packet>
>While theoritically possible the tool converting the binary data to XML
>always dumps out one element per line e.g.
><packet>
><proto name="geninfo" pos="0" showname="General information" size="114">
><field name="num" pos="0" show="1" showname="Number" value="1" size="114"/>
><field name="len" pos="0" show="114" showname="Packet Length" value="72" size="114"/>
><field name="caplen" pos="0" show="114" showname="Captured Length" value="72" size="114"/>
><field name="timestamp" pos="0" show="Sep 30, 2005 11:34:22.158787000" showname="Captured Time" value="1128060262.158787000" size="114"/>
></proto>
Quite a few attributes there. Looks good so far. Let me
look it over for a while.
>....
></packet>
>
>> Thats the funny thing about xml, you can edit it in notepad.
>
>>> if($range==1){
>> will it even get here if $range == 0?
>no, refer perldoc perlop for range operator.
>>> $data="<JVS_PARSER>";
>>> }
>>> $data="$data $_";
>> what are you grabbing here? i thought you just needed packet data?
>Thats right. I am reading each line from XML, as long as it falls
>between the <packet></packet> tags appending it to $data to form one
>single scalar with the required data.
>>> if ($range=~/E0/){
>>> $data="$data</JVS_PARSER>";
>>>........ various calls to function getval and Other processingstuff
>>> }
>>> }
>>>}
>>>
>>>sub getval(){
>> sub getval(???){ ## ?
>prototype declared at top of file (not included in code snippet)
>>> my ($data,$name,$attribute)=@_;
>>> return unless defined($data);
>>> my $doc = $parser->parse($data);#Should be parsing this just once outside the function call
>>> foreach my $element ($doc->getElementsByTagName('field')){
>>> if ($element->getAttribute('name') eq $name){
>>> return $element->getAttribute($attribute);
>>> }
>>> }
>>> return -1; #Error
>>>}
>>>
>>>--------------------------End Code-------------------------
>>>
>>>Regards
>>>Jahagirdar Vijayvithal S
>>
>> Don't know how you got anything to work on this.
>I hope the comments above would clarify some of your doubts.
>> You would be better off if you use SAX to start with.
>while I have been following the SAX vs ... debate in other thread. I was
>already familiar with XML::DOM and had a code base using DOM to build
>on. which facilitated a quick workable code which could be used to
>perform the actual processing I am interested in.
>> I know the whole XXX gigabyte attracts attention to you
>> question but you should give more details.
>
>Regards
>Jahagirdar Vijayvithal S
------------------------------
Date: Fri, 02 Dec 2005 22:36:28 -0800
From: robic0
Subject: Re: My experience with XML::DOM VS XML::LibXML
Message-Id: <h2f2p11ieabjve0a2kum3bkfkuk0ag384b@4ax.com>
On Thu, 1 Dec 2005 15:58:17 +0530, Jahagirdar Vijayvithal S
<jvs@india.ti.com> wrote:
>* robic0 <robic0> wrote:
[snip]
>>>-----------------------------Code-----------------
>>>use XML::DOM;
>>>my $parser = XML::DOM::Parser->new();
>>>open XML ,"$tethereal -r $pcapfile -T pdml|" or die "Cant open a simple pipe? go smoke one!";
>>> while(<XML>){
>>> #print;
>>> if(my $range=/<packet>/.../<\/packet>/){
>> ^ ^^^
>> bad Perl? (...) capture?
>bad perl: why?
>... => range operator with evaluation of RHS pattern in the next cycle.
>and $range capturing the sequence number.
Ok, think I got how your using the range operator here (from perlops):
It is false as long as its left operand is false.
Once the left operand is true, the range operator stays true until
the right operand is true, AFTER which the range operator becomes
false again.
[snip]
If you don't want it to test the right operand till the next
evaluation, as in sed,
just use three dots (``...'') instead of two.
- check your other stuff, brb...
------------------------------
Date: Fri, 02 Dec 2005 22:38:22 -0800
From: robic0
Subject: Re: My experience with XML::DOM VS XML::LibXML
Message-Id: <09f2p1955a8r7khaedla9g98k9j38tp9gq@4ax.com>
On Fri, 02 Dec 2005 22:36:28 -0800, robic0 wrote:
>On Thu, 1 Dec 2005 15:58:17 +0530, Jahagirdar Vijayvithal S
><jvs@india.ti.com> wrote:
>
>>* robic0 <robic0> wrote:
>[snip]
>>>>-----------------------------Code-----------------
>>>>use XML::DOM;
>>>>my $parser = XML::DOM::Parser->new();
>>>>open XML ,"$tethereal -r $pcapfile -T pdml|" or die "Cant open a simple pipe? go smoke one!";
>>>> while(<XML>){
>>>> #print;
>>>> if(my $range=/<packet>/.../<\/packet>/){
>>> ^ ^^^
>>> bad Perl? (...) capture?
>>bad perl: why?
>>... => range operator with evaluation of RHS pattern in the next cycle.
>>and $range capturing the sequence number.
>Ok, think I got how your using the range operator here (from perlops):
>
>It is false as long as its left operand is false.
>Once the left operand is true, the range operator stays true until
>the right operand is true, AFTER which the range operator becomes
>false again.
>[snip]
>If you don't want it to test the right operand till the next
>evaluation, as in sed,
>just use three dots (``...'') instead of two.
>
>- check your other stuff, brb...
>
But this is scary. you had better hope there's only one element
per line..... still checking
------------------------------
Date: Fri, 02 Dec 2005 22:43:26 -0800
From: robic0
Subject: Re: My experience with XML::DOM VS XML::LibXML
Message-Id: <ucf2p1t2hjrabkqj6ta0vhr29sv1d4m11h@4ax.com>
On Fri, 02 Dec 2005 22:38:22 -0800, robic0 wrote:
>On Fri, 02 Dec 2005 22:36:28 -0800, robic0 wrote:
>
>>On Thu, 1 Dec 2005 15:58:17 +0530, Jahagirdar Vijayvithal S
>><jvs@india.ti.com> wrote:
>>
>>>* robic0 <robic0> wrote:
>>[snip]
>>>>>-----------------------------Code-----------------
>>>>>use XML::DOM;
>>>>>my $parser = XML::DOM::Parser->new();
>>>>>open XML ,"$tethereal -r $pcapfile -T pdml|" or die "Cant open a simple pipe? go smoke one!";
>>>>> while(<XML>){
>>>>> #print;
>>>>> if(my $range=/<packet>/.../<\/packet>/){
>>>> ^ ^^^
>>>> bad Perl? (...) capture?
>>>bad perl: why?
>>>... => range operator with evaluation of RHS pattern in the next cycle.
>>>and $range capturing the sequence number.
>>Ok, think I got how your using the range operator here (from perlops):
>>
>>It is false as long as its left operand is false.
>>Once the left operand is true, the range operator stays true until
>>the right operand is true, AFTER which the range operator becomes
>>false again.
>>[snip]
>>If you don't want it to test the right operand till the next
>>evaluation, as in sed,
>>just use three dots (``...'') instead of two.
>>
>>- check your other stuff, brb...
>>
>But this is scary. you had better hope there's only one element
>per line..... still checking
Forgot to include this comment last time:
In SAX you don't have to preprocess like your doing here.
You could start parsing without doing this, even though your
looking for inner xml structures. Thats not necessary with SAX.
But using dom, thats the way you have to do it, to reduce the
memory consumption by taking the "whole" into consideration.
....... more to come.
------------------------------
Date: Fri, 02 Dec 2005 23:13:15 -0800
From: robic0
Subject: Re: My experience with XML::DOM VS XML::LibXML
Message-Id: <cnf2p15427279p2um7rk9cph49ei7vsj53@4ax.com>
On Fri, 02 Dec 2005 22:43:26 -0800, robic0 wrote:
>On Fri, 02 Dec 2005 22:38:22 -0800, robic0 wrote:
>
>>On Fri, 02 Dec 2005 22:36:28 -0800, robic0 wrote:
>>
>>>On Thu, 1 Dec 2005 15:58:17 +0530, Jahagirdar Vijayvithal S
>>><jvs@india.ti.com> wrote:
>>>
>>>>* robic0 <robic0> wrote:
>>>[snip]
>>>>>>-----------------------------Code-----------------
>>>>>>use XML::DOM;
>>>>>>my $parser = XML::DOM::Parser->new();
>>>>>>open XML ,"$tethereal -r $pcapfile -T pdml|" or die "Cant open a simple pipe? go smoke one!";
>>>>>> while(<XML>){
>>>>>> #print;
>>>>>> if(my $range=/<packet>/.../<\/packet>/){
>>>>> ^ ^^^
>>>>> bad Perl? (...) capture?
>>>>bad perl: why?
>>>>... => range operator with evaluation of RHS pattern in the next cycle.
>>>>and $range capturing the sequence number.
>>>Ok, think I got how your using the range operator here (from perlops):
>>>
>>>It is false as long as its left operand is false.
>>>Once the left operand is true, the range operator stays true until
>>>the right operand is true, AFTER which the range operator becomes
>>>false again.
>>>[snip]
>>>If you don't want it to test the right operand till the next
>>>evaluation, as in sed,
>>>just use three dots (``...'') instead of two.
>>>
>>>- check your other stuff, brb...
>>>
>>But this is scary. you had better hope there's only one element
>>per line..... still checking
>
>Forgot to include this comment last time:
>In SAX you don't have to preprocess like your doing here.
>You could start parsing without doing this, even though your
>looking for inner xml structures. Thats not necessary with SAX.
>But using dom, thats the way you have to do it, to reduce the
>memory consumption by taking the "whole" into consideration.
>....... more to come.
Again forgot. Let me give you an example. You have a 4 gig xml
file. You only want to extract data in random, 4k slices xml.
What you are trying to do with preprocessing here (to reduce)
the dom overhead is done automatically with SAX stream
processing. You just set the handlers to filter the data you
want to an array of structures or database. If in your case
the aggregate xml is extremely large, and your only interrested
in a few blocks of xml (which itself is very large) only SAX
can do that. You set up simple handlers (callbacks) for start
tag, end tag, content (and by the way for every other w3c
constuct). Glean you tag data.
In your case the data your interrested in has a container tag
"<packet>" (I won't go into the case of unclosed nested tags)
which spans a series of tags:
<packet>
<proto name="geninfo" pos="0" showname="General information"
size="114">
<field name="num" pos="0" show="1" showname="Number" value="1"
size="114"/>
<field name="len" pos="0" show="114" showname="Packet Length"
value="72" size="114"/>
<field name="caplen" pos="0" show="114" showname="Captured Length"
value="72" size="114"/>
<field name="timestamp" pos="0" show="Sep 30, 2005 11:34:22.158787000"
showname="Captured Time" value="1128060262.158787000" size="114"/>
</proto>
....
</packet>
Once the "<packet>" start tag is flagged, you can start collecting
tag data (attributes and content) into a structure that can be
serialized out to a database. You do this until the end tag
"</packet>" is reached.
SAX is event driven, totally serialized. Its so fast you can't
believe it.
In fact your example is so simple I could write the data structures
and the custum part of the template model in less than a day.
Including creating database records. The only limit would be
on the amount of hard drive space.
You have to learn SAX, especially on such a large model. The rudiments
of the event handlers can be for the most part be templated for all
data models you want to extract.
If you want a good template, or will be doing more of this.
I could provide it to you on a contract basis. I've done alot of
SAX and know the ins and outs extremely well.
With SAX you have granularity down to the & level.
Let me know what you decide.
Hope this helps....
gluck
dr
------------------------------
Date: Fri, 02 Dec 2005 23:26:54 -0800
From: robic0
Subject: Re: My experience with XML::DOM VS XML::LibXML
Message-Id: <qkh2p1dlvqkel3ueq4t60ec5d7hiqg8n35@4ax.com>
On Fri, 02 Dec 2005 23:13:15 -0800, robic0 wrote:
>On Fri, 02 Dec 2005 22:43:26 -0800, robic0 wrote:
>
>>On Fri, 02 Dec 2005 22:38:22 -0800, robic0 wrote:
>>
>>>On Fri, 02 Dec 2005 22:36:28 -0800, robic0 wrote:
>>>
>>>>On Thu, 1 Dec 2005 15:58:17 +0530, Jahagirdar Vijayvithal S
>>>><jvs@india.ti.com> wrote:
>>>>
>>>>>* robic0 <robic0> wrote:
>>>>[snip]
>>>>>>>-----------------------------Code-----------------
>>>>>>>use XML::DOM;
>>>>>>>my $parser = XML::DOM::Parser->new();
>>>>>>>open XML ,"$tethereal -r $pcapfile -T pdml|" or die "Cant open a simple pipe? go smoke one!";
>>>>>>> while(<XML>){
>>>>>>> #print;
>>>>>>> if(my $range=/<packet>/.../<\/packet>/){
>>>>>> ^ ^^^
>>>>>> bad Perl? (...) capture?
>>>>>bad perl: why?
>>>>>... => range operator with evaluation of RHS pattern in the next cycle.
>>>>>and $range capturing the sequence number.
>>>>Ok, think I got how your using the range operator here (from perlops):
>>>>
>>>>It is false as long as its left operand is false.
>>>>Once the left operand is true, the range operator stays true until
>>>>the right operand is true, AFTER which the range operator becomes
>>>>false again.
>>>>[snip]
>>>>If you don't want it to test the right operand till the next
>>>>evaluation, as in sed,
>>>>just use three dots (``...'') instead of two.
>>>>
>>>>- check your other stuff, brb...
>>>>
>>>But this is scary. you had better hope there's only one element
>>>per line..... still checking
>>
>>Forgot to include this comment last time:
>>In SAX you don't have to preprocess like your doing here.
>>You could start parsing without doing this, even though your
>>looking for inner xml structures. Thats not necessary with SAX.
>>But using dom, thats the way you have to do it, to reduce the
>>memory consumption by taking the "whole" into consideration.
>>....... more to come.
>Again forgot. Let me give you an example. You have a 4 gig xml
>file. You only want to extract data in random, 4k slices xml.
>What you are trying to do with preprocessing here (to reduce)
>the dom overhead is done automatically with SAX stream
>processing. You just set the handlers to filter the data you
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Actually a clarification here. You can set any handler you want.
To that end, when you have flagged that you are in the right
location, you can set custom handlers (your subs) to process
new incoming tags, etc... The ways to do this programatically
could include an array of custom handlers. Its very simple
and in actuality there is no reason not to have content
wrap. In other words. There is NO way to fool good code at all.
Its about as simple as it can be.... Let me know -gluck
>want to an array of structures or database. If in your case
>the aggregate xml is extremely large, and your only interrested
>in a few blocks of xml (which itself is very large) only SAX
>can do that. You set up simple handlers (callbacks) for start
>tag, end tag, content (and by the way for every other w3c
>constuct). Glean you tag data.
>
>In your case the data your interrested in has a container tag
>"<packet>" (I won't go into the case of unclosed nested tags)
>which spans a series of tags:
>
><packet>
><proto name="geninfo" pos="0" showname="General information"
>size="114">
><field name="num" pos="0" show="1" showname="Number" value="1"
>size="114"/>
><field name="len" pos="0" show="114" showname="Packet Length"
>value="72" size="114"/>
><field name="caplen" pos="0" show="114" showname="Captured Length"
>value="72" size="114"/>
><field name="timestamp" pos="0" show="Sep 30, 2005 11:34:22.158787000"
>showname="Captured Time" value="1128060262.158787000" size="114"/>
></proto>
>....
></packet>
>
>Once the "<packet>" start tag is flagged, you can start collecting
>tag data (attributes and content) into a structure that can be
>serialized out to a database. You do this until the end tag
>"</packet>" is reached.
>
>SAX is event driven, totally serialized. Its so fast you can't
>believe it.
>In fact your example is so simple I could write the data structures
>and the custum part of the template model in less than a day.
>Including creating database records. The only limit would be
>on the amount of hard drive space.
>
>You have to learn SAX, especially on such a large model. The rudiments
>of the event handlers can be for the most part be templated for all
>data models you want to extract.
>
>If you want a good template, or will be doing more of this.
>I could provide it to you on a contract basis. I've done alot of
>SAX and know the ins and outs extremely well.
>
>With SAX you have granularity down to the & level.
>Let me know what you decide.
>
>Hope this helps....
>gluck
>
>dr
>
------------------------------
Date: Fri, 02 Dec 2005 23:30:42 -0800
From: robic0
Subject: Re: My experience with XML::DOM VS XML::LibXML
Message-Id: <09i2p1t6vs7f1c3d5a7hcsckm7ebbia4is@4ax.com>
On Fri, 02 Dec 2005 23:26:54 -0800, robic0 wrote:
>On Fri, 02 Dec 2005 23:13:15 -0800, robic0 wrote:
>
>>On Fri, 02 Dec 2005 22:43:26 -0800, robic0 wrote:
>>
>>>On Fri, 02 Dec 2005 22:38:22 -0800, robic0 wrote:
>>>
>>>>On Fri, 02 Dec 2005 22:36:28 -0800, robic0 wrote:
>>>>
>>>>>On Thu, 1 Dec 2005 15:58:17 +0530, Jahagirdar Vijayvithal S
>>>>><jvs@india.ti.com> wrote:
>>>>>
>>>>>>* robic0 <robic0> wrote:
>>>>>[snip]
>>>>>>>>-----------------------------Code-----------------
>>>>>>>>use XML::DOM;
>>>>>>>>my $parser = XML::DOM::Parser->new();
>>>>>>>>open XML ,"$tethereal -r $pcapfile -T pdml|" or die "Cant open a simple pipe? go smoke one!";
>>>>>>>> while(<XML>){
>>>>>>>> #print;
>>>>>>>> if(my $range=/<packet>/.../<\/packet>/){
>>>>>>> ^ ^^^
>>>>>>> bad Perl? (...) capture?
>>>>>>bad perl: why?
>>>>>>... => range operator with evaluation of RHS pattern in the next cycle.
>>>>>>and $range capturing the sequence number.
>>>>>Ok, think I got how your using the range operator here (from perlops):
>>>>>
>>>>>It is false as long as its left operand is false.
>>>>>Once the left operand is true, the range operator stays true until
>>>>>the right operand is true, AFTER which the range operator becomes
>>>>>false again.
>>>>>[snip]
>>>>>If you don't want it to test the right operand till the next
>>>>>evaluation, as in sed,
>>>>>just use three dots (``...'') instead of two.
>>>>>
>>>>>- check your other stuff, brb...
>>>>>
>>>>But this is scary. you had better hope there's only one element
>>>>per line..... still checking
>>>
>>>Forgot to include this comment last time:
>>>In SAX you don't have to preprocess like your doing here.
>>>You could start parsing without doing this, even though your
>>>looking for inner xml structures. Thats not necessary with SAX.
>>>But using dom, thats the way you have to do it, to reduce the
>>>memory consumption by taking the "whole" into consideration.
>>>....... more to come.
>>Again forgot. Let me give you an example. You have a 4 gig xml
>>file. You only want to extract data in random, 4k slices xml.
>>What you are trying to do with preprocessing here (to reduce)
>>the dom overhead is done automatically with SAX stream
>>processing. You just set the handlers to filter the data you
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>Actually a clarification here. You can set any handler you want.
>To that end, when you have flagged that you are in the right
>location, you can set custom handlers (your subs) to process
>new incoming tags, etc... The ways to do this programatically
>could include an array of custom handlers. Its very simple
>and in actuality there is no reason not to have content
>wrap. In other words. There is NO way to fool good code at all.
>Its about as simple as it can be.... Let me know -gluck
>
>>want to an array of structures or database. If in your case
>>the aggregate xml is extremely large, and your only interrested
>>in a few blocks of xml (which itself is very large) only SAX
>>can do that. You set up simple handlers (callbacks) for start
>>tag, end tag, content (and by the way for every other w3c
>>constuct). Glean you tag data.
>>
>>In your case the data your interrested in has a container tag
>>"<packet>" (I won't go into the case of unclosed nested tags)
>>which spans a series of tags:
>>
>><packet>
>><proto name="geninfo" pos="0" showname="General information"
>>size="114">
>><field name="num" pos="0" show="1" showname="Number" value="1"
>>size="114"/>
>><field name="len" pos="0" show="114" showname="Packet Length"
>>value="72" size="114"/>
>><field name="caplen" pos="0" show="114" showname="Captured Length"
>>value="72" size="114"/>
>><field name="timestamp" pos="0" show="Sep 30, 2005 11:34:22.158787000"
>>showname="Captured Time" value="1128060262.158787000" size="114"/>
>></proto>
>>....
>></packet>
>>
>>Once the "<packet>" start tag is flagged, you can start collecting
>>tag data (attributes and content) into a structure that can be
>>serialized out to a database. You do this until the end tag
>>"</packet>" is reached.
>>
>>SAX is event driven, totally serialized. Its so fast you can't
>>believe it.
>>In fact your example is so simple I could write the data structures
>>and the custum part of the template model in less than a day.
>>Including creating database records. The only limit would be
>>on the amount of hard drive space.
>>
>>You have to learn SAX, especially on such a large model. The rudiments
>>of the event handlers can be for the most part be templated for all
>>data models you want to extract.
>>
>>If you want a good template, or will be doing more of this.
>>I could provide it to you on a contract basis. I've done alot of
>>SAX and know the ins and outs extremely well.
>>
>>With SAX you have granularity down to the & level.
>>Let me know what you decide.
>>
>>Hope this helps....
>>gluck
>>
>>dr
>>
If you want me to write the core or whole of you project, I can do
that too. But I know you guys are programmers. Just could use a good
master to work from...
------------------------------
Date: 6 Apr 2001 21:33:47 GMT (Last modified)
From: Perl-Users-Request@ruby.oce.orst.edu (Perl-Users-Digest Admin)
Subject: Digest Administrivia (Last modified: 6 Apr 01)
Message-Id: <null>
Administrivia:
#The Perl-Users Digest is a retransmission of the USENET newsgroup
#comp.lang.perl.misc. For subscription or unsubscription requests, send
#the single line:
#
# subscribe perl-users
#or:
# unsubscribe perl-users
#
#to almanac@ruby.oce.orst.edu.
NOTE: due to the current flood of worm email banging on ruby, the smtp
server on ruby has been shut off until further notice.
To submit articles to comp.lang.perl.announce, send your article to
clpa@perl.com.
#To request back copies (available for a week or so), send your request
#to almanac@ruby.oce.orst.edu with the command "send perl-users x.y",
#where x is the volume number and y is the issue number.
#For other requests pertaining to the digest, send mail to
#perl-users-request@ruby.oce.orst.edu. Do not waste your time or mine
#sending perl questions to the -request address, I don't have time to
#answer them even if I did know the answer.
------------------------------
End of Perl-Users Digest V10 Issue 8726
***************************************
