[31141] in Perl-Users-Digest
Perl-Users Digest, Issue: 2386 Volume: 11
daemon@ATHENA.MIT.EDU (Perl-Users Digest)
Sat May 2 18:09:41 2009
Date: Sat, 2 May 2009 15:09:05 -0700 (PDT)
From: Perl-Users Digest <Perl-Users-Request@ruby.OCE.ORST.EDU>
To: Perl-Users@ruby.OCE.ORST.EDU (Perl-Users Digest)
Perl-Users Digest Sat, 2 May 2009 Volume: 11 Number: 2386
Today's topics:
Re: "Perl has been pretty much forgotten" says Spolsky <brian.d.foy@gmail.com>
new CPAN modules on Sat May 2 2009 (Randal Schwartz)
parsing packets and user defined data structures <bergenpeak@comcast.net>
Re: parsing packets and user defined data structures <smallpond@juno.com>
read must its opportunity for u daniswati007@gmail.com
read must its opportunity for u daniswati007@gmail.com
Reading from pipe or from stdin, how? <DaLoveRhino@hotmail.com>
Re: Reading from pipe or from stdin, how? <jimsgibson@gmail.com>
Re: RSS XML Streams sln@netherlands.com
Digest Administrivia (Last modified: 6 Apr 01) (Perl-Users-Digest Admin)
----------------------------------------------------------------------
Date: Sat, 02 May 2009 15:07:17 -0500
From: brian d foy <brian.d.foy@gmail.com>
Subject: Re: "Perl has been pretty much forgotten" says Spolsky
Message-Id: <020520091507170516%brian.d.foy@gmail.com>
In article
<2c1ddc85-7d2c-440a-94e8-8d2a4e073375@d39g2000pra.googlegroups.com>,
Ben Bullock <benkasminbullock@gmail.com> wrote:
> In case anyone's interested, this is Joel Spolsky of stackoverflow.com
> on Perl:
>
> http://www.youtube.com/watch?v=NWHfY_lvKIQ
To be fair, Joel is talking about starting a rewrite project from
scratch, and was mostly talked about how the Netscape to Mozilla
translation almost killed the project. The Perl 6 comment was something
he didn't dwell on, wasn't false, and we would do well to recognize as
history.
Despite what you think of Perl, how much you love it, or how much you
want Perl 6 to succeed, the community botched Perl 6 between 2001 and
2005. There was a lot of initial excitement in 2000 and 2001, and when
everyone completely missed the completely naïve goals set out in 2000,
a lot of people did stop paying attention. Audrey Tang revitalized it
in 2005 with an implementation on Haskell, and since then things have
been going very nicely. As I tell people, serious development began in
2005. That's embarrassing, but it's the truth.
------------------------------
Date: Sat, 2 May 2009 04:42:28 GMT
From: merlyn@stonehenge.com (Randal Schwartz)
Subject: new CPAN modules on Sat May 2 2009
Message-Id: <KJ02Es.1FJz@zorch.sf-bay.org>
The following modules have recently been added to or updated in the
Comprehensive Perl Archive Network (CPAN). You can install them using the
instructions in the 'perlmodinstall' page included with your Perl
distribution.
Cache-FastMmap-1.29
http://search.cpan.org/~robm/Cache-FastMmap-1.29/
Uses an mmap'ed file to act as a shared memory interprocess cache
----
Cache-FastMmap-WithWin32-1.28.1
http://search.cpan.org/~ash/Cache-FastMmap-WithWin32-1.28.1/
Uses an mmap'ed file to act as a shared memory interprocess cache
----
Catalyst-Model-PayPal-IPN-0.02
http://search.cpan.org/~rkitover/Catalyst-Model-PayPal-IPN-0.02/
Handle Instant Payment Notifications and PayPal Button Generation
----
Catalyst-Plugin-Session-Store-FastMmap-0.08
http://search.cpan.org/~bobtfish/Catalyst-Plugin-Session-Store-FastMmap-0.08/
FastMmap session storage backend.
----
CatalystX-RoleApplicator-0.005
http://search.cpan.org/~hdp/CatalystX-RoleApplicator-0.005/
apply roles to your Catalyst application-related classes
----
Class-Accessor-Fast-XS-0.03
http://search.cpan.org/~ruz/Class-Accessor-Fast-XS-0.03/
XS replacement for Class::Accessor::Fast
----
Elive-0.12
http://search.cpan.org/~warringd/Elive-0.12/
Elluminate Live (c) client library
----
HTML-FormHandler-0.22
http://search.cpan.org/~gshank/HTML-FormHandler-0.22/
form handler written in Moose
----
HTML-Template-Convert-TT-0.02
http://search.cpan.org/~mbait/HTML-Template-Convert-TT-0.02/
Perl extension for blah blah blah
----
HTML-Template-Convert-TT-0.0201
http://search.cpan.org/~mbait/HTML-Template-Convert-TT-0.0201/
----
HTML-Template-Convert-TT-0.0202
http://search.cpan.org/~mbait/HTML-Template-Convert-TT-0.0202/
----
HTTP-Engine-Middleware-0.12
http://search.cpan.org/~yappo/HTTP-Engine-Middleware-0.12/
middlewares distribution
----
Lexical-Types-0.06
http://search.cpan.org/~vpit/Lexical-Types-0.06/
Extend the semantics of typed lexicals.
----
Lingua-Jspell-1.62
http://search.cpan.org/~ambs/Lingua-Jspell-1.62/
Perl interface to the Jspell morphological analyser.
----
Mail-ClamAV-0.29_1
http://search.cpan.org/~converter/Mail-ClamAV-0.29_1/
Perl extension for the clamav virus scanner
----
Mail-ClamAV-0.30_1
http://search.cpan.org/~converter/Mail-ClamAV-0.30_1/
Perl extension for the clamav virus scanner
----
Mail-IMAPClient-3.17_02
http://search.cpan.org/~plobbes/Mail-IMAPClient-3.17_02/
An IMAP Client API
----
Module-Install-0.86
http://search.cpan.org/~adamk/Module-Install-0.86/
Standalone, extensible Perl module installer
----
Module-Install-AutoLicense-0.02
http://search.cpan.org/~bingos/Module-Install-AutoLicense-0.02/
A Module::Install extension to automagically generate LICENSE files
----
MooseX-Declare-0.20
http://search.cpan.org/~flora/MooseX-Declare-0.20/
Declarative syntax for Moose
----
MooseX-Method-Signatures-0.16
http://search.cpan.org/~flora/MooseX-Method-Signatures-0.16/
Method declarations with type constraints and no source filter
----
MooseX-Role-XMLRPC-Client-0.04
http://search.cpan.org/~rsrchboy/MooseX-Role-XMLRPC-Client-0.04/
provide the needed bits to be a XML-RPC client
----
MooseX-Types-Structured-0.14
http://search.cpan.org/~flora/MooseX-Types-Structured-0.14/
Structured Type Constraints for Moose
----
MooseX-YAML-0.03
http://search.cpan.org/~nuffin/MooseX-YAML-0.03/
DWIM loading of Moose objects from YAML
----
MySQL-Sandbox-2.0.99c
http://search.cpan.org/~gmax/MySQL-Sandbox-2.0.99c/
Quickly installs MySQL side server, either standalone or in groups
----
Nagios-Plugin-OverHTTP-0.01
http://search.cpan.org/~dougdude/Nagios-Plugin-OverHTTP-0.01/
Nagios plugin to check over the HTTP protocol.
----
Nagios-Plugin-OverHTTP-0.02
http://search.cpan.org/~dougdude/Nagios-Plugin-OverHTTP-0.02/
Nagios plugin to check over the HTTP protocol.
----
Net-GitHub-0.12
http://search.cpan.org/~fayland/Net-GitHub-0.12/
Perl Interface for github.com
----
Net-Redmine-0.01
http://search.cpan.org/~gugod/Net-Redmine-0.01/
A mechanized-based programming API against redmine server.
----
OpenResty-0.5.4
http://search.cpan.org/~agent/OpenResty-0.5.4/
General-purpose web service platform for web applications
----
OpenResty-0.5.5
http://search.cpan.org/~agent/OpenResty-0.5.5/
General-purpose web service platform for web applications
----
POE-Component-IKC-0.2100
http://search.cpan.org/~gwyn/POE-Component-IKC-0.2100/
POE Inter-Kernel Communication
----
POE-Component-IKC-0.2101
http://search.cpan.org/~gwyn/POE-Component-IKC-0.2101/
POE Inter-Kernel Communication
----
POE-Component-SNMP-Session-0.1201
http://search.cpan.org/~rdb/POE-Component-SNMP-Session-0.1201/
Wrap Net-SNMP's SNMP::Session in POE
----
POE-Component-WWW-Shorten-1.18
http://search.cpan.org/~bingos/POE-Component-WWW-Shorten-1.18/
A non-blocking POE wrapper around WWW::Shorten.
----
POE-Filter-IRCD-2.40
http://search.cpan.org/~bingos/POE-Filter-IRCD-2.40/
A POE-based parser for the IRC protocol.
----
RDF-Simple-0.412
http://search.cpan.org/~mthurn/RDF-Simple-0.412/
read and write RDF without complication
----
Rinchi-CIGIPP-0.02
http://search.cpan.org/~bmames/Rinchi-CIGIPP-0.02/
Perl extension for the Common Image Generator Interface.
----
Rose-DBx-Object-Renderer-0.47
http://search.cpan.org/~danny/Rose-DBx-Object-Renderer-0.47/
Web UI Rendering for Rose::DB::Object
----
Sphinx-Search-0.20
http://search.cpan.org/~jjschutz/Sphinx-Search-0.20/
Sphinx search engine API Perl client
----
Syntax-Highlight-Perl6-0.49
http://search.cpan.org/~azawawi/Syntax-Highlight-Perl6-0.49/
Perl 6 Syntax Highlighter
----
Template-Plugin-LinkTo-0.06
http://search.cpan.org/~hirafoo/Template-Plugin-LinkTo-0.06/
like link_to in Ruby on Rails
----
Test-Continuous-0.64
http://search.cpan.org/~gugod/Test-Continuous-0.64/
Run your tests suite continusouly when developing.
----
Text-CSV-Encoded-0.08
http://search.cpan.org/~makamaka/Text-CSV-Encoded-0.08/
Encoding aware Text::CSV.
----
UMLS-Similarity-0.17
http://search.cpan.org/~btmcinnes/UMLS-Similarity-0.17/
A suite of Perl modules that implement a number of semantic similarity measures. The measures use the UMLS-Interface module to access the UMLS to generate similarity scores between concepts.
----
URI-git-0.01
http://search.cpan.org/~miyagawa/URI-git-0.01/
git URI scheme
----
Verilog-Perl-3.202
http://search.cpan.org/~wsnyder/Verilog-Perl-3.202/
----
WebService-Cath-FuncNet-0.07
http://search.cpan.org/~isillitoe/WebService-Cath-FuncNet-0.07/
Interface to the CATH FuncNet webservice
----
XML-Atom-0.35
http://search.cpan.org/~miyagawa/XML-Atom-0.35/
Atom feed and API implementation
----
Xtract-0.07
http://search.cpan.org/~adamk/Xtract-0.07/
Take any data source and deliver it to the world
----
namespace-autoclean-0.05
http://search.cpan.org/~flora/namespace-autoclean-0.05/
Keep imports out of your namespace
If you're an author of one of these modules, please submit a detailed
announcement to comp.lang.perl.announce, and we'll pass it along.
This message was generated by a Perl program described in my Linux
Magazine column, which can be found on-line (along with more than
200 other freely available past column articles) at
http://www.stonehenge.com/merlyn/LinuxMag/col82.html
print "Just another Perl hacker," # the original
--
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<merlyn@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
Smalltalk/Perl/Unix consulting, Technical writing, Comedy, etc. etc.
See http://methodsandmessages.vox.com/ for Smalltalk and Seaside discussion
------------------------------
Date: Sat, 2 May 2009 05:47:57 -0700 (PDT)
From: bergenpeak <bergenpeak@comcast.net>
Subject: parsing packets and user defined data structures
Message-Id: <92343d3b-125b-4f22-80a6-e052da6a2f09@e23g2000vbe.googlegroups.com>
I'm using the perl pcap module to read in captured pcap files. I
would like to write code to parse certain packet types. In the past
when using C to do this type of processing, I would define a structure
that details the format of the underlying data, and then overlay the
structure at the right byte offset in the packet and then pull out the
fields in the structure (p->len, p->data, etc.).
Is there a similar mechanism in perl? Right now, I'm doing this by
manually walking through the packet with perl code per field and
underlying data type. It's doable, but time consuming and prone to
errors. I'm hoping there's a cleaner way to do this by overlaying a
structure and pulling the data out through the structure.
Note that the packets I'm interesting in parsing don't have any
predefined perl modules to help with the parsing (eg IP, UDP, etc.)
Thanks
------------------------------
Date: Sat, 2 May 2009 06:57:19 -0700 (PDT)
From: smallpond <smallpond@juno.com>
Subject: Re: parsing packets and user defined data structures
Message-Id: <64d5f4b2-06c0-43e0-86cf-3b595b211bb5@s21g2000vbb.googlegroups.com>
On May 2, 8:47=A0am, bergenpeak <bergenp...@comcast.net> wrote:
> I'm using the perl pcap module to read in captured pcap files. =A0I
> would like to write code to parse certain packet types. =A0In the past
> when using C to do this type of processing, I would define a structure
> that details the format of the underlying data, and then overlay the
> structure at the right byte offset in the packet and then pull out the
> fields in the structure (p->len, p->data, etc.).
>
> Is there a similar mechanism in perl? =A0Right now, I'm doing this by
> manually walking through the packet with perl code per field and
> underlying data type. =A0It's doable, but time consuming and prone to
> errors. =A0I'm hoping there's a cleaner way to do this by overlaying a
> structure and pulling the data out through the structure.
>
> Note that the packets I'm interesting in parsing don't have any
> predefined perl modules to help with the parsing (eg IP, UDP, etc.)
>
> Thanks
perl calls it a template.
binmode FH;
read FH,$packet,$recordlen or die "Cannot read";
($len, $data, $etc) =3D unpack 'l2A3',$packet;
reads two 4-byte integers and 3 ASCII chars such as "ETC".
------------------------------
Date: Sat, 2 May 2009 04:19:42 -0700 (PDT)
From: daniswati007@gmail.com
Subject: read must its opportunity for u
Message-Id: <54801355-d2ef-4974-8728-2d048ec501c9@z23g2000prd.googlegroups.com>
free study in uk and free visa more details
visit www.googleloves.tk
------------------------------
Date: Sat, 2 May 2009 04:19:53 -0700 (PDT)
From: daniswati007@gmail.com
Subject: read must its opportunity for u
Message-Id: <99bd09ee-78a6-4cdc-bd57-0c70bf70732f@b7g2000pre.googlegroups.com>
free study in uk and free visa more details
visit www.googleloves.tk
------------------------------
Date: Fri, 1 May 2009 15:52:49 -0700 (PDT)
From: DaLoverhino <DaLoveRhino@hotmail.com>
Subject: Reading from pipe or from stdin, how?
Message-Id: <2067c26a-a7ae-44b3-abea-37701147e0e9@r34g2000vbi.googlegroups.com>
Hello. I have a program where you specify the name of the file you
want to read from command line. But, I want to modify it so that if
no file is specified, it reads from stdin. so here's a code snippet:
my $file = "";
# Parse command line to get file option.
open( FILE, "<", $file) or die "Could not open $file\n";
my $line;
while( ($file && $line = <FILE> ) ||
(!$file && $line = <STDIN>)) {
do_big_loop_here;
}
This doesn't seem to work:
Unix> echo "blah" | ./pxmlgrep
Value of <HANDLE> construct can be "0"; test with defined() at ./
pxmlgrep line 233.
Can't modify logical and (&&) in scalar assignment at ./pxmlgrep line
234, near "<FILE> )
"
Execution of ./pxmlgrep aborted due to compilation errors.
Any suggestions?
thanks.
------------------------------
Date: Fri, 01 May 2009 16:16:32 -0700
From: Jim Gibson <jimsgibson@gmail.com>
Subject: Re: Reading from pipe or from stdin, how?
Message-Id: <010520091616324389%jimsgibson@gmail.com>
In article
<2067c26a-a7ae-44b3-abea-37701147e0e9@r34g2000vbi.googlegroups.com>,
DaLoverhino <DaLoveRhino@hotmail.com> wrote:
> Hello. I have a program where you specify the name of the file you
> want to read from command line. But, I want to modify it so that if
> no file is specified, it reads from stdin. so here's a code snippet:
The null file handle <> has just that behavior. See 'perldoc perlop'
and search for "I/O Operators".
You can use the following while loop in your program:
while(<>) {
...
}
to do what you want. The elements of @ARGV are used as file names to
read. If empty, standard input will be read. You can also use "-" as a
file name to read standard input. You can manipulate the contents of
@ARGV before the loop to specify the files to read.
--
Jim Gibson
------------------------------
Date: Fri, 01 May 2009 19:29:17 -0700
From: sln@netherlands.com
Subject: Re: RSS XML Streams
Message-Id: <gb9nv4dgap19gktv6s7j9m8e7624gukgk7@4ax.com>
On Fri, 1 May 2009 11:01:02 +0200, pbnew@tin.it (isecc) wrote:
>Hi all,
>
>I'd like to process RSS XML downloaded from the web.
>
>What I mean is that:
>I start downloading the RSS_XML from the website and while
>downloading I start processing the data, in this way I can choose what
>to do with the data without completing the download.
>
>For this I'm using
>LWP::UserAgent that is calling my callback routine ie:
>
>my $res = $ua->request(
> HTTP::Request->new( GET => $rss ), \&rss_stream_SAX
> );
>
>and
>
>XML:SAX with my MySAXHandler
>
>What I was wondering is if there are some CPAN modules already written
>for this goal.
>
>Thanks.
>
>PS.
>I've already searched on CPAN for several modules... but I found nothing
>for my task. But since there are so many options I'd like a feedback
>from somebody "in the field" ;-)
Not from the field but I know what you are looking for.
Actually you probably are not looking for streams parsing.
Instead you need to instantiate a non-blocking parser that knows how
to parse indiscriminate chunks at a time of a continous document.
This requires that after the chunk is parsed, the parser maintain a small buffer
consisting of just the data from the last buffer that couldn't be parsed yet.
Then passing in the next chunk to be appended to the last chunk. The cycle
repeats until you have no more chunks left to pass in.
Meanwhile, as the parser is working on the chunks, it is dishing out SAX callbacks
as they are found.
From your SAX callback handlers, you will be able to set a flag that can be read within
your Agent request handler where you can in turn cancel the Parse and then 'die' within the
agent handler and bail out of the request.
As I noticed, XML::Parser (which is a wrapper for Expat) just happens to have ExpatNB (non-blocking).
Its only non-blocking in as much as it will instantiate the parser, then allow you to pass in 'chunks'
of xml to be parsed while you wait.
So 'parse_more( $string ) ' -> it calls your SAX callback where you set a flag when you get what you want ->
back from parsing chunk, check if the flag was set.
If you want to bail out of the parse, call 'parse_done'. Then bail out of your request handler with a 'die'.
I don't know if Parse will allow you to continue if it encounters a real xml error though, it may emit a die on ya.
I don't like this if it does it.
I already have this mechanism in place with a xml parser of my own, and if this doesen't get you what you want,
let me know and I can work with you.
Some INFO:
--------------------
XML::Parser::ExpatNB Methods
======================================
The class XML::Parser::ExpatNB is a subclass of XML::Parser::Expat used for non-blocking access to the expat library.
It does not support the parse, parsestring, or parsefile methods, but it does have these additional methods:
parse_more(DATA) - Feed expat more text to munch on.
parse_done - Tell expat that it's gotten the whole document.
XML::Parse
==========================
parse_start([ OPT => OPT_VALUE [...]])
Create and return a new instance of XML::Parser::ExpatNB. Constructor options may be provided.
If an init handler has been provided, it is called before returning the ExpatNB object. Documents are
parsed by making incremental calls to the parse_more method of this object, which takes a string.
A single call to the parse_done method of this object, which takes no arguments, indicates
that the document is finished.
If there is a final handler installed, it is executed by the parse_done method before returning
and the parse_done method returns whatever is returned by the final handler.
===================================
The below code is not fleshed out, however you would create a new parser, add youre SAX callback functions,
set up flags. In stream_h under TEST:
sub stream_h
{
my $string = shift;
print $string; # or to a file, or append to $result->content
# TEST:
$parse->parse_more ($string); # send more junk to the parser
if ($we_have_sax_info_lets_quit) # check your SAX function set flags (parse_more won't return until its done)
{
$parse->parse_done; # tell parser your done
die "dont need no more"; # tell ua request to stop collecting data
return '';
}
return $string; # docs say to do this but they are wrong, the content is not asigned.
# at this point you could catenate this to the $result-content, buy why would you
}
=====================================
use strict;
use warnings;
use Data::Dumper;
require LWP::UserAgent;
require HTTP::Request;
my $cnt = 0;
sub stream_h
{
my $string = shift;
print $string; # or to a file, or append to $result->content
# TEST: stop the request if $cnt > 2
# check x-died in the result header below or just set a user flag
die "dont need no more" if (++$cnt > 2);
return $string;
}
my $ua = LWP::UserAgent->new;
$ua->timeout(10);
my $request = HTTP::Request->new(GET => 'http://rss.cnn.com/rss/cnn_topstories.rss');
my $result = $ua->request( $request, \&stream_h );
print "\n\n", Dumper($result);
====================================
What I found out is that Agent has a special thread that fills up a default
buffer of 4k chunks. After that it tends to sleep until the buffer falls
below that level then requests more data. The buffer empty's upon each return from
request handler and will stop when the buffer is full. So there won't be a condition
where the download will continue in the background even if you pause the program
prompting for user input or something. This can be checked with various sleep, check
code, but I think this is the case.
Good luck, let me know what comes of this.
Like I said, I have special parse code if this parser doesen't do what you need.
-sln
------------------------------
Date: 6 Apr 2001 21:33:47 GMT (Last modified)
From: Perl-Users-Request@ruby.oce.orst.edu (Perl-Users-Digest Admin)
Subject: Digest Administrivia (Last modified: 6 Apr 01)
Message-Id: <null>
Administrivia:
#The Perl-Users Digest is a retransmission of the USENET newsgroup
#comp.lang.perl.misc. For subscription or unsubscription requests, send
#the single line:
#
# subscribe perl-users
#or:
# unsubscribe perl-users
#
#to almanac@ruby.oce.orst.edu.
NOTE: due to the current flood of worm email banging on ruby, the smtp
server on ruby has been shut off until further notice.
To submit articles to comp.lang.perl.announce, send your article to
clpa@perl.com.
#To request back copies (available for a week or so), send your request
#to almanac@ruby.oce.orst.edu with the command "send perl-users x.y",
#where x is the volume number and y is the issue number.
#For other requests pertaining to the digest, send mail to
#perl-users-request@ruby.oce.orst.edu. Do not waste your time or mine
#sending perl questions to the -request address, I don't have time to
#answer them even if I did know the answer.
------------------------------
End of Perl-Users Digest V11 Issue 2386
***************************************
