[22102] in Perl-Users-Digest
Perl-Users Digest, Issue: 4324 Volume: 10
daemon@ATHENA.MIT.EDU (Perl-Users Digest)
Tue Dec 31 21:06:24 2002
Date: Tue, 31 Dec 2002 18:05:24 -0800 (PST)
From: Perl-Users Digest <Perl-Users-Request@ruby.OCE.ORST.EDU>
To: Perl-Users@ruby.OCE.ORST.EDU (Perl-Users Digest)
Perl-Users Digest Tue, 31 Dec 2002 Volume: 10 Number: 4324
Today's topics:
Re: "use autouse" question (was Re: Not loading unneede (Tim Shoppa)
$^O variable (WAS: how to load a module depending on 'i <jurgenex@hotmail.com>
A new breed of free email! Worldwide access including W <fredh@bigfoot.com>
Auto-restart of blocking system calls after catching si (Tony Skelding)
Re: AWK vs PERL - splitting fields <goldbb2@earthlink.net>
Bijeenkomst Amsterdam Perl Mongers, Dinsdag 7 januari 2 (Johan Vromans)
dealing with mail/news-like headers <wobh@helium.barsoom.invalid>
Re: dealing with mail/news-like headers (Malcolm Dew-Jones)
Re: dealing with mail/news-like headers <wobh@helium.barsoom.invalid>
Dealing with split() and quotes (Max Power)
Re: Dealing with split() and quotes <shondell@cis.ohio-state.edu>
Re: Dealing with split() and quotes <mdudley@execonn.com>
Re: Dealing with split() and quotes (Anno Siegel)
Re: Dealing with split() and quotes (Joe Smith)
Re: Dealing with split() and quotes <shondell@cis.ohio-state.edu>
Re: Dealing with split() and quotes (Anno Siegel)
Direct Shared Memory Mapping? <gordan@_NOSPAM_bobich.net>
Re: Direct Shared Memory Mapping? <goldbb2@earthlink.net>
embeding perl / getting results (=?ISO-8859-1?Q?Hans_Schl=FCter?=)
Re: embeding perl / getting results (Anno Siegel)
Re: Getting Connecting Host Info from inetd <gk@pobox.com>
Digest Administrivia (Last modified: 6 Apr 01) (Perl-Users-Digest Admin)
----------------------------------------------------------------------
Date: 30 Dec 2002 08:07:11 -0800
From: shoppa@trailing-edge.com (Tim Shoppa)
Subject: Re: "use autouse" question (was Re: Not loading unneeded modules with use strict?)
Message-Id: <bec993c8.0212300807.759e8c2e@posting.google.com>
Benjamin Goldberg <goldbb2@earthlink.net> wrote in message news:<3E0F672D.1B80A591@earthlink.net>...
> Tim Shoppa wrote:
> > I'm assuming that my only problem here is syntax (e.g. "how do I
> > distinguish between the two autoused "new"'s) but maybe the disconnect
> > is more fundamental. I looked at the online docs for "use autouse" as
> > well as in the Perl Cookbook but they're really very vague about how I
> > can use them to abstract two classes with identical method names into
> > one "universal" class, like I want.
>
> I hadn't really thought about your problem enough when I first tried to
> answer your problem.
That's OK; you came a lot closer than the other responders!
> Assuming that everything for using GD::Image and
> PDF::API2 is object oriented, then a better solution would probably be
> something like:
>
> for my $pack (qw(GD::Image PDF::API2)) {
> (my $file = $pack.".pm") =~ s,::,/,g;
> next if $INC{ $file };
> my $glob = do { no strict 'refs'; \*{$pack."::new"} };
> *$glob = sub {
> undef &$glob;
> require $file;
> goto &{$pack->can("new") or die "$pack can't ->new"};
> };
> }
>
> If you stick this up top, then GD::Image->new and PDF::API2->new will
> automatically load up GD::Image and PDF::API2.
Cool. It looks like this creates a dummy entry in the symbol table for all
the "new" methods that I list, does the "require" when I actually
invoke each specific new, and destroys the dummy after it's actually loaded.
Under Perl 5.8.0, I had to comment out the "undef &$glob;" line because
it gave me the run-time message
Can't undef active subroutine at Uni.pm line 10.
It *seems* to be enough to just replace the name of the subroutine when
the require takes place. My understanding is that if the dummy subroutine
goes out of scope (is no longer referenced and is not being executed)
it gets destroyed, so I'm not sure that the "undef &$glob;" was needed.
Otherwise, the concept seems to work. It's a neat concept, because if
everything after the "->new" is accessed via object-oriented methods, then
the only thing I'll ever have to name at the top is the "->new".
Of course, not all modules are purely OO, but I've learned enough that I
think I can deal with those now! (Possibly there will be some ugly stuff
with any BEGIN blocks, etc...)
Thank you!
Tim.
------------------------------
Date: Tue, 31 Dec 2002 17:17:22 GMT
From: "Jürgen Exner" <jurgenex@hotmail.com>
Subject: $^O variable (WAS: how to load a module depending on 'if' condition ?)
Message-Id: <CikQ9.27267$ac.21718@nwrddc01.gnilink.net>
Andras Malatinszky wrote:
[...]
> The bottom line is that it's
> much safer to query Perl's $^O special variable, which contains your
> OS name (it returns 'linux' on my Linux box and 'MSWin32' on my Win
> 98 box).
Actually I wonder if this is always true. According to perldoc perlvar $^O
is
$^O The name of the operating system under which this copy of Perl
was built, as determined during the configuration process.
I read this as the values of $^O will be set at compile time. Now, what
happens if someone compiles perl on a cross compiler? Will $^O have the
value of the system perl was compiled on or the value of the system perl was
compiled for? Maybe it depends upon how the person doing the cross
compilation configured the configuration...
Also, the config man page warns:
WARNING
Because this information is not stored within the perl executable itself
it is possible (but unlikely) that the information does not relate to
the actual perl binary which is being used to access it.
And what happens in case of emulations? Like someone is brave/stupid enough
to run ActiveState Perl in the Windows emulator on Linux? Or isn't there
some runtime interface that allows execution of BSD binaries on Linux, too
(not sure about that)?
I guess this is mostly splitting hairs and not very relevant in everyday
live. But still it would be interesting to know.
jue
------------------------------
Date: Wed, 1 Jan 2003 01:43:28 +0000
From: "Fred Higgins" <fredh@bigfoot.com>
Subject: A new breed of free email! Worldwide access including WWW, Outlook, mobiles/PDAs etc.!
Message-Id: <3e1247c2$0$234$cc9e4d1f@news.dial.pipex.com>
Being a gadget freak I own a T68I, T39, 6310I, 7650, 8310, and an IPAQ 3970. I've spent many an hour looking for a unified email solution that would enable me to access my email from not only my *favorite email client & web browser* but also my *mobile devices* as well. Eventually I found a solution offered by UPO, here's the link :
http://www.universalpostoffice.com
They also offer you a variety of available email addresses including such as mymobile.net, and various other phone/PDA type domains. They're even including 10mb of storage and POP3 access and aren't charging a cent. For how long I don't know. BTW, easy access via GPRS/CSD with any built-in phone email client like the 7650 or T68(I) has!
------------------------------
Date: 31 Dec 2002 06:52:56 -0800
From: tony@skelding.co.uk (Tony Skelding)
Subject: Auto-restart of blocking system calls after catching signals on Linux
Message-Id: <a78026a1.0212310652.33900dc9@posting.google.com>
The Camel (edition 3) pp416. suggests that newer (i.e. BSD flavoured
systems including Linux) will automatically restart blocking system
calls if they are interrupted by a signal. Partly, this relieves the
programmer from the need to explicitly check for EINTR errors from
these calls. I am running Linux (2.4.2-2) and I don't appear to be
seeing this behaviour.
I suspect the real problem is my lack of understanding.
The following bit of code, which I've more or less lifted, works. I
don't think it should...
my $done;
$SIG{TERM} = sub { $done = 1 };
until ($done)
{
my $c = $socket->accept or next;
handle_call($c);
close $c;
}
As soon as I send a TERM signal it drops out of the loop - obviously
as intended. However, I would have expected is the underlying call to
accept() would have been automagically restarted by Perl and control
not returned until a real connection (or some exception) occurred.
------------------------------
Date: Tue, 31 Dec 2002 02:58:01 -0500
From: Benjamin Goldberg <goldbb2@earthlink.net>
Subject: Re: AWK vs PERL - splitting fields
Message-Id: <3E114E09.33B7083@earthlink.net>
Miguel Angelo Lapa Duarte wrote:
[snip]
> for($l = 0; $l < $ARGV[0]; ++$l) {
> for($c = 0; $c < $ARGV[1] - 1; ++$c) {
> print int(rand(100)) . ',';
> }
> print int(rand(100));
> print "\n";
> }
This will run faster if you rewrite it as:
for my $n ( @ARGV ) {
print int(rand(100)), "\n" for 1 .. $n;
}
--
$..='(?:(?{local$^C=$^C|'.(1<<$_).'})|)'for+a..4;
$..='(?{print+substr"\n !,$^C,1 if $^C<26})(?!)';
$.=~s'!'haktrsreltanPJ,r coeueh"';BEGIN{${"\cH"}
|=(1<<21)}""=~$.;qw(Just another Perl hacker,\n);
------------------------------
Date: 30 Dec 2002 21:24:54 +0100
From: jvromans@squirrel.nl (Johan Vromans)
Subject: Bijeenkomst Amsterdam Perl Mongers, Dinsdag 7 januari 2003
Message-Id: <m23cofntk9.fsf@phoenix.squirrel.nl>
[English version follows the dutch text]
Amsterdam.pm staat voor de "Amsterdamse Perl Mongers", een groep van
gebruikers van Perl. In tegenstelling tot wat de naam suggereert is
Amsterdam.pm niet beperkt tot alleen Amsterdam, maar functioneert, tot
er meer gebruikersgroepen in Nederland zijn, als Nederlandse
gebruikersgroep.
Amsterdam.pm organiseert informele bijeenkomsten waar Perl gebruikers
kunnen samenkomen en informatie en gebruikservaringen met betrekking
tot Perl kunnen uitwisselen. Deze bijeenkomsten vinden normaliter
plaats op elke eerste dinsdag van de maand. De voertaal binnen
Amsterdam.pm is in pricipe Nederlands, maar indien nodig zal Engels
worden gebruikt, b.v. om te communiceren met niet-Nederlandssprekende
aanwezigen.
De eerstvolgende bijeenkomst vindt plaats op dinsdag 7 januari van
20:00 tot 22:00 uur op het kantoor van XS4All, Eekholt 42, Diemen.
Voor een routebeschrijving, zie
http://www.xs4all.nl/contact/routebeschrijving.html
Liefhebbers van een etentje vooraf kunnen tussen 17:45 en 18:15
verzamelen. Om 18:15 (écht om 18:15!) zoeken we een restaurantje in de
buurt om een hapje te eten.
Bezoek onze Web site http://amsterdam.pm.org voor meer details.
[English version]
Amsterdam.pm stands for the Amsterdam Perl Mongers. We're basically a
Perl user group. Despite its name, it is not local to the Amsterdam
environment, but it welcomes Perl mongers from all over the
Netherlands.
Amsterdam.pm organises informal meetings where Perl users can meet,
and exchange information and experiences with regard to using Perl.
The meetings are normally held every first Tuesday of the month.
Although the preferred language for communication is Dutch, English
will be spoken if necessary.
Our next meeting is Tuesday January 7th, from 20:00 till 22:00 at the
office of XS4All, Eekholt 42, Diemen.
See http://www.xs4all.nl/contact/routebeschrijving.html for directions to
get there.
Should you want to join some of us for dinner, please gather between
17:45 and 18:15. At 18:15 sharp we'll leave for a restaurant somewhere
nearby for dinner.
See http://amsterdam.pm.org for more details.
------------------------------------------------------------------------------
Johan Vromans jvromans@squirrel.nl
Squirrel Consultancy Haarlem, the Netherlands
http://www.squirrel.nl http://www.squirrel.nl/people/jvromans
PGP Key 2048/4783B14D KFP = 65 44 CA 66 B3 50 0B 34 CE 0E FB CA 2D 95 34 D0
------------------------ "Arms are made for hugging" -------------------------
------------------------------
Date: 30 Dec 2002 19:08:23 GMT
From: William Clifford <wobh@helium.barsoom.invalid>
Subject: dealing with mail/news-like headers
Message-Id: <slrnb115h1.oqo.wobh@helium.barsoom>
I've been trying to organize a bunch of plain text files and I hit on
the idea of putting mail and news-like header information in them to
describe them and their relationships with each other. I started writing
a package to parse the header information when it occured to me that
something like this has probably already been written. After cursory
experimentation with the mail and news reading modules I learned that
they are too specialized. I poked around CPAN but I don't know if there
is any generic name for this sort of format. I might not have any
required headers, or consistent or convienent delimiters, and I could
piling the files up into big mbox-like archives or organizing them
across several directories. :)
Oh, alright, I'll try to come up with a delimiter or (more likely) a
required header or something, but the idea is to be as loose with this
as I can be so I can write scripts to deal with the specifics.
Anyway are there modules for this sort of thing (Text::ParseWords looks
sort of promising, maybe)? If not, any advice for writing one? I've been
tinkering with perl for a long time but I haven't done anything serious
with it really.
--
William Clifford wobh@mindspring.com http://wobh.home.mindspring.com
------------------------------
Date: 30 Dec 2002 13:11:10 -0800
From: yf110@vtn1.victoria.tc.ca (Malcolm Dew-Jones)
Subject: Re: dealing with mail/news-like headers
Message-Id: <3e10b66e@news.victoria.tc.ca>
William Clifford (wobh@helium.barsoom.invalid) wrote:
: I've been trying to organize a bunch of plain text files and I hit on
: the idea of putting mail and news-like header information in them to
: describe them and their relationships with each other. I started writing
: a package to parse the header information when it occured to me that
: something like this has probably already been written. After cursory
: experimentation with the mail and news reading modules I learned that
: they are too specialized. I poked around CPAN but I don't know if there
: is any generic name for this sort of format. I might not have any
: required headers, or consistent or convienent delimiters, and I could
: piling the files up into big mbox-like archives or organizing them
: across several directories. :)
: Oh, alright, I'll try to come up with a delimiter or (more likely) a
: required header or something, but the idea is to be as loose with this
: as I can be so I can write scripts to deal with the specifics.
: Anyway are there modules for this sort of thing (Text::ParseWords looks
: sort of promising, maybe)? If not, any advice for writing one? I've been
: tinkering with perl for a long time but I haven't done anything serious
: with it really.
perldoc -f split
has an example of general purpose parsing of mail-like headers.
------------------------------
Date: 31 Dec 2002 14:49:00 GMT
From: William Clifford <wobh@helium.barsoom.invalid>
Subject: Re: dealing with mail/news-like headers
Message-Id: <slrnb13amm.q60.wobh@helium.barsoom>
In <3e10b66e@news.victoria.tc.ca>,
Malcolm Dew-Jones <yf110@vtn1.victoria.tc.ca> wrote:
> William Clifford (wobh@helium.barsoom.invalid) wrote:
>: I've been trying to organize a bunch of plain text files and I hit on
>: the idea of putting mail and news-like header information in them to
>: describe them and their relationships with each other. I started writing
>: a package to parse the header information when it occured to me that
>: something like this has probably already been written.
[snip]
>: Anyway are there modules for this sort of thing (Text::ParseWords looks
>: sort of promising, maybe)? If not, any advice for writing one? I've been
>: tinkering with perl for a long time but I haven't done anything serious
>: with it really.
>
> perldoc -f split
>
> has an example of general purpose parsing of mail-like headers.
Hey, and it's a better regex than the one I copied out of the camel book
(second ed.). Thanks!
--
William Clifford wobh@mindspring.com http://wobh.home.mindspring.com
------------------------------
Date: 31 Dec 2002 11:54:31 -0800
From: Mike@Kordik.net (Max Power)
Subject: Dealing with split() and quotes
Message-Id: <1e89f371.0212311154.348424e1@posting.google.com>
I have a comma delimted file that I need to parse. I need to do
somethign with each element in between commas however there are a few
strings that have commas in them. For example:
1, high, "text1, text2", some_more_text
I need the "text1, text 2" string to count as one element. When I use
split and split on a comma I get two elements. Now all strings have
commas in them though. How do I get PERL to treat text in between
quotes as one element even thought it might have the delimiter in it?
Thanks,
Max
------------------------------
Date: 31 Dec 2002 15:37:35 -0500
From: Ryan Shondell <shondell@cis.ohio-state.edu>
Subject: Re: Dealing with split() and quotes
Message-Id: <xcwhectexgw.fsf@psi.cis.ohio-state.edu>
Mike@Kordik.net (Max Power) writes:
> I have a comma delimted file that I need to parse. I need to do
> somethign with each element in between commas however there are a few
> strings that have commas in them. For example:
>
> 1, high, "text1, text2", some_more_text
>
> I need the "text1, text 2" string to count as one element. When I use
> split and split on a comma I get two elements. Now all strings have
> commas in them though. How do I get PERL to treat text in between
> quotes as one element even thought it might have the delimiter in it?
perldoc -q split
Also, it's either Perl (the language) or perl (the interpreter), but
never PERL.
--
Ryan Shondell <shondell@cis.ohio-state.edu>
------------------------------
Date: Tue, 31 Dec 2002 16:25:01 -0500
From: Marshall Dudley <mdudley@execonn.com>
Subject: Re: Dealing with split() and quotes
Message-Id: <3E120B2C.CE2D1327@execonn.com>
Max Power wrote:
> I have a comma delimted file that I need to parse. I need to do
> somethign with each element in between commas however there are a few
> strings that have commas in them. For example:
>
> 1, high, "text1, text2", some_more_text
>
> I need the "text1, text 2" string to count as one element. When I use
> split and split on a comma I get two elements. Now all strings have
> commas in them though. How do I get PERL to treat text in between
> quotes as one element even thought it might have the delimiter in it?
>
> Thanks,
> Max
You might try the following code which I wrote for this very thing:
$delimiter = ',';
$line = 'this is a test,"this is field 2",",","and, this is field 3",",
field beginning with a comma","field ending with a comma,","this line has
a ""double quote"" in it",the last field';
if ($delimiter eq ',') { #comma delimited fields
$line =~ s/\"\"/\"\;/g; #replace double quotes with one quote
tag
my (@temp) = split /\,/,$line; #split between all commas
foreach my $field (@temp) {
if ($field eq '"') { #toggle if inside a quote or not, if a
comma is just inside or outside a field quote
if ($inquote) {
$inquote = "";
push @fields,$section;
} else {
$section .= ","; #add the comma deleted earlier by
the split
$inquote = 1;
}
} elsif ($field =~ /^\"(.*)\"$/) { #quote on both ends
push @fields,$1; #no commas in the middle,
so save as is
} elsif ($field =~ s/^\"//) { #begin quoted section
$section = "$field,";
$inquote = 1;
} elsif ($field =~ s/\"$//) { #end quoted section
$section .= $field;
push @fields,$section;
$inquote = "";
$section = "";
} elsif ($inquote) {
$section .= "$field,";
} else {
push @fields,$field;
}
}
} else {
(@fields) = split /$delimiter/,$line;
}
foreach my $entry (@fields) {
$entry =~ s/\"/"/g; #put the " back in place of the
" tag
print "$entry\n";
}
"$delimiter"defines what delimiter is being used, and if it is a comma,
then it has to look for the quotes around a field, and ignore commas and
quotes inside the field. Quotes inside the field are expressed as two
quotes, so it has to look for that as well.
For both comma and non-comma delimited fields, the final array is in
@fields. ALL fields will end up with any embedded quotes as a tag of
", so if this is not going out to a browser, you will need to loop
through the fields and replae them with a single " again as shown in the
loop at the end that prints the fields.
perl ckcomdelim.pl
this is a test
this is field 2
,
and, this is field 3
, field beginning with a comma
field ending with a comma,
this line has a "double quote" in it
the last field
Marshall
------------------------------
Date: 31 Dec 2002 22:06:56 GMT
From: anno4000@lublin.zrz.tu-berlin.de (Anno Siegel)
Subject: Re: Dealing with split() and quotes
Message-Id: <aut4e0$pgb$3@mamenchi.zrz.TU-Berlin.DE>
Ryan Shondell <shondell@cis.ohio-state.edu> wrote in comp.lang.perl.misc:
> Mike@Kordik.net (Max Power) writes:
>
> > I have a comma delimted file that I need to parse. I need to do
> > somethign with each element in between commas however there are a few
> > strings that have commas in them. For example:
> >
> > 1, high, "text1, text2", some_more_text
> >
> > I need the "text1, text 2" string to count as one element. When I use
> > split and split on a comma I get two elements. Now all strings have
> > commas in them though. How do I get PERL to treat text in between
> > quotes as one element even thought it might have the delimiter in it?
>
> perldoc -q split
Hardly. There are no provisions in split() that facilitate what the OP
wants to do.
However, the question is a faq. See "perldoc -q delimited" for a
comprehensive answer. The upshot is, there are (at least) two modules
on CPAN that do this.
> Also, it's either Perl (the language) or perl (the interpreter), but
> never PERL.
True, and another faq. "perldoc -q difference".
Anno
------------------------------
Date: Tue, 31 Dec 2002 22:10:30 GMT
From: inwap@inwap.com (Joe Smith)
Subject: Re: Dealing with split() and quotes
Message-Id: <qBoQ9.1579$io.69159@iad-read.news.verio.net>
In article <1e89f371.0212311154.348424e1@posting.google.com>,
Max Power <Mike@Kordik.net> wrote:
>I have a comma delimted file that I need to parse.
Re-inventing the wheel is such a waste; there are modules designed for
that very purpose.
http://search.cpan.org/search?query=csv
--
See http://www.inwap.com/ for PDP-10 and "ReBoot" pages.
------------------------------
Date: 31 Dec 2002 17:29:52 -0500
From: Ryan Shondell <shondell@cis.ohio-state.edu>
Subject: Re: Dealing with split() and quotes
Message-Id: <xcwd6nhes9r.fsf@psi.cis.ohio-state.edu>
anno4000@lublin.zrz.tu-berlin.de (Anno Siegel) writes:
> Ryan Shondell <shondell@cis.ohio-state.edu> wrote in comp.lang.perl.misc:
> > Mike@Kordik.net (Max Power) writes:
> >
> > > I have a comma delimted file that I need to parse. I need to do
> > > somethign with each element in between commas however there are a few
> > > strings that have commas in them. For example:
> > >
> > > 1, high, "text1, text2", some_more_text
> > >
> > > I need the "text1, text 2" string to count as one element. When I use
> > > split and split on a comma I get two elements. Now all strings have
> > > commas in them though. How do I get PERL to treat text in between
> > > quotes as one element even thought it might have the delimiter in it?
> >
> > perldoc -q split
>
> Hardly. There are no provisions in split() that facilitate what the OP
> wants to do.
Absolutely true. However, I did not suggest perldoc -f split, but
perldoc -q split, which (at least for me) finds the FAQ entry.
--
Ryan Shondell <shondell@cis.ohio-state.edu>
------------------------------
Date: 31 Dec 2002 22:33:32 GMT
From: anno4000@lublin.zrz.tu-berlin.de (Anno Siegel)
Subject: Re: Dealing with split() and quotes
Message-Id: <aut5vs$pgb$5@mamenchi.zrz.TU-Berlin.DE>
Ryan Shondell <shondell@cis.ohio-state.edu> wrote in comp.lang.perl.misc:
> anno4000@lublin.zrz.tu-berlin.de (Anno Siegel) writes:
>
> > Ryan Shondell <shondell@cis.ohio-state.edu> wrote in comp.lang.perl.misc:
> > > Mike@Kordik.net (Max Power) writes:
> > >
> > > > I have a comma delimted file that I need to parse. I need to do
> > > > somethign with each element in between commas however there are a few
> > > > strings that have commas in them. For example:
> > > >
> > > > 1, high, "text1, text2", some_more_text
> > > >
> > > > I need the "text1, text 2" string to count as one element. When I use
> > > > split and split on a comma I get two elements. Now all strings have
> > > > commas in them though. How do I get PERL to treat text in between
> > > > quotes as one element even thought it might have the delimiter in it?
> > >
> > > perldoc -q split
> >
> > Hardly. There are no provisions in split() that facilitate what the OP
> > wants to do.
>
> Absolutely true. However, I did not suggest perldoc -f split, but
> perldoc -q split, which (at least for me) finds the FAQ entry.
Oh. Right.
Anno
------------------------------
Date: Tue, 31 Dec 2002 18:52:36 +0000
From: Gordan <gordan@_NOSPAM_bobich.net>
Subject: Direct Shared Memory Mapping?
Message-Id: <%HlQ9.15$X41.13532@newsfep2-gui>
Hi.
Is there a way to get something mapped directly into shared memory, and then
access it efficiently via, say, a pointer in Perl?
I have looked at IPC::Shareable, which seems to be unable to use memory
segments bigger than 64 KB (haven't worked out why yet), and
IPC::ShareLite, which only supports scalar input. They BOTH require the
memory contents to be copied into a local variable which means that for
each parallel process reading the area, the memory has to be copied out,
worked on, and then put back. This becomes even worse when the
IPC::Shareable module is used or the Storeable module is used to serialize
the data at every read-write to the memory.
What I am really looking for is a way to map a piece of shared memory, and
then manipulate it as a hash, and/or an array/list WITHOUT copying the
contents into a local, unshared variable.
Is there a module out there to do this in Perl?
The reason I'm asking this is because I am designing a parallel application.
The only two options I have are threads in Perl 5.8.0, which is unstable
(my program crashes intermittently), or fork(). I have ported my program to
use IPC::Shareable instead (on Perl 5.6.1), but it seems that the shared
variables are limited to 64 KB - otherwise it is more stable than threads
in 5.8.0.
If I use threads, I can mark a variable as shared, which works great. But if
I use fork(), then I have to map the variable into shared memory, to make
it available to all processes, and there doesn't appear to be an efficient
way to do this, avoiding the lock->fetch->change->write->unlock process on
the ENTIRE memory segment. I guess I could map EACH element of an array
into a SEPARATE segment, and store pointers/keys into a list, so only that
list segment has to be completely copied in and out of memory on each
iteration, but this seems far too complicated, and potentially wasteful for
the memory, as it can only be allocated in 4KB chunks.
Shared memory can only be allocated in blocks of PAGE_SIZE, which is 4096
bytes on x86, so to optimize memory usage, each segment would have to
consist of multiple array/list elements, thus giving a list of lists
arrangement, which would limit parallelism, because only each "chunk" could
be processed in parallel to save on shared memory bandwidth and
consumption. Or I could map one segment as "cache" that would only be
re-filled when it is empty, but this seems vastly overcomplicated, and
still doesn't solve the initial problem.
All these solutions seem to involve a huge amount of working around the
problem, and ultimately mean that shared memory STILL has to be copied into
unshared memory first, rather than operating on the shared memory directly.
Is there no sensible way to do this directly? Help!
TIA.
Gordan
------------------------------
Date: Tue, 31 Dec 2002 16:39:21 -0500
From: Benjamin Goldberg <goldbb2@earthlink.net>
Subject: Re: Direct Shared Memory Mapping?
Message-Id: <3E120E89.3A37DA8C@earthlink.net>
Gordan wrote:
>
> Hi.
>
> Is there a way to get something mapped directly into shared memory,
> and then access it efficiently via, say, a pointer in Perl?
Perl doesn't have pointers.
It has scalars, hashes, and arrays.
> I have looked at IPC::Shareable, which seems to be unable to use
> memory segments bigger than 64 KB (haven't worked out why yet), and
> IPC::ShareLite, which only supports scalar input. They BOTH require
> the memory contents to be copied into a local variable which means
> that for each parallel process reading the area, the memory has to be
> copied out, worked on, and then put back.
Unless you want to write your program using XS or Inline::C, this is
completely unavoidable.
The best you would be able to do would be to use tieing or overloading
to make it *appear* as if the "copy out, modify, put back" was a single
step.
> This becomes even worse when the IPC::Shareable module is used or the
> Storeable module is used to serialize the data at every read-write to
> the memory.
You should only be using Storable if you're dealing with complex,
arbitrary data structures. If you're using simple data structures (or
complex but fixed-structure data structures), then there are likely
better techniques.
> What I am really looking for is a way to map a piece of shared memory,
> and then manipulate it as a hash, and/or an array/list WITHOUT copying
> the contents into a local, unshared variable.
Err, manipulate how? And an hash/array/list of what?
> Is there a module out there to do this in Perl?
>
> The reason I'm asking this is because I am designing a parallel
> application. The only two options I have are threads in Perl 5.8.0,
> which is unstable (my program crashes intermittently), or fork(). I
> have ported my program to use IPC::Shareable instead (on Perl 5.6.1),
> but it seems that the shared variables are limited to 64 KB -
> otherwise it is more stable than threads in 5.8.0.
>
> If I use threads, I can mark a variable as shared, which works great.
Umm, you do realize that with threads, there is actually a hidden
fetch/modify/store done behind the scenes? (Sharing a variable actually
ties it to a special class)
> But if I use fork(), then I have to map the variable into shared
> memory, to make it available to all processes, and there doesn't
> appear to be an efficient way to do this,
How about forks.pm and forks::shared from CPAN?
Just like threads::shared, it still does a fetch/modify/store, except
that instead of fetching it from a special "shared" thread/memory pool,
it fetches it from actual shared/mmapped memory.
> avoiding the lock->fetch->change->write->unlock process on
> the ENTIRE memory segment. I guess I could map EACH element of an
> array into a SEPARATE segment, and store pointers/keys into a list, so
> only that list segment has to be completely copied in and out of
> memory on each iteration, but this seems far too complicated, and
> potentially wasteful for the memory, as it can only be allocated in
> 4KB chunks.
That would depend entirely on how you did it.
Eg, consider the situation where your data consists of just numbers.
You could do:
use constant PACKFMT => 'd';
use constant PACKLEN => pack(PACKFMT);
use constant MAXSHARED => 4096;
die if MAXSHARED % PACKLEN;
sub TIEARRAY {
my ($class, $size) = @_;
my @self = $size;
$size *= PACKLEN;
while( $size > 0 ) {
my $get = $size > MAXSHARED ? MAXSHARED : $size;
my $id = shmget IPC_PRIVATE, $get, S_IRWXU;
push @self, $id;
$size -= MAXSHARED;
}
bless \@self, $class;
}
sub FETCH {
my ($self, $index) = @_;
my $size = $self->[0];
die if $index >= $size;
$index *= PACKLEN;
my $id = int($index / MAXSHARED);
my $idx = $index % MAXSHARED;
shmread( $id, my( $packed_num ), $idx, PACKLEN );
unpack PACKFMT, $packed_num;
}
sub STORE {
my ($self, $index, $num) = @_;
my $size = $self->[0];
die if $index >= $size;
$index *= PACKLEN;
my $id = int($index / MAXSHARED);
my $idx = $index % MAXSHARED;
shmwrite( $id, pack(PACKFMT, $num), $idx, PACKLEN );
}
sub FETCHSIZE { $_[0][0] }
After you've done this, you simply tie your array variable to the class,
then fork, and use the array as normal. No memory is wasted.
> Shared memory can only be allocated in blocks of PAGE_SIZE, which is
> 4096 bytes on x86, so to optimize memory usage, each segment would
> have to consist of multiple array/list elements, thus giving a list of
> lists arrangement, which would limit parallelism, because only each
> "chunk" could be processed in parallel to save on shared memory
> bandwidth and consumption. Or I could map one segment as "cache" that
> would only be re-filled when it is empty, but this seems vastly
> overcomplicated, and still doesn't solve the initial problem.
It would be nice if you could give a clear explanation of your initial
problem; what exactly is the problem you're trying to solve.
> All these solutions seem to involve a huge amount of working around
> the problem, and ultimately mean that shared memory STILL has to be
> copied into unshared memory first, rather than operating on the shared
> memory directly.
The closest thing you would be able to get to working on the shared
memory directly is a combination of overload.pm, and use of XS or
Inline::C. I'm not going to write code to do that unless I *know* that
it's really what you need/want, and that it will be used.
> Is there no sensible way to do this directly? Help!
--
$..='(?:(?{local$^C=$^C|'.(1<<$_).'})|)'for+a..4;
$..='(?{print+substr"\n !,$^C,1 if $^C<26})(?!)';
$.=~s'!'haktrsreltanPJ,r coeueh"';BEGIN{${"\cH"}
|=(1<<21)}""=~$.;qw(Just another Perl hacker,\n);
------------------------------
Date: 31 Dec 2002 11:59:17 -0800
From: hans@schlueters.de (=?ISO-8859-1?Q?Hans_Schl=FCter?=)
Subject: embeding perl / getting results
Message-Id: <4209e1d7.0212311159.718cabe4@posting.google.com>
Hi,
I've got C-Application and I want to run Perl from it. I read man
perlembed and the chapter about embedding Perl to C in the "Advanced
Perl programming" book. My source looks like this (shorted, I hope I
dind't forget an importent part):
------
static PerlInterpreter *my_perl
...
STRLEN n_a;
char *embedding[] = {"", "-e", "0" };
char *code; // My Perl-Code.
...
my_perl = perl_alloc();
perl_construct(my_perl);
perl_parse(my_perl, NULL, 3, embedding, NULL);
PL_exit_flags |= PERL_EXIT_DESTRUCT_END;
perl_run(my_perl);
eval_pv(code, 1);
-----
char *code holds my Perl-Source and is executed sucessfully. My
Problem is that it prints different things to STDOUT and I want to
collect these infos, so I can use these infos from within my C code.
If important: I work on a Linux 2.4.20 box with glibc 2.3.1, gcc 3.2.1
and Perl 5.8.0.
Thanks
Hans
------------------------------
Date: 31 Dec 2002 21:37:19 GMT
From: anno4000@lublin.zrz.tu-berlin.de (Anno Siegel)
Subject: Re: embeding perl / getting results
Message-Id: <aut2mf$pgb$1@mamenchi.zrz.TU-Berlin.DE>
Hans Schlüter <hans@schlueters.de> wrote in comp.lang.perl.misc:
> Hi,
>
> I've got C-Application and I want to run Perl from it. I read man
> perlembed and the chapter about embedding Perl to C in the "Advanced
> Perl programming" book. My source looks like this (shorted, I hope I
> dind't forget an importent part):
>
> ------
> static PerlInterpreter *my_perl
> ...
> STRLEN n_a;
> char *embedding[] = {"", "-e", "0" };
> char *code; // My Perl-Code.
> ...
> my_perl = perl_alloc();
> perl_construct(my_perl);
> perl_parse(my_perl, NULL, 3, embedding, NULL);
> PL_exit_flags |= PERL_EXIT_DESTRUCT_END;
> perl_run(my_perl);
> eval_pv(code, 1);
> -----
>
> char *code holds my Perl-Source and is executed sucessfully. My
> Problem is that it prints different things to STDOUT and I want to
> collect these infos, so I can use these infos from within my C code.
Well, printing things isn't the way to return values to the calling
C program. As perlembed indicates, you can deposit results in package
variables and use get_sv() plus the SvXV macros to convert them to
C types. The return value of eval_pv() (an SV*) can be similarly
used while avoiding package variables.
Anno
------------------------------
Date: Mon, 30 Dec 2002 13:16:16 -0500
From: George Kuetemeyer <gk@pobox.com>
Subject: Re: Getting Connecting Host Info from inetd
Message-Id: <auq2fm$53h$1@omle.tju.edu>
Benjamin Goldberg wrote:
> George Kuetemeyer wrote:
>
>>I want to use inetd to kick off a Perl script. How can the script
>>itself determine connecting host info, since inetd is brokering
>>connections?
>
>
> Inetd merely does accept, and then uses the dup2() function to make that
> socket into the stdin and stdout (any maybe stderr, I don't recall) of
> the newly started perl script.
>
> This means that you should be able to do:
> use IO::Socket;
> # Tell perl that STDIN is actually a network socket.
> my $in = bless \*STDIN, 'IO::Socket::INET';
> my $peeraddr = $in->peeraddr;
> my $peerport = $in->peerport;
> print MYLOGFILE "The remote host is ", $in->peerhost, "\n";
> [untested]
You can delete '[untested]' from your reply. Your code works as
advertised (I'm using xinetd on RH 7.2). Many thanks!!
GK
------------------------------
Date: 6 Apr 2001 21:33:47 GMT (Last modified)
From: Perl-Users-Request@ruby.oce.orst.edu (Perl-Users-Digest Admin)
Subject: Digest Administrivia (Last modified: 6 Apr 01)
Message-Id: <null>
Administrivia:
The Perl-Users Digest is a retransmission of the USENET newsgroup
comp.lang.perl.misc. For subscription or unsubscription requests, send
the single line:
subscribe perl-users
or:
unsubscribe perl-users
to almanac@ruby.oce.orst.edu.
To submit articles to comp.lang.perl.announce, send your article to
clpa@perl.com.
To request back copies (available for a week or so), send your request
to almanac@ruby.oce.orst.edu with the command "send perl-users x.y",
where x is the volume number and y is the issue number.
For other requests pertaining to the digest, send mail to
perl-users-request@ruby.oce.orst.edu. Do not waste your time or mine
sending perl questions to the -request address, I don't have time to
answer them even if I did know the answer.
------------------------------
End of Perl-Users Digest V10 Issue 4324
***************************************
