[18262] in Perl-Users-Digest
Perl-Users Digest, Issue: 430 Volume: 10
daemon@ATHENA.MIT.EDU (Perl-Users Digest)
Tue Mar 6 21:05:36 2001
Date: Tue, 6 Mar 2001 18:05:12 -0800 (PST)
From: Perl-Users Digest <Perl-Users-Request@ruby.OCE.ORST.EDU>
To: Perl-Users@ruby.OCE.ORST.EDU (Perl-Users Digest)
Message-Id: <983930712-v10-i430@ruby.oce.orst.edu>
Content-Type: text
Perl-Users Digest Tue, 6 Mar 2001 Volume: 10 Number: 430
Today's topics:
Re: Feeding value pairs to ASP scripts? <peter.sundstrom-eds@eds.com>
Re: flock and close with empty read strangeness <mischief@velma.motion.net>
Re: flock and close with empty read strangeness <mischief@velma.motion.net>
Re: flock and close with empty read strangeness <groovyt@erols.com>
Re: flock and close with empty read strangeness <groovyt@erols.com>
Re: flock and close with empty read strangeness <groovyt@erols.com>
Re: flock and close with empty read strangeness (Ilya Zakharevich)
Re: flock and close with empty read strangeness (Martien Verbruggen)
Re: flock and close with empty read strangeness (Martien Verbruggen)
Re: flock and close with empty read strangeness <groovyt@erols.com>
Graphics generation - looking for some advice on module (Glyndwr)
How to convert morse to ascii using perl? <ckzchang02@home.com>
Re: How to convert morse to ascii using perl? <tony_curtis32@yahoo.com>
Re: Is Perl right for me? (Miguel Cruz)
Re: Opening files on other Windows machines (Miguel Cruz)
Re: Perl for System V <mischief@velma.motion.net>
Digest Administrivia (Last modified: 16 Sep 99) (Perl-Users-Digest Admin)
----------------------------------------------------------------------
Date: Wed, 7 Mar 2001 12:35:05 +1300
From: "Peter Sundstrom" <peter.sundstrom-eds@eds.com>
Subject: Re: Feeding value pairs to ASP scripts?
Message-Id: <983s7l$uop$1@hermes.nz.eds.com>
<horribly@bogus.com> wrote in message
news:3aa56bbc.341916580@news.berkeley.edu...
> On Tue, 06 Mar 2001 00:39:41 GMT, sjs@linux.ca (Steven Smolinski)
> wrote:
>
> >[Note more evidence for the spamtrapped email bogosity indicator.]
> >horribly@bogus.com <horribly@bogus.com> wrote:
> >> Is there a different "trick" for
> >> feeding value pairs to asp scripts?
> >
> >What did they say on the ASP newsgroup?
>
> They said 'Ciao bella'. The only asp specific groups I can find are in
> Italian and Czech. I thought others with LWP experience might have
> faced the same issue.
Try microsoft.public.inetserver.asp.general
------------------------------
Date: Tue, 06 Mar 2001 23:31:19 -0000
From: Chris Stith <mischief@velma.motion.net>
Subject: Re: flock and close with empty read strangeness
Message-Id: <taasq7bqh4foee@corp.supernews.com>
ZepHead <groovyt@erols.com> wrote:
>>
>> Mandatory LOCK_EX is a bad thing, because it would prevent anyone from
>> even reading the file (which would happen with a LOCK_SH, if locking
>> always happened).
> well all writes should be LOCK_EX IMHO
> and this thread is about writing and race conditions
> the open > causes when another process reads the truncated
> file before the write which happens in overloaded
> servers a lot and why I'm here. ; )
Soetimes I _want_ to blow away a file even if it's being read.
I often want to write to one that's being read (like log files
that I'm tailing in another window).
>>
>> +<< could be used for O_CREAT|O_RDWR, but I can't think of anything
>> logical for the other one.
> i said << but that looks to much like a read and so much so
> i called it a read by mistake a post back
> I guess +<< is the way to go and with +<
> it makes sense.
I understand that it's a read and write combined, but it is
still messing with my head. It doesn't mimic the shell at all.
It just looks funky to me. (I know that it shouldn't, since
the >>+ looks normal. I jsut can't help myself.) Maybe I just
can't deal with more than a certain number of symbolic options
to the same function call. :-/
>>
>> > as always just my opinion : P
>>
>> Understood.
>>
>> Martien
> same here. I like talking about this stuff ; )
Me too. I just don't think we should have YAFOM.
(yet another file opening mode)
Chris
--
Christopher E. Stith
The purpose of a language is not to help you learn the
language, but to help you learn other things by using the
language. --Larry Wall, The Culture of Perl, August 1997
------------------------------
Date: Tue, 06 Mar 2001 23:39:22 -0000
From: Chris Stith <mischief@velma.motion.net>
Subject: Re: flock and close with empty read strangeness
Message-Id: <taat9a4uohefd8@corp.supernews.com>
Anno Siegel <anno4000@lublin.zrz.tu-berlin.de> wrote:
> According to Ilya Zakharevich <ilya@math.ohio-state.edu>:
>> [A complimentary Cc of this posting was sent to Anno Siegel
>> <anno4000@lublin.zrz.tu-berlin.de>],
>> who wrote in article <982e24$rqt$1@mamenchi.zrz.TU-Berlin.DE>:
>> > > You mean, you want to turn your filesystem into a database?
>> >
>> > If I had to design an OS, I'd give that serious consideration.
>>
>> You do not need to design an OS to have this. All of the contemporary
>> OSes already have this up to some extend. At least they get
>> predictable locking, including locking of regions, and
>> logarithmic-time directory search. No relational stuff, but for
>> the simple tie-AnyDBM-like access you can use the plain filesystems API.
> Certainly, and there are successful DB implementations that rely
> only on file system functions. OTOH, quite a few of those have
> proven non-scalable under growing load. So maybe it's a good idea
> to turn things around and base the file system on a "real" database.
As an aside, there are experimental OSes out there that are built
from the ground up with persistence in mind. Many of these use a
database as the only means of information storage, and make the
implementation of the database seamless to the application programmer.
Everything looks like core. None of these are mainstream, but many good
ideas in OS design take a while to become mainstream. Perhaps one day
this will be a popular alternative.
But this isn't about Perl any longer.
Chris
--
Christopher E. Stith
People understand instinctively that the best way for computer programs to
communicate with each other is for each of the them to be strict in what they
emit, and liberal in what they accept. The odd thing is that people themselves
are not willing to be strict in how they speak, and liberal in how they listen.
-- Larry Wall, 2nd State of the Onion Address, August 1998
------------------------------
Date: Tue, 06 Mar 2001 19:41:16 -0500
From: ZepHead <groovyt@erols.com>
Subject: Re: flock and close with empty read strangeness
Message-Id: <groovyt-BCC1DD.19411506032001@virt-reader.news.rcn.net>
In article <slrn9a99qa.onq.abigail@tsathoggua.rlyeh.net>,
abigail@foad.org (Abigail) wrote:
>
> You mean, you want to turn your filesystem into a database?
most all good OS at low level, read/write to ram buffers
first then flush later. this is also done for speed and so
multiple processes can share/read/write resources at the same time
(just not to the same place , until the flush)
as long as admin was able to write when they wanted to write
(your test of a good system in your last post) then who cares if the
flush was delayed a few seconds so as to guarantee no race conditions.
>
> Eh? So, what's the OS going to do if there's already a read lock, and
> a request for a write comes? Close the filehandles that are reading?
>
>
> Abigail
during a read with shared lock the write has to wait
but on some systems if another read comes in
then the write gets pushed back even when the second
read came in AFTER the write. This could push the write back
forever if you have enoughs reads coming in over lapping each other.
This is bad since the new DATA the write has is what the reads
want but are not getting. All these reads are now getting stale data
These system are based on greedy coding. this is not good
for real time systems like this clients site.
better systems give write higher priority and reads that come in after
a write have to wait which is a good thing since the read WANTS the new data
and not the stale data that it would get if we let the read push back the write
and get the old data.
so to answer your question the read finishes
then the write goes (even though shared lock read requests came in
while the first read was still going on. these reads must wait for the write
if there is NO write in queue then these reads read right then with no waiting
just as noraml shared lock read should do)
using your test, admin gets their new accounts in at the fastest possible time
the system can which is after the first read is over but before the come from
behind reads.
of course if all reads were lock ex then it is moot
but we are talking shared lock reads here ; )
both systems have pros and cons but the server
that this client is on is greedy and i have seen
a write pushed back MINUTES due to many reads coming in
after the write.
yes this means the stock price quote is STALE (minutes old)
not good in this case but may be fine for other systems
that do not live or die on data being fresh as possible.
------------------------------
Date: Tue, 06 Mar 2001 19:57:57 -0500
From: ZepHead <groovyt@erols.com>
Subject: Re: flock and close with empty read strangeness
Message-Id: <groovyt-7517D9.19575706032001@virt-reader.news.rcn.net>
In article <slrn9a9ahj.efn.mgjv@martien.heliotrope.home>,
mgjv@tradingpost.com.au (Martien Verbruggen) wrote:
> That would mean that while Abigail is reading the file, the admin might
> add one account. THen go away for coffee. The admin's assistant comes
> in, and check the password file (which is allowed, bevause it's only
> opened for read). He would see the copy that isn't updated, so he
> changes the file, to add another account, but he accidentally picks the
> same user id.
not at all.
the second admin person sees the changes because the OS
is smart enough to show him the cache since cache always has priority
over hard drive space.
heck this is how CPU from L1 L2 L3 RAM VM works basically too.
someone has to have the freshest data and you do NOT go to HD first
to find out who. you go to L1 etc....
this is basic OS stuff and trust me has all been
figured out by a lot smarter people than myself a long time ago ; )
think about why this thread even started.
you have open > that truncates before the lock etc....
well the reads come in and gets an empty file
these reads are reading from cache not HD
since the write has not happened yet and
no flush and buffer shut down occurred.
your above example of the second admin
coming in also reads from the cache and of course
if the write did get it's chance to write to HD
then the cache is flushed and closed down
so the second person then reads from that .
so they ALWAYS get the fresh data.
------------------------------
Date: Tue, 06 Mar 2001 20:05:58 -0500
From: ZepHead <groovyt@erols.com>
Subject: Re: flock and close with empty read strangeness
Message-Id: <groovyt-1D6CC7.20055806032001@virt-reader.news.rcn.net>
In article <slrn9a9anu.efn.mgjv@martien.heliotrope.home>,
mgjv@tradingpost.com.au (Martien Verbruggen) wrote:
>
> In other words, to prevent confusion again, you want an equivalent of
>
> sysopen FH, $name, O_WRONLY|O_CREAT;
>
> right?
>
> And we're full circle. I just ave you that command :)
>
> Martien
lol yeah this could go on forever : )
but I am having fun because I have not thought
about this stuff in years and being new to perl
it sorta has me living in the past : )
we still have not figured out if we should just
use sysopen or add to open like Bart? suggested
using +<< (which I like)
I feel adding +<< to open is the way to go
if there are no low level reasons that would
stop this.
I mean there has to be reason this was not included
with open , right? could this just be a simple over sight?
------------------------------
Date: 7 Mar 2001 01:07:05 GMT
From: ilya@math.ohio-state.edu (Ilya Zakharevich)
Subject: Re: flock and close with empty read strangeness
Message-Id: <9841jp$shd$1@charm.magnus.acs.ohio-state.edu>
[A complimentary Cc of this posting was sent to ZepHead
<groovyt@erols.com>],
who wrote in article <groovyt-7517D9.19575706032001@virt-reader.news.rcn.net>:
> > That would mean that while Abigail is reading the file, the admin might
> > add one account. THen go away for coffee. The admin's assistant comes
> > in, and check the password file (which is allowed, bevause it's only
> > opened for read). He would see the copy that isn't updated, so he
> > changes the file, to add another account, but he accidentally picks the
> > same user id.
>
> not at all.
>
> the second admin person sees the changes because the OS
> is smart enough to show him the cache since cache always has priority
> over hard drive space.
?! What were you smoking? How can the second admin see the changes
done by the first one *before* these changes are made? Remember, the
program run by the first admin had no chance to lock the file yet, not
speaking of writing anything to it...
Sometimes the update may happen by an
create-another-file-then-atomically-rename-to-the-destination-name
method, but we are discussion locking here, not this method.
Ilya
------------------------------
Date: Wed, 07 Mar 2001 01:32:36 GMT
From: mgjv@tradingpost.com.au (Martien Verbruggen)
Subject: Re: flock and close with empty read strangeness
Message-Id: <slrn9ab3tk.nts.mgjv@verbruggen.comdyn.com.au>
On Tue, 06 Mar 2001 19:57:57 -0500,
ZepHead <groovyt@erols.com> wrote:
> In article <slrn9a9ahj.efn.mgjv@martien.heliotrope.home>,
> mgjv@tradingpost.com.au (Martien Verbruggen) wrote:
>
>
>> That would mean that while Abigail is reading the file, the admin might
>> add one account. THen go away for coffee. The admin's assistant comes
>> in, and check the password file (which is allowed, bevause it's only
>> opened for read). He would see the copy that isn't updated, so he
>> changes the file, to add another account, but he accidentally picks the
>> same user id.
>
> not at all.
>
> the second admin person sees the changes because the OS
> is smart enough to show him the cache since cache always has priority
> over hard drive space.
>
> heck this is how CPU from L1 L2 L3 RAM VM works basically too.
> someone has to have the freshest data and you do NOT go to HD first
> to find out who. you go to L1 etc....
>
> this is basic OS stuff and trust me has all been
> figured out by a lot smarter people than myself a long time ago ; )
But we're talking file systems. We're talking about writes having been
commited. If your scheme requires all this stuff to sit in a volatile
cache, it isn't going to be what is needed.
You either have to keep a rolling log, which you can't commit until
the last lock has gone, and which gets replayed for everyone accessing
the file system after that commit, or you have to block the processes
on the write, instead of on the lock. The first is doable, and is
done, but kills performance. The second is just silly. There may be
other possibilities, but without significant performance penalties,
this cannot be done.
> think about why this thread even started.
I know, and this is now very far away from what it started as.
> you have open > that truncates before the lock etc....
> well the reads come in and gets an empty file
>
> these reads are reading from cache not HD
> since the write has not happened yet and
> no flush and buffer shut down occurred.
>
> your above example of the second admin
> coming in also reads from the cache and of course
> if the write did get it's chance to write to HD
> then the cache is flushed and closed down
> so the second person then reads from that .
> so they ALWAYS get the fresh data.
This scheme means that different processes will no longer agree on
what the content of a file is.
imagine a file with records, in order, and a few processes that access
that file, processing the records in order when reading, updating,
appending or deleting records when writing.
process 1 opens a file with a shared lock, reads it
process 2 later opens with an exclusive lock, updates a record,
appends a few, and deletes a few.
This goes into the 'cache', and is allowed under your scheme,
because subsequent opens will get this 'updated' information.
process 3 opens the file, shared, read.
process 1 and 3 start communicating, depending on the contents of the
file.
Oops.
That's a fairly significant race condition.
or do we now also have a file open mode that says 'give me the copy
that is committed to disk only, without any subsequent writes
replayed'? So, apart from the performance penalty, we now have made
the possibility of error on the part of the programmer larger. Not a
gain, IMO.
Martien
--
Martien Verbruggen |
Interactive Media Division | "In a world without fences,
Commercial Dynamics Pty. Ltd. | who needs Gates?"
NSW, Australia |
------------------------------
Date: Wed, 07 Mar 2001 01:38:08 GMT
From: mgjv@tradingpost.com.au (Martien Verbruggen)
Subject: Re: flock and close with empty read strangeness
Message-Id: <slrn9ab480.nts.mgjv@verbruggen.comdyn.com.au>
On Tue, 06 Mar 2001 20:05:58 -0500,
ZepHead <groovyt@erols.com> wrote:
> In article <slrn9a9anu.efn.mgjv@martien.heliotrope.home>,
> mgjv@tradingpost.com.au (Martien Verbruggen) wrote:
>>
>> sysopen FH, $name, O_WRONLY|O_CREAT;
>
> we still have not figured out if we should just
> use sysopen or add to open like Bart? suggested
> using +<< (which I like)
> I mean there has to be reason this was not included
> with open , right? could this just be a simple over sight?
I suspect that C<open> is modelled on fopen(3), and fopen doesn't
provide this mode. C<sysopen> is modelled on open(2), which does
provide a combination of flags that support this. Changing C<open> to
accept this argument would mean you'd need to check for that case
especially, and you'd need to make a system call, instead of a library
call.
I'm not even sure whether it would be possible to implement this on
all platforms Perl runs on. Apart from needing a decent C library,
you'd now also need a decent POSIX library, or a system call that can
do the equivalent of open(2).
Martien
--
Martien Verbruggen |
Interactive Media Division | "In a world without fences,
Commercial Dynamics Pty. Ltd. | who needs Gates?"
NSW, Australia |
------------------------------
Date: Tue, 06 Mar 2001 21:07:45 -0500
From: ZepHead <groovyt@erols.com>
Subject: Re: flock and close with empty read strangeness
Message-Id: <groovyt-0CE37A.21074506032001@virt-reader.news.rcn.net>
In article <9841jp$shd$1@charm.magnus.acs.ohio-state.edu>,
ilya@math.ohio-state.edu (Ilya Zakharevich) wrote:
>
> ?! What were you smoking?
ok
>How can the second admin see the changes
> done by the first one *before* these changes are made?
they are made to cache (until a write can get a lock to flush)
If you read earlier posts I'm comparing greedy system to non greedy
as well.
obviously you can see anything you want
until you take whole thread in context
> Remember, the
> program run by the first admin had no chance to lock the file yet,
moot
the lock you are talking about is needed FOR A FLUSH not for a split cache
store. the write has lock_ex on the cache itself not on the
file (be it hd or read cache) yet until the read is over THEN the write gets
lock_ex on that as well THEN you can flush.
a cache is meant to hold the temp data until
it can be put to hd in this case. The cache
itself is where ALL reads come from now
while waiting for a lock ex and flush.
why do you think some systems talk about filehandles
and the like and not the file itself? think about it.
> not
> speaking of writing anything to it...
The first admin sure did write, he wrote to cache
but yes he did NOT FLUSH yet but this is moot too
since all reads now read from this cache that was marked
by the OS as highest priority. (freshest data) for that
file.
also this SPEEDS UP all reads since the new data is in RAM
right then and there.
do you acutally want an OS to WAIT for the fresh date to be on HD for real
before you can read it? heck no.lets not even get into
the fact that your HD has it's own 2meg + ram cache scheme as well.
you do realize that on some systems the data will never
hit the HD if the new write comes in right behind the first
write and the first write has not flushed yet.
the first cache can be marked as stale etc....
this is how 10,000 read writes can go on so fast
in minute on a 1 box i.e the hd is never hit but a few dozen
times ; )
IMHO of ocurse lol
------------------------------
Date: Tue, 06 Mar 2001 23:38:42 GMT
From: glynFOOdwr@FSCKdeleteEmeD.co.uk (Glyndwr)
Subject: Graphics generation - looking for some advice on module choice
Message-Id: <slrn9aat8d.9ud.glynFOOdwr@glyndwr.dyndns.org>
I'm currently tasked with making a simple animation system. It's for an
artificial life system I'm working on for my dissertation; it has to
take formatted text files and generate very simple animations of them,
just as blobs moving around a square. So, basically, I'm looking to
drive a simple pixmap device.
I need this done sooner rather than later, and the parsing of the input
files etc in Perl is easy. Plus, Perl is the language I can work in
fastest, although C and Java are also possiblities. However, I'm
slightly dubious about the graphics output. A quick scout around CPAN
threw up Graphics::Simple, which looked fairly promising but dies
hideously on my Perl 5.6.0 (default RedHat 7 install) every time I try
and use it. I daresay Perl/Tk will do this too, but I don't know any
Tcl/Tk at the moment so that might be a bit heavyweight.
Can anyone suggest a possible alternative I'm overlooking? Or should I
bite the bullet and do it in Tk?
--
-=G=-
Web: http://www.fscked.co.uk ICQ: 66545073
------------------------------
Date: Wed, 07 Mar 2001 00:31:55 GMT
From: "Andy" <ckzchang02@home.com>
Subject: How to convert morse to ascii using perl?
Message-Id: <%jfp6.19924$V6.75693@news1.gvcl1.bc.home.com>
I am new to perl and am learning how to convert a morse code to acsii. Any
suggestions where to look to get started?
Andy
------------------------------
Date: 06 Mar 2001 18:35:41 -0600
From: Tony Curtis <tony_curtis32@yahoo.com>
Subject: Re: How to convert morse to ascii using perl?
Message-Id: <878zmi5sbm.fsf@limey.hpcc.uh.edu>
>> On Wed, 07 Mar 2001 00:31:55 GMT,
>> "Andy" <ckzchang02@home.com> said:
> I am new to perl and am learning how to convert a morse
> code to acsii. Any suggestions where to look to get
> started?
Go to http://search.cpan.org/ and search for an obvious
string...
hth
t
--
The avalanche has already started.
It is too late for the pebbles to vote.
------------------------------
Date: Tue, 06 Mar 2001 23:09:25 GMT
From: mnc@admin.u.nu (Miguel Cruz)
Subject: Re: Is Perl right for me?
Message-Id: <F6ep6.4306$6A6.2306891@typhoon2.ba-dsg.net>
Gwyn Judd <tjla@guvfybir.qlaqaf.bet> wrote:
> SOME_OTHER_SCRIPT=`head -c 1024 /dev/random`
>
> Like I said, I'm still working on it
If you wait long enough I'm sure it'll turn out fine.
miguel
------------------------------
Date: Tue, 06 Mar 2001 23:27:33 GMT
From: mnc@admin.u.nu (Miguel Cruz)
Subject: Re: Opening files on other Windows machines
Message-Id: <Fnep6.4336$6A6.2315602@typhoon2.ba-dsg.net>
Iain Hosking <iain_hosking@hotmail.com> wrote:
> How do I open a file on another Windows machine?
>
> I can't use a drive mapping as this is a CGI script and it's running in the
> system account.
>
> Microsoft's \\machine\drive\foo\bar syntax doesn't work.
Your CGI context probably doesn't have the proper permissions. This is a
particularly hateful part of Windows and is sure to make you tear out your
hair and want to switch to Unix.
miguel
------------------------------
Date: Wed, 07 Mar 2001 00:07:54 -0000
From: Chris Stith <mischief@velma.motion.net>
Subject: Re: Perl for System V
Message-Id: <taauuq68a7ecb5@corp.supernews.com>
Garrett Groff <groffg@gmgdesign.com> wrote:
> I need to install Perl on System V. I'd like to use the source, but I
> downloaded one compressed source file, containing dozens upon dozens of
> files. Is this really going to compile??? How should I compile it? g++ *.c
> I'm dubious that that is going to work w/o time-consuming revisions to the
> source. Can anyone HELP?!
The source distribution comes with a Makefile and an INSTALL file.
I do believe sVr4 is one of the platforms for which it compiles
with few if any strings attached.
INSTALL has very explicit and short instructions. If you follow
these and it doesn't work, you might want to ask with the text
of whatever errors you encounter.
It takes about 5 steps to install perl, albeit with some substeps
within some of those.
Chris
--
Christopher E. Stith
You can never entirely stop being what you once were. That's
why it's important to be the right person today, and not put
it off till tomorrow. -- Larry Wall, 3rd State of the Onion
------------------------------
Date: 16 Sep 99 21:33:47 GMT (Last modified)
From: Perl-Users-Request@ruby.oce.orst.edu (Perl-Users-Digest Admin)
Subject: Digest Administrivia (Last modified: 16 Sep 99)
Message-Id: <null>
Administrivia:
The Perl-Users Digest is a retransmission of the USENET newsgroup
comp.lang.perl.misc. For subscription or unsubscription requests, send
the single line:
subscribe perl-users
or:
unsubscribe perl-users
to almanac@ruby.oce.orst.edu.
| NOTE: The mail to news gateway, and thus the ability to submit articles
| through this service to the newsgroup, has been removed. I do not have
| time to individually vet each article to make sure that someone isn't
| abusing the service, and I no longer have any desire to waste my time
| dealing with the campus admins when some fool complains to them about an
| article that has come through the gateway instead of complaining
| to the source.
To submit articles to comp.lang.perl.announce, send your article to
clpa@perl.com.
To request back copies (available for a week or so), send your request
to almanac@ruby.oce.orst.edu with the command "send perl-users x.y",
where x is the volume number and y is the issue number.
For other requests pertaining to the digest, send mail to
perl-users-request@ruby.oce.orst.edu. Do not waste your time or mine
sending perl questions to the -request address, I don't have time to
answer them even if I did know the answer.
------------------------------
End of Perl-Users Digest V10 Issue 430
**************************************
