[28060] in Perl-Users-Digest
Perl-Users Digest, Issue: 9424 Volume: 10
daemon@ATHENA.MIT.EDU (Perl-Users Digest)
Tue Jul 4 18:06:02 2006
Date: Tue, 4 Jul 2006 15:05:04 -0700 (PDT)
From: Perl-Users Digest <Perl-Users-Request@ruby.OCE.ORST.EDU>
To: Perl-Users@ruby.OCE.ORST.EDU (Perl-Users Digest)
Perl-Users Digest Tue, 4 Jul 2006 Volume: 10 Number: 9424
Today's topics:
Re: about ssh remote <tadmc@augustmail.com>
ANNOUNCE: OOPS 1.004 - Object Oriented Persistent Store <muir@idiom.com>
Re: Crimson Editor query <VSRawat@Invalid.none>
Re: Crimson Editor query <sherm@Sherm-Pendleys-Computer.local>
Problem with split <r.ted.byers@rogers.com>
Re: Problem with split xhoster@gmail.com
Re: ReadKey/ ReadLine query <VSRawat@Invalid.none>
Re: ReadKey/ ReadLine query <VSRawat@Invalid.none>
Re: ReadKey/ ReadLine query <VSRawat@Invalid.none>
Re: ReadKey/ ReadLine query <DJStunks@gmail.com>
Digest Administrivia (Last modified: 6 Apr 01) (Perl-Users-Digest Admin)
----------------------------------------------------------------------
Date: Tue, 4 Jul 2006 12:39:15 -0500
From: Tad McClellan <tadmc@augustmail.com>
Subject: Re: about ssh remote
Message-Id: <slrneal9u3.9d0.tadmc@magna.augustmail.com>
debbie523 <Deb.Fang@gmail.com> wrote:
> $output=`ssh -l userName serverName echo hello`;
> When I test this line through
> browser, $output is nothing.
>
> But if I execute test12.cgi in command window, it does print "hello"
perldoc -q CGI
What is the correct form of response from a CGI script?
My CGI script runs from the command line but not the browser. (500
Server Error)
How can I get better error messages from a CGI program?
> Is there somebody could tell me why it does not show anything from the
> browser?
We have not been given enough information to help you with that.
--
Tad McClellan SGML consulting
tadmc@augustmail.com Perl programming
Fort Worth, Texas
------------------------------
Date: Tue, 4 Jul 2006 17:53:21 GMT
From: David Muir Sharnoff <muir@idiom.com>
Subject: ANNOUNCE: OOPS 1.004 - Object Oriented Persistent Store
Message-Id: <J1w5GB.1FDy@zorch.sf-bay.org>
You can find this in CPAN at:
http://www.cpan.org/authors/id/MUIR/modules/OOPS-0.1004.tar.gz
-Dave
...CHANGELOG...................................................
2006/07/04 0.1004 - focus on: bugfixes, PostgreSQL schema change, SQLite2 vs SQLite 3
Change the license to GPL.
This release tested on:
perl 5.8.7, PostgreSQL 8.1.4, Ubuntu 6.06LTS (Dapper Drake)
perl 5.8.7, DBD::SQLite2 0.33 (sqlite 2.8.15), Ubuntu 6.06LTS (Dapper Drake)
perl 5.8.7, mysql 4.1.15, Ubuntu 6.06LTS (Dapper Drake)
Not supported (did not pass tests):
perl 5.8.7, DBD::SQLite 1.12 (sqlite 3.2.7), Ubuntu 6.06LTS (Dapper Drake)
New features:
none
Bugfixes:
References to hash keys where the key is a large string now work
(except when using mysql).
Change one column definition in the PostgreSQL schema. This
fixes a PostgreSQL compatability bug and this change is required
whem moving from PostgreSQL 7.* to PostgreSQL 8.*. Reported
by Jon Schindler. http://rt.cpan.org/Ticket/Display.html?id=19723
Bugfix: schema upgrades from 1001 to 1003 weren't always recorded
correctly.
Don't fail tests if Test::MultiFork isn't installed.
SQLite 3 wouldn't auto-initialize.
SQLite 3 gives different error messages and that caused errors
rather than raising a deadlock exception.
Old versions of Data::Compare will recursse infinately. Don't
even try! Unfortunantly Debian (and Ubuntu) include very old
versions of Data::Compare.
Other changes:
Ran ispell on OOPS.pod and made other edits.
Added new tests (misc2.t, big.t, upgrade1003.t, failures.t)
Added a new test harness: supercross7()
Added support for DBD::SQLite2, auto-detect SQLite version.
Dropped ab-loopspeed.t in favor of Time::HiRes.
...POD.........................................................
NAME
OOPS - Object Oriented Persistent Store
SYNOPSIS
use OOPS;
transaction(sub {
$oops = new OOPS
dbi_dsn => $DBI_DSN,
username => $username,
password => $password,
table_prefix => "MY";
$oops->commit;
$oops->{my_1st_bit_of_data} = 'a scalar';
$oops->{my_2nd_bit_of_data} = { A => 'hash' };
$oops->{my_3rd_bit_of_data} = [ qw(An Array) ];
$oops->{my_4rd_bit_of_data} = \'a reference to a scalar, ref, hash, or array' ];
my $old_value = $oops->{multiple}{level}{dereference};
my $object = $oops->load_object($integer_object_id);
my $dbh = $oops->dbh();
$oops->workaround27555($reference);
my $was_virtual = $oops->virtual_object(\%{$oops->{some}{hash}, [ $new_value ]);
});
my $ref = getref(%hash, 'key')
DESCRIPTION
The goal of OOPS is to make perl objects transparently persistent. OOPS
handles deeply nested and cross-linked objects -- even object
hierarchies that are too large to fit in memory and (with a hint)
individual hash tables that are too large for memory. Objects will be
demand-loaded into memory as they are accessed. All changes to your
object hierarchy will be saved with a single commit().
Full transactional consistency is the only operational mode. Either all
of your changes are saved or none of them are. While your program runs,
you will see a consistent view of the data: no other running
transactions will change the data you see. If another transaction
changes data that you are using then at least one of the transactions
must abort. OOPS will die() to abort the transaction.
OOPS maps all perl objects to the same RDBMS schema. No advance schema
definition is required on the part of the user of OOPS. The name of the
package (OOPS) comes from the realization that perl's data model is much
more complicated than I initially understood. Internally, the RDBMS
schema uses four tables: a table of objects, a table of attributes (keys
and values), a table of large attributes that big the normal column
widths, and a table of counters.
At this time, OOPS is expecting a web-like work flow:
create OOPS instance
access some objects
modify some objects
commit
exit
If you need more than one transaction in a program, create more than one
OOPS instance.
To make your data persistent, make a reference to your data from the
OOPS object. To later retrieve your data, simply access it through the
OOPS object.
EXAMPLE PROGRAM
use OOPS;
transaction(sub {
my $oops = new OOPS
dbi_dsn => 'DBI:mysql:database=MY-DATABASE-NAME;host=localhost',
username => 'MY-USERNAME',
password => 'MY-PASSWORD',
table_prefix => "MY-TABLE-PREFIX";
my $p = $oops->{pages}{"/some/path"};
$p->{next} = $oops->{pages}{"/some/other/path"};
$p->{jpgs} = [ read_file("x.jpg"), read_file("y.jpg") ];
$oops->commit;
});
exit;
SUPPORTED DATA TYPES
Perl HASHes, REFs, SCALARs, and ARRAYs are supported. Currently, HASH
keys may not be longer than 255 characters. Class names may not be more
than 128 characters long. References to hash keys and array elements are
supported.
At the current time, large ARRAYs are not efficient. Use HASHes instead
if this matters to you. References to array elements and hash values are
not efficient.
Large HASHes are supported by only loading keys that are accessed.
HASHes, array elements, and REFs are implemented with tie(). ARRAYs are
not currently tie()d because of bugs in perl. Multiple references to the
same scalar are supported. References to array elements and hash values
are supported. Persistent data is reference counted and cycles must be
manually broken to assure de-allocation.
SUPPORTED PLATFORMS
The following RDBMSs are supported in the code:
PostgreSQL OOPS 1.004 has been tested with PostgreSQL version 8.1.4
on Linux. OOPS 1.003 was tested with 7.4.2 and 7.3.5 on
Linux and 7.3.5 on pre-release DragonflyBSD. Somewhere
between 7.4.2 and 8.1.4 PostgreSQL got more strict about
TEXT versus BYTEA types. OOPS 1.003 was using TEXT where
it should have been using BYTEA. Be sure to upgrade OOPS
to 1.004 *before* upgrading PostgreSQL to 8.x.
Biggest issue: PostgreSQL runs the test suite slowly.
mysql OOPS 1.004 has been tested with mysql 4.1.15 on Linux.
OOPS 1.003 was tested with mysql 4.0.16 and 4.0.18 using
InnoDB tables.
The OOPS schema for mysql does not support large (>256
byte) hash keys.
Biggest issue: Mysql 4.1.15 (at least) detects some
deadlocks very slowly.
SQLite DBD::SQLite versions 0.x use SQLite 2.x. DBD::SQLite
versions 1.x use SQLite 3.x. Unfortunately, the 1.x
versions of DBD::SQLite that have been tested with OOPS
have failed to pass. They die consistently a few thousand
transactions in on certain tests. Use DBD::SQLite2
instead.
If you're using OOPS 1.003 and DBD::SQLite 0.x then
upgrade OOPS to 1.004 and switch to DBD::SQLite2. Make
both changes at the same time.
OOPS 1.004 has been tested with DBD::SQLite2 0.33. OOPS
1.003 was tested with DBD::SQLite 0.31. Use DBD::SQLite
1.x at your own risk -- versions 1.11 and 1.12 cannot
pass the OOPS regression tests.
DBD::SQLite2 is not 8-bit clean with respect to "\0".
OOPS uses a DBD::SQLite2 feature to translate binary
nulls. A side-effect is that backslash will be doubled
"\" -> "\\" in DBD::SQLite2 (the extras will be removed
transparently by DBD::SQLite2. OOPS does not use the
"counters" table with DBD::SQLite2 (or DBD::SQLite). As
of 1.004, OOPS doesn't make use of the newer (version
1.x) DBD::SQLite features for handling nulls.
<http://www.hwaci.com/sw/sqlite/>
Biggest issues: SQLite3 fails; SQLite2 leaks memory; both
do database-level locking only.
Perl versions 5.8.2 through 5.8.7 are supported. Prior to 5.8.2, it
wasn't possible to untie scalars from within the a tied scalar access
method. An ugly workaround is possible if there is enough interest.
OOPS 1.004 has been tested on Linux 2.6.15 (Ubuntu 6.06LTS - Dapper
Drake). OOPS 1.003 was tested on Linux 2.4.23 (Debian unstable as of
April '04); on FreeBSD 4.9; and on DragonflyBSD prerelease.
As far as performance goes, mysql and SQLite are both about twice as
fast as PostgreSQL for applications that only have one transaction at at
time. SQLite is particularly slow when there are multiple transactions
as its lock granularity is the entire database. Mysql also slows down
when there are multiple simultaneous transactions. I don't know yet
which back-end is fastest for applications with many simultaneous
transactions.
The RDBMS schema for SQLite is different than for mysql or PostgreSQL
(which are nearly the same as each other).
FUNCTIONS
"transaction($funcref, @args)"
"transaction()" is a wrapper for a complete transaction.
Transactions that fail due to deadlock with other processes will be
re-run automatically.
The first parameter is a reference to a function. Any additional
parameters will be passed as parameters to that function. The return
value of "transaction()" is the return value of "&$funcref()".
It is not necessary to use the "transaction()" method. Beware that
nearly any operation on persistent date (even read operations) can
cause deadlock.
Any use of persistent data can trigger a deadlock. The
"transaction()" function catches this and retries automatically up
to $OOPS::transaction_maxtries times (15 times unless you change
it). If you don't use "transaction()" you might want to catch the
exceptions that "transaction()" catches. To do this, you can regex
match $@ against $OOPS::transfailrx.
Basically, transaction is a slightly fancier version of the
following:
sub transaction
{
my ($sub, @args) = @_;
for (0..15) {
eval {
&$sub(@args);
}
last unless $@;
next if $@ =~ /$OOPS::transfailrx/;
die $@;
}
}
The important thing to notice is that your code will be called over
and over until the transaction succeeds. This means you should write
your code so that it doesn't have any external side effects until
after it calls commit().
"getref(%hash, $key)"
References to tied hash keys are buggy in all perls through 5.8.7
(and beyond?). Use "getref(%hash, $key)" to create your reference to
a tied hash key. See:
<http://rt.perl.org/rt3/Ticket/Display.html?id=27555> and
<http://rt.perl.org/rt3/Ticket/Display.html?id=29224>.
$ref = getref(%hash, $key);
Alternatively, use "$oops->workaround27555($ref)".
Getref() and workaround27555() work around all the perl bugs with
tied hash key references. Failure to use them may result in
unexpected and inconsistent results.
PUBLIC CLASS METHODS
"OOPS->new(%parameters)"
Creates a OOPS object instance. More than one object instance is
allowed at the same time. Instances are unaware of each other.
Making a reference from a persistent object in one instance to a
persistent object in another instance will cause recursive copying
from one instance to the other. (Untested).
The %parameters are documented in the next section.
"OOPS->initial_setup(%parameters)"
Drops and recreates the database tables. Don't use it too often :-)
The regression suite drops and re-creates the tables many times.
The %parameters are documented in the next section.
PARAMETERS FOR NEW
The "new()" and "initial_setup()" methods take a hash specification for
their behavior. Here are the parameters allowed:
"dbi_dsn" / $ENV{OOPS_DSN} / $ENV{OOPS_DRIVER} / $ENV{DBI_DSN} /
"$ENV{DBI_DRIVER"
Many ways to specify the DSN for DBI: as an argument; as an OOPS
environment variable; as a DBI environment variable. Use at least
one :-) See the DBI documentation for more details.
"user" / $ENV{OOPS_USER} / $ENV{DBI_USER}
Three ways to specify the user for DBI: as an argument; as an OOPS
environment variable; as a DBI environment variable. Not required
for all databases.
"password" / $ENV{OOPS_PASS} / $ENV{DBI_PASS}
Three ways to specify the password for DBI: as an argument; as an
OOPS environment variable; as a DBI environment variable. Not
required for all databases.
"table_prefix" / $ENV{OOPS_PREFIX}
OOPS allows a prefix to be supplied for it's internal table names.
If you set a prefix of "FOO_" then it will use a "FOO_object" table
instead of an "object" table. This can be set as an argument to
"new()" or it can be set with the environment variable $OOPS_PREFIX.
This allows multiple separate object spaces to exist within the same
backend SQL database. It's intended use is to support testing vs.
production environments but it could also be used to segregate
object spaces that don't link to each other.
The current tables and indexes are:
object Each row is an object.
attribute Each row is an attribute value in an object.
big Each row is a large attribute value.
counters Counters for things like object ids.
temp A temporary table.
group_index Object grouping index.
value_index Lookup by attribute value.
"auto_upgrade" / $ENV{OOPS_UPGRADE}
The relational schema for OOPS is not same for all versions of OOPS.
If this is set, then an older schema will be upgraded to the current
schema. If this is not set, then OOPS will use the older schema
unchanged. In most cases, this means that OOPS will use a historical
version of itself rather than the current version. See the SCHEMA
VERSIONS section of this document.
"auto_initialize" / $ENV{OOPS_INIT}
The DBMS must be initialized before it can be used: tables created,
a few rows inserted, etc. This can be handled by calling
"initial_setup()" or it can be handled automatically by "new()" if
"auto_initialize" or $OOPS_AUTO_INIT is set.
"default_synchronous" / $ENV{OOPS_SYNC}
With SQLite, an additional parameter to "OOPS->new()" is recognized:
"default_synchronous". Possible values are:
FULL Sync() all transactions to disk before returning.
NORMAL The default: sync() at critical moments only - protects
against program failure, but not all power or OS failures.
OFF Don't sync() at all and go really fast.
PUBLIC OBJECT METHODS
"->commit()"
Writes any changed objects back to the database and commits the
transaction. Currently only one commit() call is allowed. Do not
access your persistent data after commit() -- it may work but this
is not covered well in the regression suite.
"->virtual_object(\%hash [,$new_value])"
Queries [and sets] the load-virtual flag on a persistent hash.
Hashes that load virtual will do separate queries for each key
rather than load the entire hash. This is a good thing if your has
has lots of keys. This flag takes effect the next time the hash is
loaded. The value is a perl boolean.
This may be handled automatically in the future.
"->workaround27555($reference)"
References to tied hash keys are buggy in all perls through 5.8.7
(and beyond?). Use workaround27555($reference) to register your new
tied hash key references so that they can be transformed into
references that actually work correctly.
$ref = \%hash{$key};
$oops->workaround27555($ref);
"workaround27555()" is harmless if called on other sorts of
references so it is safe to use indiscriminately. See
<http://rt.perl.org/rt3/Ticket/Display.html?id=27555>.
Alternatively, use "getref(%hash, $key)".
"->dbh()"
This returns the main DBI database handle used by OOPS. This
function is provided for those who want to hand-write queries.
Please note: no changes are written to the DBMS until commit().
"->load_object($id)"
This will load a persistent object by number. It returns the object
or undef if the object doesn't exists. This function is provided for
those who want to hand-write queries.
ERRATA, DEVELOPMENT STATUS, COMMUNITY, BUGS, AND BUG BOUNTY
OOPS has been thoroughly tested. The regression suite is very well
developed: there is twice as much code in the test suite as there is in
the module itself. The suite does over 1.5million tests. On my fastest
computer it takes over six hours to run. I have so much confidence in my
test suite, I'm offering a bounty on bugs!
Bug Bounty
I'll pay (via paypal) US$5.00 to anyone who submits a new bug (with
regression test) that is caused by a problem in my OOPS module. To
qualify, bugs must be serious: they must cause data corruption, false
results, or program failure. Bugs must not be listed here.
I'll report the number of bounties paid in the CHANGELOG.
Community
There is a mailing list for OOPS. Send a message to
oops-subscribe@idiom.com to subscribe.
Known bugs in OOPS
memory leaks
OOPS currently has memory leaks. This may or may not matter to your
application. The rate of leakage varies depending on which RDBMS is
used. SQLite seems to have the most significant problems. Most of
the leaks are not in OOPS itself but in the modules it uses and thus
are not easily fixed by the OOPS developers.
delayed DESTROY
Additional references to the in-memory copies of persistent data are
kept by OOPS. These extra references will prevent DESTROY methods
from being called as soon as they otherwise would be. They'll
usually be delayed until the OOPS object is itself DESTROYed.
other magic
Other perl magic attributes are not currently stored persistently.
Many probably could be supported, but many could not. For example,
taint does not work on tied hashes:
<http://rt.perl.org/rt3/Ticket/Display.html?id=6758>.
unreferenced blessed scalars
When you bless a reference to a scalar value, the blessing is stored
with the scalar, not the reference. The blessing remains even if
there is no reference to the scalar. The following code prints
"true".
my $x = 'foobar';
my $y = \$x;
bless $y, 'baz';
$y = 7;
$y = \$x;
my $z = ref($y);
print "true\n" if $z eq 'baz';
At the current time, OOPS does not store such blessings. OOPS does
remember blessings when there isn't a reference.
re-blessing the OOPS object
If you re-bless the OOPS object, your data is likely to become
inaccessible. I list this here so nobody claims a bounty for
breaking OOPS in this way.
DBD::Pg and ASCII NULL
DBD::Pg does not easily support ASCII NULL. OOPS has only partial
support for ASCII NULL with PostgreSQL: don't have ASCII NULL in
your class names.
Circular references sometimes break
OOPS mostly allows you to make circular self-references:
my $x;
$x = \$x;
In some (rare) cases circular self-references sometimes break. Since
I can't think on any good reason to make a circular self-reference,
I've left this hanging.
Long hash keys
The database schema for OOPS does not support huge hash keys in
RDBMS backends.
Bugs in perl that effect OOPS
References to hash keys
Persistent hashes are implemented with tie. There are bugs with
perl's implementation of references to tied hash keys. These bugs
will be triggered in several situations: creating a reference to
tied hash key that doesn't exist yet; deleting a key that has a
reference tied to it; assigning through a reference to a key that
has multiple references.
All of the above can either be avoided or you can workaround them by
either calling "workaround27555"($YOUR_REFERENCE) whenever you
create a tied hash key reference or by using "getref(%hash, $key)"
to create your reference.
The perl bugs are documented in:
<http://rt.perl.org/rt3/Ticket/Display.html?id=27555> and
<http://rt.perl.org/rt3/Ticket/Display.html?id=29224>.
"local" and tie
"local(%some_tied_hash)" doesn't work right. Thus
"local(%some_persistent_hash)" won't work right either:
<http://rt.perl.org/rt3/Ticket/Display.html?id=6017>.
"scalar(%hash)"
Tied "scalar(%hash)" support was added in perl 5.8.3 and does not
exist in 5.8.2.
Tied arrays don't work right
There are a couple of bugs with tied arrays that prevent OOPS from
using them: <http://rt.perl.org/rt3/Ticket/Display.html?id=22570>
and <http://rt.perl.org/rt3/Ticket/Display.html?id=22571>. OOPS
fully loads arrays into memory to work-around this problem. This
isn't a big deal unless you've got big arrays.
SQLite and perl's malloc().
If SQLite is used with a perl that has been compiled to use perl's
"malloc()", it will report LOTS of "Bad free() ignored (PERL_CORE)"
errors. It is not currently known if these errors are harmful beyond
generating lots of output to STDERR. The default perl configuration
on FreeBSD uses perl's "malloc()".
SCHEMA VERSIONS
As OOPS is developed, the schema that OOPS uses changes. At the current
time, all versions of OOPS use forward and backwards compatible (though
not identical) schemas.
OOPS notices if the version of the schema in the DBMS is different than
the version the code currently supports. When this happens there are two
possibilities: either OOPS will upgrade the schema to the current
version or it will use an older version of OOPS to access the data. See
the "auto_upgrade" notes in the PARAMETERS FOR NEW section.
This behavior allows the OOPS module to be upgraded without disrupting
installed applications.
FUTURE DIRECTIONS
OOPS isn't done. There are a bunch of things that I am considering
adding to it. If any of these things is important to you, speak up so
that I know there is interest...
fix the bugs
There are bugs listed in the DEVELOPMENT STATUS section that could
be fixed. First up is fixing the memory leaks that are in OOPS
itself.
code cleanup and general performance enhancements
The initial releases of OOPS concentrated on correct behavior and
other aspects of the module were somewhat ignored. The code could be
cleaned up a bunch.
better concurrency
With the transaction model of SERIALIZABLE on mysql, there are
serious performance issues surrounding access to the main hash.
Change to a different transaction model or DBMS.
perl-syntax SQL query translator
SELECT Employee WHERE $Employee->{salary} > 5000
It's possible to translate perl-syntax queries into real SQL that
can be used to query the object store.
better grouping
Objects are loaded in groups rather than individually. There is much
room for improvement in choosing how groups are formed. This is
largely undeveloped as yet.
caching
Many possibilities. A cache-invalidation daemon to note when objects
have changed. Re-verification of touched data from the database.
Ability to call commit() more than once.
weak references
Support for persistent weak references is possible.
external references to objects
Currently objects are reference counted internally. You must have a
reference to something from an already existing object for it to
continue to exist.
contracts
OOPS has to do a lot of scanning of objects to see if they've
changed. Explicit notification of changes would improve performance.
OOPS could call functions before saving and after loading to
transform objects for a better or cleaner on-disk representation.
support for 'base' & accessor methods
This isn't something that I care about but maybe someone else does?
schema enforcement
Allow explicit schemas to be defined. Do not save objects that don't
conform. Eg: Hash::Util::lock_keys().
RDBMS -> object mapping
Map existing RDBMS schemas into objects.
data viewer
Viewing large datasets of deep and cross-linked data is difficult.
Perhaps a CGI-based or Tk-based data navigator would help.
support for tied data structures
It is possible to support storing tied data. The tied object is what
would need to be persistent. This would only work on some kinds of
ties.
garbage collect circular references
Like perl, OOPS uses reference counting to know when to delete an
object. Unlike a normal program, restarting your program does not
clear out the circular references.
support for other base types.
Right now, just HASH, SCALAR, REF, and ARRAY are supported. Regular
expressions, file handles, I don't know it's possible to support
code references.
on-line data migration
By doing double updates, OOPS could support live migration from one
DBMS to another.
WRITING SQL QUERIES BY HAND
If you want to query your data, then until a translator is written, your
only choice for making queries is to write them by hand. Using your data
does not require a query: anything you've got a reference to will be
loaded as you access it. Queries are for performing searches that don't
have a perl-object index.
Each perl HASH, REF/SCALAR, or ARRAY has a row in the "object" table and
multiple rows in the "attribute" and "big" tables.
Here are the columns you'll care about:
object There is one row per perl object.
id The object id.
class The blessed class name (limited to 255
characters).
otype The type of object:
'H' A HASH.
'A' An ARRAY.
'S' A SCALAR or REF.
attribute This is a table of key/value pairs. The keys correspond to
perl hash keys and perl array indexes. The values correspond
to perl hash values and array element values.
id The object id.
pkey The hash key or array index.
pval he hash value or array value. Limited to 255
characters.
ptype Flags the type of the value. Possible values are:
'0' A normal value. Numeric or string.
'B' An big value. "pval" will be a copy of the
start of the value for the first N
characters. The end of "pval" will be a MD5
checksum of the full big value.
'R' A reference to another object.
big This is a table of values that were too large for the normal
columns. Even with databases that support wide columns, a
separate big table is used so that you don't load large
scalars unless you actually need the value.
id The object id.
pkey The hash key or array index.
pval The hash value or array value. Limited to whatever
the underlying database will support as it's
largest blob.
fragno Blob fragment number. This column only exists with
SQLite. SQLite has a smallish maximum row size and
so big values must be split into multiple rows.
REFs are are special. There are several types of REFs: references to
scalar values; references to objects; secondary references to scalar
values; references to scalar values that are part of another object
(references to hash keys and references to array elements).
The representation of references is designed so that you don't need to
care what sort of REF it is when you're doing a query.
The basic REF is a ref to a value inside another object. An example:
OBJECT TABLE
id class otype
1 OOPS::NamedObj H
383 SCALAR R
384 SCALAR R
385 SCALAR R
386 REF R
400 HASH H
500 ARRAY A
ATTRIBUTE TABLE
id pkey pval ptype
1 A500 500 R
1 H400 400 R
1 R383 383 R
1 R384 384 R
1 R385 385 R
1 R386 386 R
383 400 'a-key' 0
384 384 'nopkey' 0
384 'nopkey' 'a-value' 0
385 384 'nopkey' 0
386 386 'nopkey' 0
386 'nopkey' 500 R
400 'a-key' 'a-value 0
400 'another-key' 'another-value' 0
400 'A500' 500 R
500 0 'a-value' 0
500 1 'another-value' 0
HASH 1 is %$oops.
REF 383 is a reference to the key 'a-key' in object #400 (a HASH).
REF 384 is a ref to scalar. It uses two rows to make writing queries
easier.
REF 385 is a duplicate reference to a scalar value. It duplicates REF
384. In behavior, these two REFs should be identical even though they
are represented differently in the database.
REF 386 is a ref to an object: #500 (an ARRAY).
HASH 400 is a normal hash.
ARRAY 500 is a normal hash.
This example data is what you would end up with after running code like:
my $oops = new OOPS
dbi_dsn => 'DBI:mysql:database=MY-DATABASE-NAME;host=localhost',
username => 'MY-USERNAME',
password => 'MY-PASSWORD';
$oops->{A500} = [ 'a-value', 'another-value' ];
$oops->{H400} = {
'a-key' => 'a-value',
'another-key' => 'another-value',
'A500' => $oops->{A500},
};
$oops->{R383} = \$oops->{H400}{'a-key'};
$oops->workaround27555($oops->{R383});
$oops->{R384} = \'a-value';
$oops->{R385} = $oops->{R384};
$oops->{R386} = \$oops->{A500};
$oops->commit;
SQL queries require a bunch of joins to link data structures together.
Here are some examples.
"SELECT Foobar WHERE $Foobar->{xyz} = 'abc'"
SELECT object.id
FROM object, attribute
WHERE object.class = 'Foobar'
AND object.id = attribute.id
AND object.otype = 'H'
AND attribute.pkey = 'xyz'
AND attribute.pval = 'abc'
AND attribute.ptype = '0'
"SELECT Foobar WHERE ${$Foobar->{xyz}} = 'abc'"
This example should show why an automatic translator would be a good
idea...
SELECT ohash.object
FROM object AS ohash,
attribute AS ahash,
object AS oref,
attribute AS aref,
attribute AS target
WHERE ohash.class = 'Foobar'
AND ohash.otype = 'H'
AND ahash.id = ohash.id
AND ahash.pkey = 'xyz'
AND ahash.ptype = 'R'
AND oref.id = ahash.pval
AND oref.otype = 'S' # this is the outer ref
AND oref.id = aref.id
AND aref.pval = target.pkey # here's the reference indirection
AND target.pval = 'abc'
AND target.ptype = '0'
If you construct a query like these examples that return object id's,
then use "$object = $oops->load_object($id)" to load them into memory.
I recommend that hand-written queries be read-only as there are
additional columns that must be kept consistent. For example, the object
table includes a reference count column to handle garbage collection of
the persistent data.
RUNNING THE REGRESSION TEST SUITE
The regression test suite empties and re-creates the persistent store
over and over again. To prevent the accidental erasure of production
data, all of the tests require a special environment variable to be set
$OOPSTEST_DSN. This variable replaces the normal $DBI_DSN or $OOPS_DSN.
Correspondingly there is a $OOPSTEST_USER, $OOPSTEST_PASS, and
$OOPSTEST_PREFIX.
Set these variables to something different than what you use for your
production data!
Most of the tests take a long time to run and are disabled by default.
If you can run the full suite in less than six hours please tell me
about your configuration.
Beware mysql logging. On Debian unstable, the default configuration for
mysql logs every SQL statement. Running the test suite to completion
will generate several gigabytes of log file. Running out of disk space
will cause the tests to fail. On DragonflyBSD (FreeBSD 4.9) the default
mysql configuration includes making replication master logs.
THE COMPETITION
There are a number of other modules that make perl objects persistent.
For each module, I'll provide a sentence or two that explains why I
wrote OOPS rather than using that one.
OOPS does not require any schema definition. Most other persistence
package requires schema definition and allows the nature of the
underlying RDBMS to show in both the kinds of objects you can use and
how you use them. OOPS tries to let you work with perl objects without
concerning yourself about how they'll be stored persistently.
Persistence::Object::Postgres
Uses Data::Dumper to serialize objects into BLOBs. Cannot handle
cross-linking. No query support.
pop Only supports Sybase. Uses a fixed schema defined in XML.
SPOPS
Objects are HASHes, values are scalars only. No nesting.
DBIx::RecordSet
Thin layer on top of DBI. Removes much pain from using DBI. Does not
change basic relationship to data -- still relational.
Object::Transaction
Uses Storable to preserve objects in files. No query support. No
cross-linking. Lightweight. Transactions.
Class::DBI
Must define schema and relationships to RDBMS. Accessor methods.
Doesn't support many2many. Close tie to underlying RDBMS.
Alzabo
Must define schema and relationships as relates to RDBMS. Close tie
to underlying RDBMS. Can help create RDBMS schema. Complex. Can
cache database rows.
Tangram / T2
Must define schema. RDBMS schema is created from perl schema. Can
automatically follow references (T2).
Rose::DB::Object
Must define schema in package files. RDBMS schema is created from
perl schema. Operations are straightforward mappings to SQL.
DBM::Deep
DBM::Deep handles deep and cross-linked data. It does not have
support for queries or transactions.
Pixie
Less transparent. Requires user to remember a cookie to retrieve
objects. Cannot handle cross-linking of objects? No query support?
MLDBM / Tie::MLDBM / MLDBM::Easy / MLDBM::Sync
Uses Data::Dumper or Storable or such to serialize objects and store
them as strings. Lock granularity is the whole file. No query
support. Only one (or two with MLDBM::Easy) level of access is
automated.
ObjStore
Proprietary. Not as transparent as OOPS.
see also
<http://poop.sourceforge.net/> has an overview of options.
LICENSE
This software is available with and without the GPL: please write if you
need a non-GPL license. All submissions of patches must come with a
copyright grant so that David Sharnoff remains able to change the
license on OOPS as he sees fit.
Copyright(C) 2004-2006 David Muir Sharnoff <muir@idiom.com>
This program is free software; you can redistribute it and/or modify it
under the terms of the GNU General Public License as published by the
Free Software Foundation; either version 2 of the License, or (at your
option) any later version.
This program is distributed in the hope that it will be useful, but
WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General
Public License for more details.
You should have received a copy of the GNU General Public License along
with this program; if not, write to the Free Software Foundation, Inc.,
59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
--
------------------------------
Date: Tue, 4 Jul 2006 19:50:04 +0000 (UTC)
From: "V S Rawat" <VSRawat@Invalid.none>
Subject: Re: Crimson Editor query
Message-Id: <xn0eocm0sg9mwa001@nntp.aioe.org>
Sherm Pendley wrote:
> "V S Rawat" <VSRawat@Invalid.none> writes:
>
> > Any method of making Crimson Editor accept input from me?
>
> Do you get the same results from scripts written in Python, Ruby,
> etc.?
Never used that editor for anything other than Perl.
>
> If so, you have a Crimson Editor question, not a Perl question, and
> you'll need to ask in whatever forum is appropriate for that.
The query is indeed Crimson Editor query, but it is having a Perl
interface, hence it is safe to believe that some of the Perl-users
might be using it, and might feel like giving some quick clue.
>
> sherm--
Which editor do you use for Perl?
--
Rawat
------------------------------
Date: Tue, 04 Jul 2006 16:40:09 -0400
From: Sherm Pendley <sherm@Sherm-Pendleys-Computer.local>
Subject: Re: Crimson Editor query
Message-Id: <m2ejx1t87q.fsf@Sherm-Pendleys-Computer.local>
"V S Rawat" <VSRawat@Invalid.none> writes:
> The query is indeed Crimson Editor query, but it is having a Perl
> interface, hence it is safe to believe that some of the Perl-users
> might be using it and might feel like giving some quick clue.
I *did* give you a quick clue. More abrasive folks might have answered with
a glib "did you have a Perl question" or a similarly useless response.
The first step to resolving this issue is as I said - find out if it's a
problem with how perl behaves when running under CE, or if it's a problem
with how CE runs scripts in general. Narrowing down the "problem domain"
in this fashion is the first step to solving any problem.
> Which editor do you use for Perl?
That's not relevant.
sherm--
--
Cocoa programming in Perl: http://camelbones.sourceforge.net
Hire me! My resume: http://www.dot-app.org
------------------------------
Date: 4 Jul 2006 14:18:47 -0700
From: "Ted" <r.ted.byers@rogers.com>
Subject: Problem with split
Message-Id: <1152047927.839090.107130@m73g2000cwd.googlegroups.com>
I have appended a simple test script.
The context is this. There is a directory in which all of the files
have been created with file names consisting of a root, a six digit
date, and a 3 character extension, all separated by a period. I
expected the call to split would give me the three components of the
file name. It doesn't. When I run this scriplet, I find
$file_root,$bad_date, and $fext remain empty and the print statement in
the conditional block gives me the original file name rather than the
root concatenated with the extension (i.e. the result should be, but
isn't, the original file name with the six digit date removed).
Any ideas as to what I missed?
Ted
===============================================
$some_dir = "C:/FVA/data/univeris/univeris0608";
opendir (DIR, $some_dir) || die "can't opendir $some_dir\n";
@fnames = readdir(DIR);
closedir DIR;
$count = @fnames;
print $count;print "\n";
$c = 0;
my %file_names;
for ( $i = 0 ; $i < $count ; ++$i ) {
++$c;
print "$fnames[$i]\n";
($file_root,$bad_date,$fext) = split(/./,$fnames[$i]);
print "file root = $file_root\nbad date = $bad_date\nfile extention =
$fext\n";
if ( length($file_root) > 0) {
$file_names{$fnames[$i]} = "$file_root.$fext";
print $file_names{$fnames[$i]};print "\n\n";
}
}
print "\nThere are $c files in $some_dir.\n";
------------------------------
Date: 04 Jul 2006 21:41:03 GMT
From: xhoster@gmail.com
Subject: Re: Problem with split
Message-Id: <20060704174159.431$mO@newsreader.com>
"Ted" <r.ted.byers@rogers.com> wrote:
> I have appended a simple test script.
>
> The context is this. There is a directory in which all of the files
> have been created with file names consisting of a root, a six digit
> date, and a 3 character extension, all separated by a period.
Split takes a regex. In regexes, an unescaped period matches any
character (other than \n). So you are splitting on every single
character.
> ($file_root,$bad_date,$fext) = split(/./,$fnames[$i]);
Xho
--
-------------------- http://NewsReader.Com/ --------------------
Usenet Newsgroup Service $9.95/Month 30GB
------------------------------
Date: Tue, 4 Jul 2006 19:50:09 +0000 (UTC)
From: "V S Rawat" <VSRawat@Invalid.none>
Subject: Re: ReadKey/ ReadLine query
Message-Id: <xn0eocm4mgf4th002@nntp.aioe.org>
anno4000@zrz.tu-berlin.de wrote:
> V S Rawat <VSRawat@Invalid.none> wrote in comp.lang.perl.misc:
> > 2. The following is not reaching "entered file name: " line, nor is
> > exiting at 0.
> > ----------------
> > use Term::ReadLine;
> > my $term = new Term::ReadLine 'Simple Perl calc';
> > my $prompt = "Input file name (0: Exit)? ";
> > my $OUT = $term->OUT || \*STDOUT;
> > while ( defined ($_ = $term->readline($prompt)) ) {
> > $InFile = eval($_);
> > warn $@ if $@;
> > print $OUT $InFile, "\n" unless $@;
> > $term->addhistory($_) if /\S/;
> > }
> >
> > if ($InFile == "0") {
> > exit;
> > }
> > print "entered file name: ", $InFile, "\n";
>
> Your while loop won't end except when you enter an EOF (^D under
> Unix). So the subsequent statements are never reached.
>
> Put the query for the loop end inside the loop.
>
> Anno
Hmm! not becoming clear to me.
Does the line "while ( defined ($_ = $term->readline($prompt)) ) {" has
some problem? I have copied it as such from the doc.
file:///C:/Program%20Files/Perl/html/lib/Term/ReadLine.html
---------------------start
SYNOPSIS
use Term::ReadLine;
my $term = new Term::ReadLine 'Simple Perl calc';
my $prompt = "Enter your arithmetic expression: ";
my $OUT = $term->OUT || \*STDOUT;
while ( defined ($_ = $term->readline($prompt)) ) {
my $res = eval($_);
warn $@ if $@;
print $OUT $res, "\n" unless $@;
$term->addhistory($_) if /\S/;
}
---------------------end
or, is it general behaviour that a ReadLine terminates only with an
EOF, and not with a Enter key?
thanks.
--
Rawat
------------------------------
Date: Tue, 4 Jul 2006 19:52:38 +0000 (UTC)
From: "V S Rawat" <VSRawat@Invalid.none>
Subject: Re: ReadKey/ ReadLine query
Message-Id: <xn0eocm79giy5f003@nntp.aioe.org>
V S Rawat wrote:
> anno4000@zrz.tu-berlin.de wrote:
>
> > V S Rawat <VSRawat@Invalid.none> wrote in comp.lang.perl.misc:
>
> > > 2. The following is not reaching "entered file name: " line, nor
> > > is exiting at 0.
> > > ----------------
> > > use Term::ReadLine;
> > > my $term = new Term::ReadLine 'Simple Perl calc';
> > > my $prompt = "Input file name (0: Exit)? ";
> > > my $OUT = $term->OUT || \*STDOUT;
> > > while ( defined ($_ = $term->readline($prompt)) ) {
> > > $InFile = eval($_);
> > > warn $@ if $@;
> > > print $OUT $InFile, "\n" unless $@;
> > > $term->addhistory($_) if /\S/;
> > > }
> > >
> > > if ($InFile == "0") {
> > > exit;
> > > }
> > > print "entered file name: ", $InFile, "\n";
> >
> > Your while loop won't end except when you enter an EOF (^D under
> > Unix). So the subsequent statements are never reached.
> >
> > Put the query for the loop end inside the loop.
> >
> > Anno
>
> Hmm! not becoming clear to me.
>
> Does the line "while ( defined ($_ = $term->readline($prompt)) ) {"
> has some problem? I have copied it as such from the doc.
>
> file:///C:/Program%20Files/Perl/html/lib/Term/ReadLine.html
> ---------------------start
> SYNOPSIS
>
> use Term::ReadLine;
> my $term = new Term::ReadLine 'Simple Perl calc';
> my $prompt = "Enter your arithmetic expression: ";
> my $OUT = $term->OUT || \*STDOUT;
> while ( defined ($_ = $term->readline($prompt)) ) {
> my $res = eval($_);
> warn $@ if $@;
> print $OUT $res, "\n" unless $@;
> $term->addhistory($_) if /\S/;
> }
> ---------------------end
>
> or, is it general behaviour that a ReadLine terminates only with an
> EOF, and not with a Enter key?
>
> thanks.
Adding:
doc says:
------------start
readline
gets an input line, possibly with actual readline support. Trailing
newline is removed. Returns undef on EOF.
----------end
wouldn't that mean that EOF should not be used.
--
Rawat
------------------------------
Date: Tue, 4 Jul 2006 20:02:11 +0000 (UTC)
From: "V S Rawat" <VSRawat@Invalid.none>
Subject: Re: ReadKey/ ReadLine query
Message-Id: <xn0eocmfsgvbfb004@nntp.aioe.org>
V S Rawat wrote:
> 2. The following is not reaching "entered file name: " line, nor is
> exiting at 0.
> ----------------
> use Term::ReadLine;
> my $term = new Term::ReadLine 'Simple Perl calc';
> my $prompt = "Input file name (0: Exit)? ";
> my $OUT = $term->OUT || \*STDOUT;
> while ( defined ($_ = $term->readline($prompt)) ) {
> $InFile = eval($_);
> warn $@ if $@;
> print $OUT $InFile, "\n" unless $@;
> $term->addhistory($_) if /\S/;
> }
>
> if ($InFile == "0") {
> exit;
> }
> print "entered file name: ", $InFile, "\n";
> ----------------
I also noticed that when I enter some filename like "temp.txt", it is
diplaying "temptxt" after removing ".".
Also, ending the input with ^Z (EOF for command mode in windows)
doesn't help.
--
Rawat
------------------------------
Date: 4 Jul 2006 13:53:59 -0700
From: "DJ Stunks" <DJStunks@gmail.com>
Subject: Re: ReadKey/ ReadLine query
Message-Id: <1152046439.264415.255960@a14g2000cwb.googlegroups.com>
V S Rawat wrote:
> 1. I am currently doing it like this:
>
> --------------
> do {
> print "Font Name (1: Arjun, 2: Shree709, 3: Shusha, 0: Exit)? \n";
> use Term::ReadKey;
> ReadMode 4; # Turn off controls keys
> while (not defined ($C1_map_opt = ReadKey(-1))) { # No key yet
> }
> if ( $Debug_flag == 1 ) { # $Debug_flag has already been defined
> print "Get key $C1_map_opt\n";
> }
> ReadMode 0; # Reset tty mode before exiting
> } until ( $C1_map_opt >= 0 && $C1_map_opt <= 3 );
> --------------
>
> Is there any method of giving a prompt (e.g. "Font Name (1: Arjun, 2:
> Shree709, 3: Shusha, 0: Exit)? \n") with ReadKey? I mean, the way we
> can do with ReadLine as in:
> ---------
> my $prompt = "Input file name (0: Exit)? ";
> $term->readline($prompt)
use IO::Prompt rather than Term::ReadKey.
-jp
------------------------------
Date: 6 Apr 2001 21:33:47 GMT (Last modified)
From: Perl-Users-Request@ruby.oce.orst.edu (Perl-Users-Digest Admin)
Subject: Digest Administrivia (Last modified: 6 Apr 01)
Message-Id: <null>
Administrivia:
#The Perl-Users Digest is a retransmission of the USENET newsgroup
#comp.lang.perl.misc. For subscription or unsubscription requests, send
#the single line:
#
# subscribe perl-users
#or:
# unsubscribe perl-users
#
#to almanac@ruby.oce.orst.edu.
NOTE: due to the current flood of worm email banging on ruby, the smtp
server on ruby has been shut off until further notice.
To submit articles to comp.lang.perl.announce, send your article to
clpa@perl.com.
#To request back copies (available for a week or so), send your request
#to almanac@ruby.oce.orst.edu with the command "send perl-users x.y",
#where x is the volume number and y is the issue number.
#For other requests pertaining to the digest, send mail to
#perl-users-request@ruby.oce.orst.edu. Do not waste your time or mine
#sending perl questions to the -request address, I don't have time to
#answer them even if I did know the answer.
------------------------------
End of Perl-Users Digest V10 Issue 9424
***************************************
