[6341] in s-news-athena
sed, awk, ..., and perl
daemon@ATHENA.MIT.EDU (Colin Goodall)
Wed Feb 1 11:30:25 1995
Date: Wed, 1 Feb 95 10:53:05 EST
From: Colin Goodall <colin@stat.psu.edu>
To: s-news@utstat.toronto.edu
Recent correspondence on s-news about UNIX utilities
(sed, awk, ...) leads me to this reminder about perl.
Doug Bates helped popularize perl among statisticians
a few years ago. Perl is to awk and sed as Splus is
to many other statistical packages. How's that for an ad?
Perl is available on many systems; an archive is
ftp.cis.ohio-state.edu
Here are a couple of perl programs as illustration:
(1) I have a file that I wish to read into S containing comma delimited
fields, except that many of the fields to the right are empty,
and the data manager has not used the same number of commas
on each line. The first line contains a full set of names
for read.table(). Here is the file and a perl script to
pad the rows with commas. Filter the file using
% ppad textfile
----------- cut here for textfile -----------------------------
set1,set2,set3,set4,set5,set6,set7,set8,set9,set10,set11,set12
a,a,,a,,
b,,b,,,,,
c,,c,,,
----------- cut here ------------------------------------------
----------- cut here and make executable ----------------------
#!/usr/local/bin/perl -pl
# ppad: pad with additional empty fields
if ( $. == 1 ) {
$nseps = split(',',$_) - 1;
} else {
($tmp = $_ ) =~ s/[^,]//g;
$_ .= ',' x ($nseps - length($tmp));
}
----------- cut here ------------------------------------------
(2) The next script transposes a text file. Try it on textfile above,
and also pipe the output into ppad.
% ptrans textfile
% ptrans textfile | ppad
(The function can be useful in constructing tables for tex, or in
re-organizing data for read.table().)
----------- cut here and make executable ----------------------
#!/usr/local/bin/perl -nl
# ptrans: transpose text files
$k = 'k00';
foreach $val ( split(',') ) {
$aa{ $k++ } .= ',' . $val;
}
if ( eof ) {
foreach $key ( sort keys %aa ) {
print substr($aa{$key},1);
}
}
----------- cut here ------------------------------------------
These perl scripts can be jazzed up to include, e.g., command line
arguments to specify the input and output field separators. Perl
is fast, and, as it is just 1/6th the size of Splus, reasonable
to start up for repeated small tasks.
Colin Goodall crg@stat.psu.edu
Department of Statistics colin@stat.psu.edu
Pennsylvania State University 814-865-3993
University Park PA 16802 814-863-7114 (FAX)
