[3067] in SIPB-AFS-requests
Re: FY98 budgeting and AFS
daemon@ATHENA.MIT.EDU (Mike Whitson)
Wed Jul 15 23:44:28 1998
To: sipb-machine-room@MIT.EDU, sipb-afsreq@MIT.EDU
From: Mike Whitson <mwhitson@MIT.EDU>
Date: 15 Jul 1998 23:44:18 -0400
In-Reply-To: Greg Hudson's message of Wed, 15 Jul 1998 13:29:21 -0400
So, I'm not going to raise an opinion on what the SIPB should do with
its AFS cell; I'm too tired to jump into that writhing mass of worms
that's been taken out of its can.
I will, however, explain (as best I can) ASO's thinking behind getting
these RAIDs, so that y'all can decide how much of that thinking also
applies to SIPB.
- ASO would really like the athena AFS cell to be a
high-reliability service. We really don't like it when a disk dies
and we have to tell users, "Sorry, your thesis was restored from a
two-day-old backup tape; everything else was lost." Yes, users should
keep their own backups of important material, but that doesn't mean we
don't want to try for reliability.
Recently, we've been losing a *lot* of disks, with no warning
and with complete data loss. In almost all of these recent cases,
data loss could have theoretically been prevented by a working RAID.
Yes, this is reactionary thinking. No, I don't think that makes it a
bad idea.
I think SIPB needs to decide what level of reliability it is
trying to achieve. We made a decision to throw lots of money (yes,
folks, these things *are* expensive) at an attempt to make the Athena
cell more reliable, based on the failures we've seen in the past. (We
also just bought some fairly nice new DLT stackers...)
- AFS works best when you can have some amount of redundancy
and replication. (Eggs, basket, dude.) ASO is eventually planning to
have fifteen of these RAIDs in the cell, and that's really the
smallest number we would be comfortable with. (In fact, we were
originally thinking of having about 20 36G servers, and decided
against it because the cost overhead on the RAID controller was so
high.) It may be the case that what works for ASO because of the
scale of the athena cell would not be appropriate for the SIPB cell.
One of the other technologies that ASO was considering (don't
gag, please) was using Sun's Solstice DiskSuite (or an equivalent
software RAID package) to do transparent disk-to-disk mirroring of
data. It turns out to be less expensive per usable gigabyte than any
hardware RAID unit, and it scales down to smaller cells and servers
better, since you don't have the cost overhead of the controller. ASO
decided against it, but it might be something the SIPB wants to
consider, given the characteristics of its cell.
- As someone mentioned, ASO is planning to keep around a spare
unit for replacement parts. SIPB might decide to keep a couple
replacement disks around rather than a full spare unit, which would
perhaps be more cost-effective.
- Part of the reason we chose this RAID is that configuration
is simple. You can either do it from the RAID's front panel, or via a
serial port on the RAID controller. No mucking around with
proprietary vendor-specific software tools to configure it.
- Another part of the reason we chose this RAID is that it
operates as a normal SCSI device (as compared to some others, which
require a proprietary controller card that sits on the machine's bus
and speaks directly to the RAID, and have a certain version of the OS
to run). As long as a vendor doesn't break their SCSI driver, there
shouldn't be worries about the RAID being incompatible with future OS
versions or across hardware platforms.
- The RAID controller will, if you plug it into a machine's
serial port and send it the appropriate command over the serial line,
log all events and alarms up the serial line. Box Hill has a set of
perl libraries and scripts designed to run on a machine connected to
the RAID via serial line, which parses those asynchronous notices and
does whatever you want with them. (They can be syslogged or zephyred
or mailed or paged, or whatever. Very flexible.) So, no, you
wouldn't have to patrol the machine room looking for red LEDs.
- ASO is changing around how the athena-cell backups will
work. We haven't ironed out the exact schedule (Ted McCabe is doing
that), but we know that a) incremental backups will be an integral
part of it, and b) a full dump of one of our RAIDs requires two tapes.
SIPB would need to seriously think about how it does AFS backups and
how this could be made to interact with 72G fileservers, if it
actually wants to go that route.
Anyway, hope that helps.
-wrong mike
