[625] in Moira
Moira reconstruction
daemon@ATHENA.MIT.EDU (Mark Rosenstein)
Wed Jun 30 11:16:38 1993
Date: Wed, 30 Jun 93 11:16:19 -0400
From: Mark Rosenstein <mar@MIT.EDU>
To: bug-moira@Athena.MIT.EDU, ops@Athena.MIT.EDU
Here's a recap for those who are interested:
Following the power glitch, someone restarting machines in the
basement rebooted Moira even though it was on a UPS and should have
been working fine. During this reboot, fsck removed a couple of files
owned by ingres, and the ingres database recovery process marked the
database as damaged and unrecoverable.
The Ingres manuals claim there is nothing that can be done in this
case, we need to follow up with Ingres to determine if this is the
case, and why things failed so badly in the first place. I have saved
a copy of the bad database and log files in /u1/baddb.
A new database was created, empty tables created, and then mrrestore
run to populate it from the backup created at 6am that morning. This
failed the first time because I had failed to propagate some fixes to
this program from moiradev to opssrc. I started it again yesterday
evening, and it ran all night. At 6am this morning, I ran the script
to rearrange tables and create indexes, and at 9am started a dbck. By
10am the database was back online.
Then I ran forward the changes between 6am yesterday and 9:55 when the
system was rebooted. These were done by copying lines from the
journal file to mrtest, and then going into the database with SQL to
get the modification information correct.
Everything is now back online, and we can run a special update during
the day today if we need to get any hesiod changes out.
-Mark
