[271] in athena10
Re: Cluster machine reset idea
daemon@ATHENA.MIT.EDU (Kenneth Charles Arnold)
Wed Jun 25 00:22:51 2008
Message-ID: <4861C7EE.3060306@mit.edu>
Date: Wed, 25 Jun 2008 00:22:06 -0400
From: Kenneth Charles Arnold <kcarnold@MIT.EDU>
MIME-Version: 1.0
To: Timothy G Abbott <tabbott@mit.edu>
CC: athena10@mit.edu
In-Reply-To: <Pine.LNX.4.64L.0804301714350.29779@mega-man.mit.edu>
Content-Type: multipart/mixed;
boundary="------------030806020600000500070506"
This is a multi-part message in MIME format.
--------------030806020600000500070506
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
I just had a whack at implementing setup/teardown scripts for running
each user in a LVM-snapshot chroot. See attached.
This is being sent from within such a chroot. Things seem to work,
except for some reason I don't get my krb4 tickets. Getting krb5 to port
over was a bit tricky -- schroot filters the environment.
The basic scheme wrt GDM is using schroot to allow my normal user to
chroot in /etc/gdm/Xsession. The failsafe shell doesn't get the chroot,
though. Is that a bug or a feature?
So I can sudo aptitude install subversion, or whatever. Inside my login,
I'm in full control. I log out (and you manually reap my session with
teardown-chroot, for now), and it's gone. It never was even in the main
system at all.
Now of course I can wreck plenty of havoc on the main system if I have
root in the chroot. So this is only for honest people.
I've already spent too much time on this, so I'd appreciate if someone
else could take a turn at testing. If you don't have an LVM box handy,
set one up; it's kinda cool. (Ubuntu alternate CD.) Or just drop by
Ashdown and check out this setup. You know the root password. You'll
have to login as root, setup-chroot your-username, mv Xsession
Xsession-orig; mv Xsession-test Xsession, edit the hard-coded value
there, and login to gdm.
Regards,
-Ken
Timothy G Abbott wrote:
> One problem that we will probably experience with running Debian-based
> cluster machines is that users will su to root and then apt-get
> install some packages containing programs that they want to run for
> that session. The cluster maintainance code would then have to be
> responsible for removing any such packages cleanly.
>
> I thought of the idea of having (most of) the filesystem tree that you
> see when you login graphically be a chroot containing an LVM snapshot
> of the actual Athena source filesystem, which is then destroyed when
> you log out. Directories that want to survive past the user logging
> out, like /home, /tmp, various parts of /var, etc. would be
> bind-mounted from the source filesystem, and thus preserved when users
> log out.
>
> I would not intend this to be a security measure, but instead a
> mechanism for making it difficult for users to accidentally
> reconfigure cluster machines.
>
> I'm not convinced that this idea doesn't have serious problems, but
> some variation on it might be a good way to support temporarily
> installing software on cluster machines using apt.
>
> -Tim Abbott
--------------030806020600000500070506
Content-Type: text/plain;
name="setup-chroot"
Content-Transfer-Encoding: base64
Content-Disposition: inline;
filename="setup-chroot"
IyEvYmluL2Jhc2gKIyBTZXQgdXAgdGhlIGNocm9vdCBmb3IgYSB1c2VyLgojICBVc2FnZTog
c2V0dXAtY2hyb290IHVzZXJuYW1lCgpzZXQgLWUKCiMgSG93IGJpZyB0byBtYWtlIGVhY2gg
d29ya2luZyBjb3B5CldDX1NJWkU9MkcKCiMgRGlyZWN0b3JpZXMgdG8gYmluZC1tb3VudAoj
IChGSVhNRTogdGhpcyB3YXMganVzdCBhbGwgdGhlIG5vbi1kaXNrIG1vdW50cyB0aGF0IFVi
dW50dSBzZXRzIHVwIGJ5IGRlZmF1bHQuCiMgIE1heWJlIHdlIGRvbid0IHdhbnQgc29tZSBv
ZiB0aGVtLikKQklORF9NT1VOVF9ESVJTPSIvcHJvYyAvc3lzIC9kZXYgL3Zhci9ydW4gL3Zh
ci9sb2NrIC9kZXYvc2htIC9kZXYvcHRzIC9zeXMva2VybmVsL3NlY3VyaXR5IC9hZnMgL21p
dCAvdG1wIgoKdXNlcm5hbWU9IiQxIgp3Y19uYW1lPSJ3Yy0kdXNlcm5hbWUiCiNob3N0bmFt
ZT0iJChob3N0bmFtZSkiCiMgSEFDSyBhcm91bmQga2Nhcm5vbGQncyBtaXN0YWtlIG9uIHRo
aXMgbWFjaGluZToKaG9zdG5hbWU9IkFTSERPV04wNCIKCmx2bV9kZXZpY2U9Ii9kZXYvJHto
b3N0bmFtZX0vJHt3Y19uYW1lfSIKbW91bnRfcG9pbnQ9Ii9tbnQvJHt3Y19uYW1lfSIKCiMg
RmFpbCBpZiB0aGUgTFZNIHNuYXBzaG90IGV4aXN0cy4KaWYgWyAtZSAiJHtsdm1fZGV2aWNl
fSIgXTsgdGhlbgogICAgZWNobyAiRkFJTDogTFZNIHNuYXBzaG90ICR7d2NfbmFtZX0gYWxy
ZWFkeSBleGlzdHMhIgogICAgZWNobyAiUnVuIHRlYXJkb3duLWNocm9vdCAkdXNlcm5hbWUi
CiAgICBleGl0IDEKZmkKCiMgQ3JlYXRlIHRoZSBMVk0gc25hcHNob3QKc3luYwpsdmNyZWF0
ZSAtLXNuYXBzaG90IC0tc2l6ZSAke1dDX1NJWkV9IC0tbmFtZSAiJHt3Y19uYW1lfSIgIi9k
ZXYvJHtob3N0bmFtZX0vcm9vdCIKCiMgTW91bnQgaXQuCm1rZGlyIC1wICIke21vdW50X3Bv
aW50fSIKbW91bnQgIiR7bHZtX2RldmljZX0iICIke21vdW50X3BvaW50fSIKCiMgU2V0IHVw
IGJpbmQgbW91bnRzCmZvciBkaXIgaW4gJEJJTkRfTU9VTlRfRElSUzsgZG8KICAgIG1vdW50
IC0tYmluZCAiJHtkaXJ9IiAiJHttb3VudF9wb2ludH0vJHtkaXJ9Igpkb25lCgojIEFkZCB0
aGUgdXNlciB0byBzdWRvZXJzIGluIHRoZSB0YXJnZXQuCmVjaG8gIiR1c2VybmFtZSBBTEw9
KEFMTCkgQUxMIiA+PiAiJHttb3VudF9wb2ludH0vZXRjL3N1ZG9lcnMiCgojIEFkZCB0aGUg
dXNlciB0byBhbGwgdGhlIGRlZmF1bHQgZ3JvdXBzCmZvciBncm91cCBpbiAkQUREX1RPX0dS
T1VQUzsgZG8KICAgIGNocm9vdCAiJHttb3VudF9wb2ludH0iIGdwYXNzd2QgLWEgIiR7dXNl
cm5hbWV9IiAiJHtncm91cH0iCmRvbmUKCiMgQWRkIHRoZSBjaHJvb3QgdG8gdGhlIHNjaHJv
b3QgY29uZmlnCmNhdCA+PiAvZXRjL3NjaHJvb3Qvc2Nocm9vdC5jb25mIDw8RU9GClske3dj
X25hbWV9XQpkZXNjcmlwdGlvbj1Xb3JraW5nIGNvcHkgZm9yICR1c2VybmFtZQpsb2NhdGlv
bj0ke21vdW50X3BvaW50fQp1c2Vycz0ke3VzZXJuYW1lfQplbnZpcm9ubWVudF9maWx0ZXI9
IiIKRU9G
--------------030806020600000500070506
Content-Type: text/plain;
name="teardown-chroot"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
filename="teardown-chroot"
#!/bin/bash
set -e
username="$1"
wc_name="wc-$username"
#hostname="$(hostname)"
# HACK around kcarnold's mistake on this machine:
hostname="ASHDOWN04"
lvm_device="/dev/${hostname}/${wc_name}"
mount_point="/mnt/${wc_name}"
# Unmount all bind-ed mount-points
mount | grep bind | grep "${mount_point}" | cut -d ' ' -f 3 | sort -r | while read dir; do
umount "${dir}"
done
# Kill all processes still running.
fuser -k "${mount_point}" || true
# Unmount the chroot.
if [ -e "${mount_point}" ]; then
umount "${mount_point}"
rmdir "${mount_point}"
fi
# Kill the LVM snapshot
lvremove "${lvm_device}"
--------------030806020600000500070506
Content-Type: text/plain;
name="Xsession"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
filename="Xsession"
#!/bin/bash
exec schroot -c /mnt/wc-kcarnold -p /etc/gdm/Xsession-orig "$@"
--------------030806020600000500070506--
