[290] in The Cryptographic File System users list
Re: crypto filesystem bugs
daemon@ATHENA.MIT.EDU (Robert Stampfli)
Sat Sep 27 22:54:10 2003
From owner-cfs-users@crypto.com Sun Sep 28 02:54:10 2003
Return-Path: <owner-cfs-users@crypto.com>
Delivered-To: cfs-mtg@CHARON.mit.edu
Received: (qmail 9260 invoked from network); 28 Sep 2003 02:54:10 -0000
Received: from mx.crypto.com (207.140.168.138)
by charon.mit.edu with SMTP; 28 Sep 2003 02:54:10 -0000
Received: (from majordomo@localhost)
by MultiHostMXServer (8.9.3/8.9.x4) id WAA02283
for cfs-users-list; Sat, 27 Sep 2003 22:50:14 -0400 (EDT)
Received: from dockmaster.research.att.com (H-135-207-24-155.research.att.com [135.207.24.155])
by MultiHostMXServer (8.9.3/8.9.x4) with ESMTP id WAA24268
for <cfs-users@crypto.com>; Sat, 27 Sep 2003 22:50:12 -0400 (EDT)
Received: from mail-green.research.att.com (mail-green.research.att.com [135.207.30.103])
by dockmaster.research.att.com (8.12.1/8.12.1) with ESMTP id h8S2fe2F022104
for <cfs-users@nsa.research.att.com>; Sat, 27 Sep 2003 22:41:40 -0400 (EDT)
Received: by mail-green.research.att.com (Postfix)
id 6B67F1D79CB; Sat, 27 Sep 2003 22:43:59 -0400 (EDT)
Delivered-To: cfs-users@research.att.com
Received: by mail-green.research.att.com (Postfix, from userid 612)
id 58EB41D79C9; Sat, 27 Sep 2003 22:43:59 -0400 (EDT)
Received: from janus (fpfw.research.att.com [135.207.1.2])
by mail-green.research.att.com (Postfix) with SMTP id 03F4C1D79B8
for <cfs-users@research.att.com>; Sat, 27 Sep 2003 22:43:58 -0400 (EDT)
Received: from mail-red.research.att.com ([192.20.225.110]) by janus; Sat, 27 Sep 2003 22:49:35 -0400 (EDT)
Received: from elektro.cmhnet.org (elektro.com [68.77.181.177])
by mail-red.research.att.com (Postfix) with ESMTP id C54891AB495
for <cfs-users@research.att.com>; Sat, 27 Sep 2003 21:54:10 -0400 (EDT)
Received: from colnet (nuucp@localhost)
by elektro.cmhnet.org (8.12.10/8.12.10/cs) with UUCP id h8S2nYhq011610
for cfs-users@research.att.com; Sat, 27 Sep 2003 22:49:34 -0400 (EDT)
Received: (from res@localhost)
by colnet.cmhnet.org (8.11.7-rsA/8.11.6/res) id h8S2nRO28326
for cfs-users@research.att.com; Sat, 27 Sep 2003 22:49:27 -0400 (EDT)
Date: Sat, 27 Sep 2003 22:49:27 -0400
From: Robert Stampfli <cfs@colnet.cmhnet.org>
To: cfs-users@research.att.com
Subject: Re: crypto filesystem bugs
Message-ID: <20030928024926.GA28297@colnet.cmhnet.org>
References: <20030928003550.GA26703@skaro.cthulhu.dircon.co.uk>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20030928003550.GA26703@skaro.cthulhu.dircon.co.uk>
User-Agent: Mutt/1.4.1i
X-Spam-Status: No, hits=-9.5 required=4.0
tests=BAYES_01,EMAIL_ATTRIBUTION,IN_REP_TO,QUOTED_EMAIL_TEXT,
REFERENCES,REPLY_WITH_QUOTES,USER_AGENT_MUTT
autolearn=ham version=2.55
X-Spam-Level:
X-Spam-Checker-Version: SpamAssassin 2.55 (1.174.2.19-2003-05-19-exp)
Sender: owner-cfs-users@crypto.com
Precedence: bulk
On Sun, Sep 28, 2003 at 01:35:50AM +0100, Digby Tarvin wrote:
> Recently found an annoying bug in CFS (1.3.1) compiled on Linux (SuSE 7.3)
>
> To see the fault:
> 1. Create a test directory in a CFS directory tree.
> 2. Create a bunch of files eg I used
> i=1;while [ $i != 200 ]^Jdo^Jtouch $i^Ji=`expr 1 + $i`^Jdone
>
> 3. compare the output of 'ls' to 'ls *' eg
>
> digbyt@voyager:/home/digbyt/work/secrets/test> ls
> 1 11 120 131 142 153 164 175 186 197 28 39 5 60 71 82 93
> 10 110 121 132 143 154 165 176 187 198 29 4 50 61 72 83 94
> 100 111 122 133 144 155 166 177 188 199 3 40 51 62 73 84 95
> 101 112 123 134 145 156 167 178 189 2 30 41 52 63 74 85 96
> 102 113 124 135 146 157 168 179 19 20 31 42 53 64 75 86 97
> 103 114 125 136 147 158 169 18 190 21 32 43 54 65 76 87 98
> 104 115 126 137 148 159 17 180 191 22 33 44 55 66 77 88 99
> 105 116 127 138 149 16 170 181 192 23 34 45 56 67 78 89
> 106 117 128 139 15 160 171 182 193 24 35 46 57 68 79 9
> 107 118 129 14 150 161 172 183 194 25 36 47 58 69 8 90
> 108 119 13 140 151 162 173 184 195 26 37 48 59 7 80 91
> 109 12 130 141 152 163 174 185 196 27 38 49 6 70 81 92
> digbyt@voyager:/home/digbyt/work/secrets/test> echo *
> 1 10 11 12 13 14 15 16 17 18 19 2 20 21 22 23 24 25 26 27 28 29 3 30 31 32 33 34 35 36 37 38 39 4 40 41 42 43 44 45 46 47 48 49 5 50 51 52 53 54 55 56 57 58 59 6 60 61 62 63 7 8 9
>
> digbyt@voyager:/home/digbyt/work/secrets> rm -r test
> rm: cannot remove directory `test': Input/output error
> digbyt@voyager:/home/digbyt/work/secrets> ls test
> 169 171 173 175 177 179 181 183 185 187 189 191 193 195 197 199
> 170 172 174 176 178 180 182 184 186 188 190 192 194 196 198
> digbyt@voyager:/home/digbyt/work/secrets> rm -r test
> digbyt@voyager:/home/digbyt/work/secrets>
>
> For some reason 'ls' seems to see all the files, but the shell does not,
> and neither does 'rm -r'.
>
> I had noticed that files seemed to be going missing, but it was only
> when I attempted to move the content of a CFS directoy elsewhere that
> I noticed that not all files were copied on the first attempt, and
> that I ended up with more files that I started with.
>
> It only seems to be a problem with directories that have quite a
> few files in them, and all the files are there, but sometimes
> not found by reading the directory. Does seem dangerous never the less.
>
> The same version compiled on BSD/OS does not seem to suffer this. I wonder
> if it is a Linux NFS problem...
>
> Anyone come across this before?
No, but I ran your tests on Solaris 8 w/cfs-1.4.0beta2 and noted
similar results:
1. rm -r test - each time it was run, it removed about half of
the remaining files -- took 5 iterations to remove them all.
2. echo * - unlike your system this does seems to report everything
correctly on Solaris.
Here are the files left after the first rm -r attempt:
ls test
100 108 130 138 158 165 185 192 25 47 69 77 99
101 109 131 14 159 166 186 193 26 48 70 78
102 110 132 15 16 17 187 194 41 49 71 79
103 125 133 153 160 18 188 20 42 50 72 80
104 126 134 154 161 181 189 21 43 51 73 81
105 127 135 155 162 182 19 22 44 52 74 82
106 128 136 156 163 183 190 23 45 53 75 97
107 129 137 157 164 184 191 24 46 54 76 98
Note that they seem to be bypassed in groups of ~14-17 on Solaris.
Interestingly, when I ran the same series of tests on an encrypted
directory mounted from a directory created in /tmp (a Solaris tmpfs
file system), it worked just fine, as well as being blindingly fast.
(There was really no perceptible delay in the script I used to create
the 200 files: --> for i in $(range 1 199)^Jdo^J> $i^Jdone <-- or in
the "rm -r test" command.)
Thus, I wonder if there is a problem with buffer overflow, where
requests are being queued to cfsd faster than it can normally
accomodate them.
Anyway, thanks for bringing this to our attention.
Rob
