[794] in linux-net channel archive
NFS: Bug in file attribute caching
daemon@ATHENA.MIT.EDU (Herbert Thielen)
Sat Jul 29 15:39:49 1995
From: Herbert Thielen <Herbert.Thielen@lpr.e-technik.tu-muenchen.de>
To: linux-net@vger.rutgers.edu
Date: Fri, 28 Jul 1995 15:37:55 +0200 (MET DST)
I posted this mail a few days ago in the news, but didn't see any
reaction or answer on it. So I'll mail it to this list, perhaps it goes
to the right channels then.
We had here the strange problem that some mails were lost for people who
are working with linux machines. All hosts at our site mount the
/var/spool/mail directory from the server; local delivery is at the mail
server only. So everyone has only one incoming mailbox at the site, and
one can read the own mail on every machine.
We found that the problem arised only when the recipient of the mail had
elm running while the mail server delivered the new mail. Because of
very long file attribute caching by the kernel, elm didn't recognize
that the incoming mailbox at the server changed, and wrote the modified
incoming mailbox back to the server - overwriting the just delivered
mail, so that it was lost.
Unfortunately the mount man pages isn't up to date, but by reading the
mount man pages on other machines and the linux mount sources, we found
that the linux mount understands the nfs options
Option Default value Remark
(seconds)
""""""""""""""""""""""""""""""""""""""""""""""""""""""
acregmin 3 Unused in Kernel (Bug)
acregmax 60
acdirmin 30 Unused in Kernel (Bug)
acdirmax 60
actimeo - sets all other options
The default values are given in seconds an are as in other operationg
systems. To understand what these option do, you should read ``Managing
NFS and NIS'' (O'Reilly), pp. 130 ff. There is written about ``File
Attribute Caching'':
0. NFS caches file attributes on the client side so that every
getattr() and setattr() operation does not have to go all the
way to the NFS server.
1. When a file's attributes are READ, they remain valid on the
client for some MINIMUM PERIOD OF TIME [->acregmin], typically 3
seconds.
2. If the client MODIFIES the file (updating its attributes), that
change is made in the local copy of the attributes and the cache
validity period is extended another MINIMUM TIME SLICE.
3. If the file's attributes REMAIN STATIC for some MAXIMUM PERIOD,
normally 60 seconds [->acregmax], they are flushed from the
cache and written back to the server if they have been modified.
The same works for directory attributes too, but with a greater minimal
time value (see above). We wondered why the bug didn't occur while
reading the mail on non-Linux-machines and took a look to the nfs kernel
sources.
In /usr/src/linux/fs/nfs/dir.c, function nfs_lookup_cache_add(), the
expiration date for a newly added entry is set to acregmax resp.
acdirmax.
So for case 1. above, the entries don't expire before acregmax/acdirmax
instead of acregmin/acdirmin. You can see this effect if you type the
following commands in an NFS mounted directory on Linux:
touch xy
while sleep 1; do date; ll --full-time xy; done
Then login on the server machine, go to the directory and execute
``touch xy''. It will take several seconds until the linux box
recognizes the time change. Probably you won't get the full 60 seconds,
but I got over 20 seconds on several tries (I think this depends on the
NFS cache usage - if it's full, the client will retrieve the data again
from the server).
The optimizations noted in 2. and 3. aren't handled in the linux code,
so there's no other usage of ac{reg,dir}{max,min}.
As an result, if there are often write accesses to the same files from
different machines, you _should_ add the mount option ``noac'' for this
NFS file system.
For other file systems usage, the bug is'nt as severe.
Here's the patch for Linux 1.2.11; as I know the bug is in 1.3.7 yet - I
didn't have newer versions around to take a look.
vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
--- linux/fs/nfs/dir.c.org Mon Jan 23 09:38:29 1995
+++ linux/fs/nfs/dir.c Wed Jul 19 19:17:31 1995
@@ -248,7 +248,7 @@
entry->fhandle = *fhandle;
entry->fattr = *fattr;
entry->expiration_date = jiffies + (S_ISDIR(fattr->mode)
- ? NFS_SERVER(dir)->acdirmax : NFS_SERVER(dir)->acregmax);
+ ? NFS_SERVER(dir)->acdirmin : NFS_SERVER(dir)->acregmin);
}
static void nfs_lookup_cache_remove(struct inode *dir, struct inode *inode,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Regards
Herbert.
