[600] in Info-AFS_Redistribution
Re: NFS vs. AFS builds
daemon@ATHENA.MIT.EDU (Lyle_Seaman@transarc.com)
Wed Feb 12 13:14:58 1992
Date: Wed, 12 Feb 1992 11:41:09 -0500 (EST)
From: Lyle_Seaman@transarc.com
To: Info-AFS@transarc.com
Cc: ed@pa.dec.com, tafvelin@ce.chalmers.se
In-Reply-To: <9202120746.AA04184@cert2.ce.chalmers.se>
Craig Anderson asks:
> AFS is slower to do builds. A lot slower. Why is this? Why doesn't
> a 200MB cache fix this? Why is the 2nd build no faster than the
> first?
This is wierd. AFS is usually much faster for doing builds. This is
where it should shine. I'll try to make some guesses.
> Real User Sys Comments
> AFS 1st time 7:54.13 2:42.61 3:49.02 ~82% CPU utilizaton
> 2nd time ** 7:54.49 2:42.77 3:48.24 ~82% CPU utilizaton
> NFS 1st time 4:15.01 2:15.38 25.71 ~64% CPU utilizaton
> 2nd time 4:04.51 2:15.29 26.31 ~66% CPU utilizaton
>Below is a comparison on non-compile build performance (target is up to date).
> Time for lm to determine "Target rsdd is up to date.". This is similar
> to the time from execution of "lm" command to start of first compile
> (AFS is much much slower here). AFS took about 4.5 longer than NFS
> (and consumed about 8 times more CPU time):
what is lm? I assume it's some kind of make variant, but without
knowing what it does, we're starting off on the wrong foot...
>
> Real User Sys Comments
> AFS ** 3:54.14 18.78 3:13.05 ~91% CPU utilizaton
> AFS ** 3:44.72 18.99 3:12.77 ~95% CPU utilizaton
>
> NFS 49.40 10.85 14.69 ~51% CPU utilizaton
> NFS 46.73 10.73 14.94 ~55% CPU utilizaton
Sven Tafvelin <tafvelin@ce.chalmers.se> writes:
> The reason may be that the build needs more than 200 Mbytes of files which
> means that it will remove the files soon needed when restarting the
> build.
The logic makes sense, but the time doesn't. I can't fathom a 200+++
build completing in under 10 minutes.
Besides, it's clear that the test to determine whether the target is
up to date is responsible for the lion's share of the difference.
Ed Gould writes:
> Tha answer is almost certainly that AFS synchronizes its writes to
> the server at file close time, waiting for the entire file to be
> written to the server at that time. NFS is synchronous to the
> server on a block-by-block basis, hence it doesn't wait for the
> writes to complete at any point - blocks are flushed to the server
> as required by the client buffer cache. This typically happens in
> parallel to other activity.
This is almost certainly not the answer.
A couple of questions must be answered:
1. You aren't doing this build -on- the AFS server, are you?
2. For this comparison, was the NFS server the same machine as the
AFS server, and under similar load?
3. What is "lm" doing to determine that the target is up-to-date?
4. What settings are you using for the afsd parameters, especially
-files, -dcache and -stat?
5. How large is your cache, really? I mean, when you run "fs
getcacheparms", what is the result?
6. How many files are in this build, including objects, include
files, source code, etc, and how many nodes are there in the
minimal subtree of the directory which would include only them?
7. How many users are there on the client machine at the same time?
8. How many users (network-wide) are trying to access these source &
include files simultaneously?
9. If the answer to #7 is more than one, have the other users ever
tried to do this build, and how recently?
10. How much RAM do you have on your client workstation?
Note that the large amount of time is spent in the kernel,
possibly in AFS code. My guess is that lm is attempting to stat
various files. Stat is slower in AFS than NFS when you must go to
the file server. But AFS caches stat information for 7 to 480
minutes, reducing the number of times you must go to the server.
However, the number of files (and directories) for which stat
information can be cached defaults to only 300. If you have more
than that many hot files, your performance will suffer. Kick
-stat up to 2000, and see what happens. (Still, even lots of
stats shouldn't push the CPU utilization so high -- on the
contrary, if it's being forced to go to the file server all the
time, the CPU utilization should be relatively low. That CPU
utilization bothers me. I've got to know what lm is doing.)
There's a tool that should be in the .... usr/afs.rel/bin directory
called xstat_cm_test. Run that immediately before and after doing the
AFS build, and post the results.
/afs/transarc.com/public/lws/mab.tar.Z is the modified Andrew
benchmark. It includes a couple of README files that tell how to run
it. Give it a whirl under both AFS and NFS and see what happens...
Lyle Transarc 707 Grant Street
412 338 4474 The Gulf Tower Pittsburgh 15219
