[118] in Info-AFS_Redistribution
Re: Large AFS cache
daemon@ATHENA.MIT.EDU (Wallace Colyer)
Sun May 12 18:07:11 1991
Date: Sun, 12 May 91 15:03:10 -0400 (EDT)
From: Wallace Colyer <wally+@andrew.cmu.edu>
To: Keith Gorlen <Keith_Gorlen@alw.nih.gov>, Info-AFS@transarc.com
In-Reply-To: <Yc_jtti0ts0t029Lxg@alw.nih.gov>
We have not really tried caches that large, but we have found one thing
that you should be very carefull about. If your cache tends to be
full you will have a problem with thrashing. This problem has become
much worse in 3.1 because in 3.0 the cache manager would allow you to
go over your cache size and cleanup later. NOw, every time the cache
manager writes to the cache it checks the cache size and throws
things away if needed. We noticed this because our pmaxen were
performing much worse since we upgraded to 3.1. We took out the "bug
fix" that caused this behavior and recompiled libafs.a and performance
got better.
On some of our larger machines with 100 meg caches we set the number
of files so low that the cache can never fill up. The cache cleanup
routines seem to be very slow and block all other activities. One of
the real problems is that the routines that throw away files still
keep the cache at close to 100% utilization. This is
what we use on the 100 meg caches:
/usr/vice/etc/afsd -stat 1000 -files 6000 -daemons 5
Here is what a thrashing cache looks like in 3.1:
Thu May 9 00:01:35 EDT 1991
AFS using 19675 of the cache's available 20000 1K byte blocks.
Thu May 9 00:01:38 EDT 1991
AFS using 19675 of the cache's available 20000 1K byte blocks.
Thu May 9 00:01:41 EDT 1991
AFS using 19675 of the cache's available 20000 1K byte blocks.
Thu May 9 00:01:43 EDT 1991
AFS using 19681 of the cache's available 20000 1K byte blocks.
Thu May 9 00:01:45 EDT 1991
AFS using 19701 of the cache's available 20000 1K byte blocks.
Thu May 9 00:01:48 EDT 1991
AFS using 19963 of the cache's available 20000 1K byte blocks.
Thu May 9 00:01:52 EDT 1991
AFS using 19570 of the cache's available 20000 1K byte blocks.
Thu May 9 00:01:55 EDT 1991
AFS using 19698 of the cache's available 20000 1K byte blocks.
Thu May 9 00:01:59 EDT 1991
AFS using 19890 of the cache's available 20000 1K byte blocks.
Thu May 9 00:02:02 EDT 1991
AFS using 19564 of the cache's available 20000 1K byte blocks.
Thu May 9 00:02:04 EDT 1991
AFS using 19693 of the cache's available 20000 1K byte blocks.
Thu May 9 00:02:06 EDT 1991
AFS using 19822 of the cache's available 20000 1K byte blocks.
Thu May 9 00:02:10 EDT 1991
AFS using 19818 of the cache's available 20000 1K byte blocks.
Thu May 9 00:02:13 EDT 1991
AFS using 19682 of the cache's available 20000 1K byte blocks.
Thu May 9 00:02:16 EDT 1991
AFS using 19804 of the cache's available 20000 1K byte blocks.
Thu May 9 00:02:19 EDT 1991
AFS using 19550 of the cache's available 20000 1K byte blocks.
Thu May 9 00:02:22 EDT 1991
AFS using 19617 of the cache's available 20000 1K byte blocks.
Thu May 9 00:02:24 EDT 1991
AFS using 19892 of the cache's available 20000 1K byte blocks.
Thu May 9 00:02:27 EDT 1991
AFS using 19604 of the cache's available 20000 1K byte blocks.
Thu May 9 00:02:29 EDT 1991
AFS using 19829 of the cache's available 20000 1K byte blocks.
Thu May 9 00:02:32 EDT 1991
AFS using 19866 of the cache's available 20000 1K byte blocks.
Thu May 9 00:02:34 EDT 1991
AFS using 19354 of the cache's available 20000 1K byte blocks.
Thu May 9 00:02:39 EDT 1991
AFS using 19762 of the cache's available 20000 1K byte blocks.
Thu May 9 00:02:42 EDT 1991
AFS using 19338 of the cache's available 20000 1K byte blocks.
Thu May 9 00:02:47 EDT 1991
AFS using 19555 of the cache's available 20000 1K byte blocks.
Thu May 9 00:02:49 EDT 1991
AFS using 19777 of the cache's available 20000 1K byte blocks.
Thu May 9 00:02:52 EDT 1991
AFS using 19526 of the cache's available 20000 1K byte blocks.
Thu May 9 00:02:55 EDT 1991
AFS using 19846 of the cache's available 20000 1K byte blocks.
Thu May 9 00:02:58 EDT 1991
AFS using 19405 of the cache's available 20000 1K byte blocks.
Thu May 9 00:03:02 EDT 1991
AFS using 19725 of the cache's available 20000 1K byte blocks.
Thu May 9 00:03:05 EDT 1991
AFS using 19565 of the cache's available 20000 1K byte blocks.
Thu May 9 00:03:10 EDT 1991
AFS using 19693 of the cache's available 20000 1K byte blocks.
Thu May 9 00:03:13 EDT 1991
AFS using 19566 of the cache's available 20000 1K byte blocks.
Thu May 9 00:03:16 EDT 1991
AFS using 19758 of the cache's available 20000 1K byte blocks.
Thu May 9 00:03:20 EDT 1991
AFS using 19600 of the cache's available 20000 1K byte blocks.
Thu May 9 00:03:22 EDT 1991
AFS using 19920 of the cache's available 20000 1K byte blocks.
Thu May 9 00:03:25 EDT 1991
AFS using 19289 of the cache's available 20000 1K byte blocks.
With the bug fix removed:
Wed May 8 23:55:47 EDT 1991
AFS using 21075 of the cache's available 20000 1K byte blocks.
[Cache guideline deliberately exceeded due to small cache size.]
Wed May 8 23:55:55 EDT 1991
AFS using 19170 of the cache's available 20000 1K byte blocks.
Wed May 8 23:55:59 EDT 1991
AFS using 19170 of the cache's available 20000 1K byte blocks.
Wed May 8 23:56:02 EDT 1991
AFS using 19170 of the cache's available 20000 1K byte blocks.
Wed May 8 23:56:05 EDT 1991
AFS using 19228 of the cache's available 20000 1K byte blocks.
Wed May 8 23:56:08 EDT 1991
AFS using 19580 of the cache's available 20000 1K byte blocks.
Wed May 8 23:56:11 EDT 1991
AFS using 19691 of the cache's available 20000 1K byte blocks.
Wed May 8 23:56:14 EDT 1991
AFS using 19960 of the cache's available 20000 1K byte blocks.
Wed May 8 23:56:19 EDT 1991
AFS using 20081 of the cache's available 20000 1K byte blocks.
[Cache guideline deliberately exceeded due to small cache size.]
Wed May 8 23:56:21 EDT 1991
AFS using 20223 of the cache's available 20000 1K byte blocks.
[Cache guideline deliberately exceeded due to small cache size.]
Wed May 8 23:56:24 EDT 1991
AFS using 20484 of the cache's available 20000 1K byte blocks.
[Cache guideline deliberately exceeded due to small cache size.]
Wed May 8 23:56:27 EDT 1991
AFS using 20608 of the cache's available 20000 1K byte blocks.
[Cache guideline deliberately exceeded due to small cache size.]
Wed May 8 23:56:30 EDT 1991
AFS using 20670 of the cache's available 20000 1K byte blocks.
[Cache guideline deliberately exceeded due to small cache size.]
Wed May 8 23:56:33 EDT 1991
AFS using 20802 of the cache's available 20000 1K byte blocks.
[Cache guideline deliberately exceeded due to small cache size.]
Wed May 8 23:56:36 EDT 1991
AFS using 21059 of the cache's available 20000 1K byte blocks.
[Cache guideline deliberately exceeded due to small cache size.]
Wed May 8 23:56:39 EDT 1991
AFS using 21123 of the cache's available 20000 1K byte blocks.
[Cache guideline deliberately exceeded due to small cache size.]
Wed May 8 23:56:42 EDT 1991
AFS using 21188 of the cache's available 20000 1K byte blocks.
[Cache guideline deliberately exceeded due to small cache size.]
Wed May 8 23:56:45 EDT 1991
AFS using 21252 of the cache's available 20000 1K byte blocks.
[Cache guideline deliberately exceeded due to small cache size.]
Wed May 8 23:56:48 EDT 1991
AFS using 21365 of the cache's available 20000 1K byte blocks.
[Cache guideline deliberately exceeded due to small cache size.]
Wed May 8 23:56:51 EDT 1991
AFS using 21435 of the cache's available 20000 1K byte blocks.
[Cache guideline deliberately exceeded due to small cache size.]
Wed May 8 23:56:54 EDT 1991
AFS using 19698 of the cache's available 20000 1K byte blocks.
Wed May 8 23:56:58 EDT 1991
AFS using 19762 of the cache's available 20000 1K byte blocks.
Wed May 8 23:57:01 EDT 1991
AFS using 20018 of the cache's available 20000 1K byte blocks.
[Cache guideline deliberately exceeded due to small cache size.]
Wed May 8 23:57:04 EDT 1991
AFS using 20146 of the cache's available 20000 1K byte blocks.
[Cache guideline deliberately exceeded due to small cache size.]
Wed May 8 23:57:07 EDT 1991
AFS using 20339 of the cache's available 20000 1K byte blocks.
[Cache guideline deliberately exceeded due to small cache size.]
Wed May 8 23:57:09 EDT 1991
AFS using 20531 of the cache's available 20000 1K byte blocks.
[Cache guideline deliberately exceeded due to small cache size.]
Wed May 8 23:57:12 EDT 1991
AFS using 20723 of the cache's available 20000 1K byte blocks.
[Cache guideline deliberately exceeded due to small cache size.]
Wed May 8 23:57:15 EDT 1991
AFS using 20851 of the cache's available 20000 1K byte blocks.
[Cache guideline deliberately exceeded due to small cache size.]
Wed May 8 23:57:19 EDT 1991
AFS using 21043 of the cache's available 20000 1K byte blocks.
[Cache guideline deliberately exceeded due to small cache size.]
Wed May 8 23:57:23 EDT 1991
AFS using 21235 of the cache's available 20000 1K byte blocks.
[Cache guideline deliberately exceeded due to small cache size.]
Wed May 8 23:57:26 EDT 1991
AFS using 21300 of the cache's available 20000 1K byte blocks.
[Cache guideline deliberately exceeded due to small cache size.]
Wed May 8 23:57:29 EDT 1991
AFS using 21620 of the cache's available 20000 1K byte blocks.
[Cache guideline deliberately exceeded due to small cache size.]
Wed May 8 23:57:32 EDT 1991
AFS using 21812 of the cache's available 20000 1K byte blocks.
[Cache guideline deliberately exceeded due to small cache size.]
Wed May 8 23:57:36 EDT 1991
AFS using 21940 of the cache's available 20000 1K byte blocks.
[Cache guideline deliberately exceeded due to small cache size.]
Wed May 8 23:57:39 EDT 1991
AFS using 22196 of the cache's available 20000 1K byte blocks.
[Cache guideline deliberately exceeded due to small cache size.]
Wed May 8 23:57:42 EDT 1991
AFS using 22388 of the cache's available 20000 1K byte blocks.
[Cache guideline deliberately exceeded due to small cache size.]
Wed May 8 23:57:45 EDT 1991
AFS using 22580 of the cache's available 20000 1K byte blocks.
[Cache guideline deliberately exceeded due to small cache size.]
Wed May 8 23:57:48 EDT 1991
AFS using 22645 of the cache's available 20000 1K byte blocks.
[Cache guideline deliberately exceeded due to small cache size.]
Wed May 8 23:57:51 EDT 1991
If you are doing multiple things at one every time something is written
to the cache you cause all AFS processed to lock until the cache is
cleaned up. This really makes for bad cache manager performance.
-Wallace
