[3870] in arla-drinkers
Re: is cache-only-prefixes an nnpfs limitation?
daemon@ATHENA.MIT.EDU (Adam Megacz)
Sun Apr 9 14:03:12 2006
From arla-drinkers-bounces@stacken.kth.se Sun Apr 09 18:03:12 2006
Return-Path: <arla-drinkers-bounces@stacken.kth.se>
Delivered-To: arla-drinkers-mtg@bloom-picayune.mit.edu
Received: (qmail 871 invoked from network); 9 Apr 2006 18:03:12 -0000
Received: from mx3.kth.se (130.237.48.97)
by charon.mit.edu with SMTP; 9 Apr 2006 18:03:12 -0000
Received: from localhost (localhost.localdomain [127.0.0.1])
by mx3.kth.se (Postfix) with ESMTP id BD6F214069B;
Sun, 9 Apr 2006 20:03:09 +0200 (CEST)
Received: from mx3.kth.se ([127.0.0.1])
by localhost (mx3.kth.se [127.0.0.1]) (amavisd-new, port 10024) with LMTP
id 18172-01-28; Sun, 9 Apr 2006 20:03:05 +0200 (CEST)
Received: from tapas.stacken.kth.se (tapas.stacken.kth.se [130.237.234.140])
by mx3.kth.se (Postfix) with ESMTP id 3E9A2140612;
Sun, 9 Apr 2006 20:03:05 +0200 (CEST)
Received: from tapas.stacken.kth.se (localhost [127.0.0.1])
by tapas.stacken.kth.se (Postfix) with ESMTP id D2335534F3;
Sun, 9 Apr 2006 20:03:04 +0200 (CEST)
X-Original-To: arla-drinkers@tapas.stacken.kth.se
Delivered-To: arla-drinkers@tapas.stacken.kth.se
Received: from brev.stacken.kth.se (brev.stacken.kth.se [130.237.234.84])
by tapas.stacken.kth.se (Postfix) with ESMTP id 7868C534F3
for <arla-drinkers@tapas.stacken.kth.se>;
Sun, 9 Apr 2006 20:03:02 +0200 (CEST)
Received: from mx2.kth.se (mx2.kth.se [130.237.48.98])
by brev.stacken.kth.se (8.12.10/8.12.10) with ESMTP id k39I31la028531
for <arla-drinkers@stacken.kth.se>;
Sun, 9 Apr 2006 20:03:01 +0200 (MET DST)
Received: from localhost (localhost.localdomain [127.0.0.1])
by mx2.kth.se (Postfix) with ESMTP id 972E8140800
for <arla-drinkers@stacken.kth.se>;
Sun, 9 Apr 2006 20:03:01 +0200 (CEST)
Received: from mx2.kth.se ([127.0.0.1])
by localhost (mx2.kth.se [127.0.0.1]) (amavisd-new,
port 10024) with LMTP
id 22010-06-59 for <arla-drinkers@stacken.kth.se>;
Sun, 9 Apr 2006 20:02:58 +0200 (CEST)
Received: from ciao.gmane.org (main.gmane.org [80.91.229.2])
by mx2.kth.se (Postfix) with ESMTP id 6A412140587
for <arla-drinkers@stacken.kth.se>;
Sun, 9 Apr 2006 20:02:58 +0200 (CEST)
Received: from list by ciao.gmane.org with local (Exim 4.43)
id 1FSeFM-00078Z-O2
for arla-drinkers@stacken.kth.se; Sun, 09 Apr 2006 20:02:48 +0200
Received: from megacz.com ([216.237.119.186])
by main.gmane.org with esmtp (Gmexim 0.1 (Debian))
id 1AlnuQ-0007hv-00
for <arla-drinkers@stacken.kth.se>; Sun, 09 Apr 2006 20:02:48 +0200
Received: from megacz by megacz.com with local (Gmexim 0.1 (Debian))
id 1AlnuQ-0007hv-00
for <arla-drinkers@stacken.kth.se>; Sun, 09 Apr 2006 20:02:48 +0200
X-Injected-Via-Gmane: http://gmane.org/
To: arla-drinkers@stacken.kth.se
From: Adam Megacz <megacz@cs.berkeley.edu>
Subject: Re: is cache-only-prefixes an nnpfs limitation?
Date: Sun, 09 Apr 2006 11:03:49 -0700
Organization: Myself
Lines: 73
Message-ID: <x38xqetym2.fsf@nowhere.com>
References: <x3mzf1bugb.fsf@nowhere.com> <lsrpsjxtyim.fsf@kashyyyk.ite.kth.se>
<x33bgskjcm.fsf@nowhere.com> <lsrpsjwsbe8.fsf@kashyyyk.ite.kth.se>
Mime-Version: 1.0
X-Complaints-To: usenet@sea.gmane.org
X-Gmane-NNTP-Posting-Host: megacz.com
X-Home-Page: http://www.megacz.com/
User-Agent: Gnus/5.1006 (Gnus v5.10.6) Emacs/21.4 (gnu/linux)
Cancel-Lock: sha1:HoK4Y3655nfVUQPBLdhpOQDFOxY=
X-Virus-Scanned: by amavisd-new at kth.se
X-Spam-Status: No, hits=-4.9 tagged_above=-200.0 required=5.0 tests=BAYES_00
X-Spam-Level:
X-BeenThere: arla-drinkers@stacken.kth.se
X-Mailman-Version: 2.1.6
Precedence: list
List-Id: Arla discussions <arla-drinkers.stacken.kth.se>
List-Unsubscribe: <https://lists.stacken.kth.se/mailman/listinfo/arla-drinkers>,
<mailto:arla-drinkers-request@stacken.kth.se?subject=unsubscribe>
List-Archive: <http://lists.stacken.kth.se/pipermail/arla-drinkers>
List-Post: <mailto:arla-drinkers@stacken.kth.se>
List-Help: <mailto:arla-drinkers-request@stacken.kth.se?subject=help>
List-Subscribe: <https://lists.stacken.kth.se/mailman/listinfo/arla-drinkers>,
<mailto:arla-drinkers-request@stacken.kth.se?subject=subscribe>
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Sender: arla-drinkers-bounces@stacken.kth.se
Errors-To: arla-drinkers-bounces@stacken.kth.se
X-Virus-Scanned: by amavisd-new at kth.se
Tomas Olsson <tol@stacken.kth.se> writes:
> At the moment I'm using 8k, but it probably won't get any smaller.
That's fine.
Also, I noticed this in docs/caching-in-blocks:
"There is just one reson that one should have blockcache. You want
to edit filer larger then you blockcache"
Actually, I'm most interested in block caching for a different reason
-- I often store large media files in AFS and seek around in them (ie
access starting at random locations in the file). Many of these files
would fit in my disk cache, but moving the entire file over the
network or crowding (almost) everything else out of my local cache
would be bad.
> If you have any thoughts on the subject (protocol?), let me know.
In particular, I'm most curious about the flexibility of the
nnpfs<->localfs interface. If I understand correctly, when used with
arlad, nnpfs never gets file data directly from arlad -- it just asks
arlad to place the file data in a particular file and then let the
kernel know that it's there (the kernel then gets that data out of the
file using the local fs driver). So, when nnpfs is used *outside* the
context of arla:
- Do remote-filesystem file have to map 1:1 exactly to local files?
Or do remote blocks map to local files? Or do remote blocks map
to local blocks? I'm assuming the block-mode caching driver must
have dealt with this issue...
- If it is known that a remote file will only be read once, is it
possible to avoid writing it to the localfs disk?
I tried reading ./nnpfs/include/nnpfs/nnpfs_message.h from the block
branch, and it seems to me that when the userland replies with an
INSTALLDATA message, it tells the kernel not only what localfs file
contains the requested block, but also *at what offset* in the localfs
file the requested remotefs block is located.
- Does this mean that the nnpfs kernel module maintains a big
complex mapping of (localfs_inode,offset)<->(remotefs_vnode,block#)?
- Or does it just cache a subset of this information and rely on
calls back to userland to "remind" it of mappings which it may
have forgotten?
Lastly, is there a separate nnpfs mailing list, or is this the right
place to discuss these questions?
> cached operations, and a huge margin to the OpenAFS client at the time. I
> haven't done any benchmarking since, but a 14x performance gain for cached
> reads is impressive, don't you think :)
Absolutely.
IMHO, the most compelling thing for me is that nnpfs is basically the
only way to:
- create a filesystem driver that runs on both Win32 and *nix
- from a single codebase
- without resorting to NFS/SMB-translators
I think that's pretty neat.
- a
--
PGP/GPG: 5C9F F366 C9CF 2145 E770 B1B8 EFB1 462D A146 C380
_______________________________________________
Arla-drinkers mailing list
Arla-drinkers@stacken.kth.se
https://lists.stacken.kth.se/mailman/listinfo/arla-drinkers
