[4170] in arla-drinkers
Re: Arla on FreeBSD
daemon@ATHENA.MIT.EDU (Tomas Olsson)
Thu Feb 15 08:20:52 2007
From arla-drinkers-bounces@stacken.kth.se Thu Feb 15 13:20:52 2007
Return-Path: <arla-drinkers-bounces@stacken.kth.se>
Delivered-To: arla-drinkers-mtg@bloom-picayune.mit.edu
Received: (qmail 1675 invoked from network); 15 Feb 2007 13:20:52 -0000
Received: from mx3.kth.se (130.237.48.97)
by charon.mit.edu with SMTP; 15 Feb 2007 13:20:52 -0000
Received: from localhost (localhost.localdomain [127.0.0.1])
by mx3.kth.se (Postfix) with ESMTP id E19A7140838;
Thu, 15 Feb 2007 14:20:50 +0100 (CET)
Received: from mx3.kth.se ([127.0.0.1])
by localhost (mx3.kth.se [127.0.0.1]) (amavisd-new, port 10024) with LMTP
id 18474-01-44; Thu, 15 Feb 2007 14:20:48 +0100 (CET)
Received: from tapas.stacken.kth.se (tapas.stacken.kth.se [130.237.234.140])
by mx3.kth.se (Postfix) with ESMTP id 1EB9E14082E;
Thu, 15 Feb 2007 14:07:30 +0100 (CET)
Received: from tapas.stacken.kth.se (localhost [127.0.0.1])
by tapas.stacken.kth.se (Postfix) with ESMTP id A13685350F;
Thu, 15 Feb 2007 14:07:28 +0100 (CET)
X-Original-To: arla-drinkers@tapas.stacken.kth.se
Delivered-To: arla-drinkers@tapas.stacken.kth.se
Received: from brev.stacken.kth.se (brev.stacken.kth.se [130.237.234.84])
by tapas.stacken.kth.se (Postfix) with ESMTP id 120395350F
for <arla-drinkers@tapas.stacken.kth.se>;
Thu, 15 Feb 2007 14:07:23 +0100 (CET)
Received: from mx2.kth.se (mx2.kth.se [130.237.48.98])
by brev.stacken.kth.se (8.12.10/8.12.10) with ESMTP id l1FD7NOA006176
for <arla-drinkers@stacken.kth.se>;
Thu, 15 Feb 2007 14:07:23 +0100 (MET)
Received: from localhost (localhost.localdomain [127.0.0.1])
by mx2.kth.se (Postfix) with ESMTP id EA0B2140810
for <arla-drinkers@stacken.kth.se>;
Thu, 15 Feb 2007 14:07:22 +0100 (CET)
Received: from mx2.kth.se ([127.0.0.1])
by localhost (mx2.kth.se [127.0.0.1]) (amavisd-new,
port 10024) with LMTP
id 17148-02-57; Thu, 15 Feb 2007 14:07:20 +0100 (CET)
Received: from kashyyyk.ite.kth.se (kashyyyk.ite.kth.se [130.237.31.35])
by mx2.kth.se (Postfix) with ESMTP id 11A70140734;
Thu, 15 Feb 2007 14:07:20 +0100 (CET)
Received: by kashyyyk.ite.kth.se (Postfix, from userid 18404)
id EF4F289CA62; Thu, 15 Feb 2007 14:07:19 +0100 (CET)
From: Tomas Olsson <tol@it.su.se>
To: Kostik Belousov <kostikbel@gmail.com>
Subject: Re: Arla on FreeBSD
References: <eqsut3$6a3$1@sea.gmane.org>
<20070213192906.U726@chrishome.localnet>
<20070214162938.GA96725@keira.kiwi-computer.com>
<20070214173211.L1054@chrishome.localnet>
<20070214170808.GC96725@keira.kiwi-computer.com>
<lsrvei45pq1.fsf@kashyyyk.ite.kth.se>
<20070215044707.GA39168@deviant.kiev.zoral.com.ua>
<lsrmz3f680v.fsf_-_@kashyyyk.ite.kth.se>
<20070215104537.GC39168@deviant.kiev.zoral.com.ua>
<lsrhctn6393.fsf@kashyyyk.ite.kth.se>
<20070215120855.GE39168@deviant.kiev.zoral.com.ua>
Date: 15 Feb 2007 14:07:19 +0100
In-Reply-To: <20070215120855.GE39168@deviant.kiev.zoral.com.ua>
Message-ID: <lsrd54b603c.fsf@kashyyyk.ite.kth.se>
Lines: 53
User-Agent: Gnus/5.09 (Gnus v5.9.0) Emacs/21.3
MIME-Version: 1.0
X-Virus-Scanned: by amavisd-new at kth.se
X-Spam-Status: No, hits=-4.9 tagged_above=-200.0 required=5.0 tests=BAYES_00
X-Spam-Level:
Cc: freebsd-fs@freebsd.org, arla-drinkers@stacken.kth.se
X-BeenThere: arla-drinkers@stacken.kth.se
X-Mailman-Version: 2.1.6
Precedence: list
List-Id: Arla discussions <arla-drinkers.stacken.kth.se>
List-Unsubscribe: <https://lists.stacken.kth.se/mailman/listinfo/arla-drinkers>,
<mailto:arla-drinkers-request@stacken.kth.se?subject=unsubscribe>
List-Archive: <http://lists.stacken.kth.se/pipermail/arla-drinkers>
List-Post: <mailto:arla-drinkers@stacken.kth.se>
List-Help: <mailto:arla-drinkers-request@stacken.kth.se?subject=help>
List-Subscribe: <https://lists.stacken.kth.se/mailman/listinfo/arla-drinkers>,
<mailto:arla-drinkers-request@stacken.kth.se?subject=subscribe>
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Sender: arla-drinkers-bounces@stacken.kth.se
Errors-To: arla-drinkers-bounces@stacken.kth.se
X-Virus-Scanned: by amavisd-new at kth.se
Kostik Belousov <kostikbel@gmail.com> writes:
> On Thu, Feb 15, 2007 at 12:59:04PM +0100, Tomas Olsson wrote:
> > Kostik Belousov <kostikbel@gmail.com> writes:
> > > I made really quick look at the places you mentioned. I have some
> > > comment for open_file(). For FreeBSD >= 6.x, the right way to open vnode
> > > from the kernel code is to use vn_open() (and then vn_close()) API.
> > >
> > Great! Sounds reasonable.
> >
> > We currently open the cache files from nnpfs' VOPs, are there any risks
> > (deadlock?) involved if one passes an absolute path to vn_open() in such a
> > context? I'd have liked to do use arlad's thread for this, but vput()
> There, you already have nnpfs vnode locked. The right lock order for vnodes
> is from root down by the tree. As such, you may end up with reversals, that
> would result in deadlocks, IMHO.
>
Ok, vn_open() must be passed curthread so we can't use arlad's thread when
in our VOP. And we cannot use an absolute path to the cache. So vn_open()
can't be used?
> > explicitly uses curthread deep down in namei. Also, users are not normally
> > allowed to access the cache files directly so some OSes complain on such a
> > lookup with user creds; would that be a problem here?
> How is the user access to cache is disabled ? And what is the cache itself ?
> Local filesystem (UFS) that stores your blocks in regular files ?
>
It's just a local dir tree, on UFS or whatever. Currently each node gets
it's own subdir, and each "block" (128kB perhaps) a plain file in that dir.
The cache is supposed to be readable only by arlad (usually root) and
through nnpfs; that's good when handling fancy ACLs etc, so the cache root
is chmod:ed to 0700 for root:wheel.
> > Of course, we wouldn't have to worry about such things if we just kept the
> > vnode handy for each cache block file. Maybe it's a price worth paying.
>
> Then, you need to take some care of cached vnode lifecircle (e.g., even
> keeping the vnode vref'ed would not prevent it from being recycled, so you
> may end with dead vnode).
>
Eep. Tricky.
So where does this leave us, is plain lookup() (or VOP_LOOKUP) and
VOP_CREATE on the cache a possible way to go? Seems to work on OpenBSD.
> Also, as Robert pointed out in his email, you probably need to decide about
> MP-safeness of nnpfs.
>
Well, it's marked "doesn't need giant"; we use our own global lock. I don't
trust it 100%, but it seems to work so far. I haven't tried it on any MP
FreeBSD boxes though.
thanks
/t
_______________________________________________
Arla-drinkers mailing list
Arla-drinkers@stacken.kth.se
https://lists.stacken.kth.se/mailman/listinfo/arla-drinkers
