[3923] in arla-drinkers
4GB read limit with 0.42 and NetBSD 3.0/i386
daemon@ATHENA.MIT.EDU (Jason White)
Sat May 20 01:18:48 2006
From arla-drinkers-bounces@stacken.kth.se Sat May 20 05:18:48 2006
Return-Path: <arla-drinkers-bounces@stacken.kth.se>
Delivered-To: arla-drinkers-mtg@bloom-picayune.mit.edu
Received: (qmail 25664 invoked from network); 20 May 2006 05:18:48 -0000
Received: from mx2.kth.se (130.237.48.98)
by charon.mit.edu with SMTP; 20 May 2006 05:18:48 -0000
Received: from localhost (localhost.localdomain [127.0.0.1])
by mx2.kth.se (Postfix) with ESMTP id 11F8E1410EB;
Sat, 20 May 2006 07:18:47 +0200 (CEST)
Received: from mx2.kth.se ([127.0.0.1])
by localhost (mx2.kth.se [127.0.0.1]) (amavisd-new, port 10024) with LMTP
id 27639-02-50; Sat, 20 May 2006 07:18:42 +0200 (CEST)
Received: from tapas.stacken.kth.se (tapas.stacken.kth.se [130.237.234.140])
by mx2.kth.se (Postfix) with ESMTP id 28780140DFD;
Sat, 20 May 2006 07:18:42 +0200 (CEST)
Received: from tapas.stacken.kth.se (localhost [127.0.0.1])
by tapas.stacken.kth.se (Postfix) with ESMTP id 6DB92534FC;
Sat, 20 May 2006 07:18:41 +0200 (CEST)
X-Original-To: arla-drinkers@tapas.stacken.kth.se
Delivered-To: arla-drinkers@tapas.stacken.kth.se
Received: from brev.stacken.kth.se (brev.stacken.kth.se [130.237.234.84])
by tapas.stacken.kth.se (Postfix) with ESMTP id 3BB1A534FC
for <arla-drinkers@tapas.stacken.kth.se>;
Sat, 20 May 2006 07:18:39 +0200 (CEST)
Received: from mx3.kth.se (mx3.kth.se [130.237.48.97])
by brev.stacken.kth.se (8.12.10/8.12.10) with ESMTP id k4K5IcU8029009
for <arla-drinkers@stacken.kth.se>;
Sat, 20 May 2006 07:18:38 +0200 (MET DST)
Received: from localhost (localhost.localdomain [127.0.0.1])
by mx3.kth.se (Postfix) with ESMTP id 35A8414057C
for <arla-drinkers@stacken.kth.se>;
Sat, 20 May 2006 07:18:38 +0200 (CEST)
Received: from mx3.kth.se ([127.0.0.1])
by localhost (mx3.kth.se [127.0.0.1]) (amavisd-new,
port 10024) with LMTP
id 17585-01-41 for <arla-drinkers@stacken.kth.se>;
Sat, 20 May 2006 07:18:36 +0200 (CEST)
Received: from mail.menelos.com (menelos.com [209.234.79.200])
by mx3.kth.se (Postfix) with ESMTP id 1A37A1406F9
for <arla-drinkers@stacken.kth.se>;
Sat, 20 May 2006 07:18:36 +0200 (CEST)
Received: by mail.menelos.com with LOCAL (Exim) id 1FhJrE-0000Ba-PW
for arla-drinkers@stacken.kth.se; Sat, 20 May 2006 00:18:32 -0500
Date: Sat, 20 May 2006 00:18:32 -0500
From: Jason White <jdw-arla@menelos.com>
To: arla-drinkers@stacken.kth.se
Subject: 4GB read limit with 0.42 and NetBSD 3.0/i386
Message-ID: <20060520051831.GA289@menelos.com>
Mime-Version: 1.0
Content-Disposition: inline
User-Agent: Mutt/1.4.2.1i
X-pgp-key: 0x5290E477, Keyserver: pgp.mit.edu
X-pgp-fingerprint: A8A2 3FDB AB33 98EB ED74 EDAA F538 9A30 5290 E477
X-Virus-Scanned: by amavisd-new at kth.se
X-Spam-Status: No, hits=-4.9 tagged_above=-200.0 required=5.0 tests=BAYES_00
X-Spam-Level:
X-BeenThere: arla-drinkers@stacken.kth.se
X-Mailman-Version: 2.1.6
Precedence: list
List-Id: Arla discussions <arla-drinkers.stacken.kth.se>
List-Unsubscribe: <https://lists.stacken.kth.se/mailman/listinfo/arla-drinkers>,
<mailto:arla-drinkers-request@stacken.kth.se?subject=unsubscribe>
List-Archive: <http://lists.stacken.kth.se/pipermail/arla-drinkers>
List-Post: <mailto:arla-drinkers@stacken.kth.se>
List-Help: <mailto:arla-drinkers-request@stacken.kth.se?subject=help>
List-Subscribe: <https://lists.stacken.kth.se/mailman/listinfo/arla-drinkers>,
<mailto:arla-drinkers-request@stacken.kth.se?subject=subscribe>
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Sender: arla-drinkers-bounces@stacken.kth.se
Errors-To: arla-drinkers-bounces@stacken.kth.se
X-Virus-Scanned: by amavisd-new at kth.se
NetBSD 3.0/i386
Arla-0.42/0.41/0.40
I have a file just over 4GB which was copied from local disk into afs using
arla-0.42 and the 'cp' command:
-rw-r--r-- 1 14773 wheel 4328521728 May 19 21:56 bigfile
Size in megabytes: 4128
When I try to reverse this procedure and copy the file using 'cp' out of afs
to local disk I hit a 4GB limit:
-rw-r--r-- 1 root wheel 4294770688 May 19 23:02 bigfile-cp-fromafs
Size in megabytes: 4095.81
I have the partial output of 'arladebug almost-all' from just before the 4GB
limit was hit available at:
/afs/iastate.edu/users/07/13/jdwhite/public/arladebug.syslog.gz (82K)
What's interesting is that arlad spews tons of debugging output until the 4GB
limit is reached. At this point the debugging output abruptly stops. Arlad
continues to utilize CPU and network as it did prior to hitting 4GB, yet
nothing beyond 4294770688 bytes is written to the file. The cache
utilization, which had been steadily increasing up to 4GB, drops down to near
zero utilization. In addition, arlad continues to receive data for much
longer than it would take to copy the additional 32MB required to complete
the transfer and arlad must be killed to stop it from chugging along
indefinitely.
I did a 'ktrace' on arlad after it started spinning its wheels and have
placed the output at:
/afs/iastate.edu/users/07/13/jdwhite/public/arlad.after_4G_hit.dump.gz (137M)
In addition, I attempted to copy the file out of AFS using 'arla-cli' and
its 'cp' command. arla-cli copied all 4128M of bigfile, but md5 sums did not
match. The original bigfile contains the string '12345678', repeated. Upon
further inspection, of the copied bigfile, everything up to and
including the first 32MB+64K is correct, but everything afterward is all
NULLs.
-Jason
--
Jason White <jdw-arla@menelos.com> Jabber: jdwhite(jabber.org)
http://www.menelos.com/~jdwhite jason.d.white(gmail.com)
PGP KeyID: 0x5290E477
_______________________________________________
Arla-drinkers mailing list
Arla-drinkers@stacken.kth.se
https://lists.stacken.kth.se/mailman/listinfo/arla-drinkers
