[105121] in North American Network Operators' Group
Re: Best utilizing fat long pipes and large file transfer
daemon@ATHENA.MIT.EDU (mdavis@drink-duff.com)
Thu Jun 12 20:19:12 2008
Date: Thu, 12 Jun 2008 20:18:56 -0400
From: mdavis@drink-duff.com
To: Sean Knox <sean@craigslist.org>
In-Reply-To: <4851A53B.4010400@craigslist.org>
Cc: nanog@merit.edu
Errors-To: nanog-bounces@nanog.org
Take a look at some of the stuff from Aspera.
Mark
On Thu, Jun 12, 2008 at 03:37:47PM -0700, Sean Knox wrote:
> Hi,
>
> I'm looking for input on the best practices for sending large files over
> a long fat pipe between facilities (gigabit private circuit, ~20ms RTT).
> I'd like to avoid modifying TCP windows and options on end hosts where
> possible (I have a lot of them). I've seen products that work as
> "transfer stations" using "reliable UDP" to get around the windowing
> problem.
>
> I'm thinking of setting up servers with optimized TCP settings to push
> big files around data centers but I'm curious to know how others deal
> with LFN+large transfers.
>
> thanks,
> Sean
