[152102] in North American Network Operators' Group
RE: Network Storage
daemon@ATHENA.MIT.EDU (Ian McDonald)
Thu Apr 12 17:18:56 2012
X-StAndrews-MailScanner-From: iam@st-andrews.ac.uk
From: Ian McDonald <iam@st-andrews.ac.uk>
To: "nanog@nanog.org" <nanog@nanog.org>, Maverick <myeaddress@gmail.com>
Date: Thu, 12 Apr 2012 21:18:05 +0000
In-Reply-To: <CA+vWMo6+REhB1gELgtkKgJ3eZ9V2b5rckfC+5OWtHZswgX7erA@mail.gmail.com>
Errors-To: nanog-bounces+nanog.discuss=bloom-picayune.mit.edu@nanog.org
Hi,
You'll need to build an array that'll random read/write upwards of 200MB/s =
if you want to get a semi-reliable capture to disk. That means SSD if you'r=
e very rich, or many spindles (preferably 15k's) in a stripe/ raid10 if you=
're building from your scrap pile. Bear in mind that write cache won't help=
you, as the io isn't going to be bursty, rather a continuous stream.
Another great help is scoping what you're looking for and pre-processing be=
fore writing out only the 'interesting' bits, thus reducing the io requirem=
ent. It does depend what you're trying to do, as headers can be adequate fo=
r many applications.
Aligning your partitions with the physical disk geometry can produce surpri=
sing speedups, as can stripe block size changes, but that's generally empir=
ical, and depends on your workload.
--
ian
-----Original Message-----
From: Maverick
Sent: 12/04/2012, 21:27
To: nanog@nanog.org
Subject: Network Storage
Hello Everyone,
Can you please comment on what is best solution for storing network
traffic. We have been graciously granted access by our network
administrator to capture traffic but the one Tera byte disk space is
no match with the data that we are seeing, so it fills up quickly. We
can't get additional space on the server itself so I am looking for
some external solutions. Can you please suggest something that would
be best for Gbps speeds .
Best,
Ali
