[130536] in North American Network Operators' Group
Re: Anyone can share the Network card experience
daemon@ATHENA.MIT.EDU (Greg Whynott)
Tue Oct 5 11:23:26 2010
From: Greg Whynott <Greg.Whynott@oicr.on.ca>
To: Heath Jones <hj1980@gmail.com>
Date: Tue, 5 Oct 2010 11:23:07 -0400
In-Reply-To: <AANLkTimu8WrN8UerkM+A9Xo_YMu03JuybD_d-jbJs3iN@mail.gmail.com>
Cc: nanog list <nanog@nanog.org>
Errors-To: nanog-bounces+nanog.discuss=bloom-picayune.mit.edu@nanog.org
Hi,
most of our traffic is heading directly into memory, not hitting the local=
disks, on the HPC end of things. Our file servers are feeding the netwo=
rk with around 24 x 10Gibit (active/active clusters), and regularly run =
at over 80 percent on all ports during runs.. this is all HPC / file move=
ment traffic. we have instruments which generate over 6TB of data per run=
, every 3 days, 7/365. we have about 20 of these instruments. so mo=
st of the data on 10Gbit is indeed static, or to/from a file server to/fro=
m HPC clusters. =20
iSCSI we run on its own network hardware, autonomous from the 'data' netw=
ork. its not in wide deployment here, only the file server is connected =
via 10Gbit, the hosts using iSCIS (predominately KVM and Vmware clusters) =
are being feed over multiple 1Gbit links for their iSCIS requirements.
Our external internet servers are connected to the internet via 1Gbit links=
, not 10Gibt, but apparently that is coming next year. The type of traff=
ic they'll see will not be very chatty/interactive. it'll be researchers d=
ownloading data sets ranging in size from a few hundred megs, to a few TB..=
=20
take care,
-g
On Oct 5, 2010, at 10:59 AM, Heath Jones wrote:
>> For 10Gbit we use Intel cards for production service machines, and Conn=
extX/Intel in the HPC cluster.
>=20
> Greg - I've not been exposed to 10G on the server side..
> Does the server handle the traffic load well (even with offloading) -
> that's a LOT of web requests / app queries per second!
>=20
> Or are you using 10G mainly for iSCSI / file serving / static content?
>=20
> Cheers
