[103233] in North American Network Operators' Group
RE: rack power question
daemon@ATHENA.MIT.EDU (Ben Butler)
Sun Mar 23 13:08:35 2008
Date: Sun, 23 Mar 2008 17:10:55 -0000
From: "Ben Butler" <ben.butler@c2internet.net>
To: <nanog@merit.edu>
Errors-To: owner-nanog@merit.edu
This is a multi-part message in MIME format.
------=_NextPart_000_00F2_01C88D08.DA5CD1B0
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
There comes a point where you cant physically transfer the energy using =
air
any more - not less you wana break the laws a physics captin (couldn't
resist sorry) - to your DX system, gas, then water, then in rack =
(expensive)
cooling, water and CO2. Sooner or later we will sink the hole room in =
oil,
much like they use to do with Cray's.
Alternatively we might need to fit the engineers with crampons, climbing
ropes and ice axes to stop them being blown over by the 70 mph winds in =
your
datacenter as we try to shift the volumes of area necessary to transfer =
the
energy back to the HVAC for heat pump exchange to remote chillers on the
roof.
In my humble experience, the problems are 1> Heat, 2> Backup UPS, 3> =
Backup
Generators, 4> LV/HV Supply to building.
While you will be very constrained by 4 in terms of upgrades unless =
spending
a lot of money to upgrade - the practicalities of 1,2&3 mean that you =
will
have spent a significant amount of money getting to the point where you =
need
to worry about 4.
Given you are not worried about 1, I wonder about the scale of the
application or your comprehension of the problem.
The bigger trick is planning for upgrades of a live site where you need =
to
increase Air con, UPS and Generators.
Economically, that 10,000KW of electricity has to be paid for in =
addition to
any charge for the rack space. Plus margined, credit risked and cash
flowed. The relative charge for the electricity consumption - which has
less about our ability to deliver and cool it in a single rack versus =
the
cost of having four racks in a 2,500KW datacenter and paying for the =
same
amount of electric. Is the racking charge really the significant =
expense
any more.
For the sake of argument, 4 racks at =A32500 pa in a 2500KW datacenter =
or 1
rack at =A310,000 pa in a 10000KW datacenter - which would you rather =
have?
Is the cost of delivering (and cooling) 10000KW to a rack more or less =
than
400% of the cost of delivering 2500KW per rack. I submit that it is =
more
that 400%. What about the hardware - per mip / cpu horse power am I =
paying
more or less in a conventional 1U pizza box format or a high density =
blade
format - I submit the blades cost more in Capex and there is no opex =
saving.
What is the point having a high density server solution if I can only =
half
fill the rack.
I think the problem is people (customers) on the whole don't understand =
the
problem and they can grasp the concept of paying for physical space, but
cant wrap their heads around the more abstract concept of electricity
consumed by what you put in the space and paying for that to come up =
with a
TCO for comparisons. So they simply see the entire hosting bill and
conslude they have to stuff as many processors as possible into the rack
space and if that is a problem is is one for the colo facility to =
deliver at
the same price.
I do find myself increasingly feeling that the current market direction =
is
simply stupid and had far to much input from sales and marketing people.
Let alone the question of is the customers business efficient in terms =
of
the amount of CPU compute power required for their business to generate =
1$
of customer sales/revenue.
Just because some colo customers have cr*ppy business models delivering
marginal benefit for very high computer overheads and an inability to =
pay
for things in a manner that reflects their worth because they are =
incapable
of extracting the value from them. Do we really have to drag the entire
industry down to the lowest common denominator of f*ckwit.
Surly we should be asking exactly is driving the demand for high density
computing and in which market sectors and is this actually the best
technical solution to solve them problem. I don't care if IBM, HP etc =
etc
want to keep selling new shiny boxes each year because they are telling =
us
we need them - do we really? ...?
Kind Regards
Ben
-----Original Message-----
From: owner-nanog@merit.edu [mailto:owner-nanog@merit.edu] On Behalf Of
Valdis.Kletnieks@vt.edu
Sent: 23 March 2008 02:34
To: Patrick Giagnocavo
Cc: nanog@nanog.org
Subject: Re: rack power question
------=_NextPart_000_00F2_01C88D08.DA5CD1B0
Content-Type: application/x-pkcs7-signature;
name="smime.p7s"
Content-Transfer-Encoding: base64
Content-Disposition: attachment;
filename="smime.p7s"
MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIEMTCCBC0w
ggOWoAMCAQICATIwDQYJKoZIhvcNAQEFBQAwgaMxCzAJBgNVBAYTAkdCMREwDwYDVQQIEwhDaGVz
aGlyZTERMA8GA1UEBxMITmFudHdpY2gxFDASBgNVBAoTC1dlc3QgMTYgTHRkMR4wHAYDVQQLExVD
ZXJ0aWZpY2F0ZSBBdXRob3JpdHkxGDAWBgNVBAMTD2NhLndlc3QxNi5jby51azEeMBwGCSqGSIb3
DQEJARYPY2FAd2VzdDE2LmNvLnVrMB4XDTA4MDMxNzE2NDQ1NloXDTE4MDMxNTE2NDQ1NlowgaMx
CzAJBgNVBAYTAkdCMREwDwYDVQQIEwhDaGVzaGlyZTERMA8GA1UEBxMITmFudHdpY2gxGDAWBgNV
BAoTD0MyIEludGVybmV0IEx0ZDEVMBMGA1UECxMMQ0xJRU5UIEVNQUlMMRMwEQYDVQQDEwpCZW4g
QnV0bGVyMSgwJgYJKoZIhvcNAQkBFhliZW4uYnV0bGVyQGMyaW50ZXJuZXQubmV0MIGfMA0GCSqG
SIb3DQEBAQUAA4GNADCBiQKBgQDKZ9DvmN9wAtKmv65tDfv23Hs6oNvyfRdAT3NMIICWzcfy93lx
jfm/uSIg+JE4Km44jE0yagvlWpSxu1P7nAguT87pi7Tezg1XeQUxTcl9CDas6qfXpBnhhF4eD+Qt
Ihl5G1AfPHs81F/BQuLS8XHa4swkHM/WZxfcbFrViKkIIwIDAQABo4IBbTCCAWkwHQYDVR0OBBYE
FAid6DB/0eLgsaD98cWmovn70A04MB8GA1UdIwQYMBaAFPrzL73gBPlDQ21aL+6ET1dO90ZGMG4G
CCsGAQUFBwEBBGIwYDAoBggrBgEFBQcwAYYcaHR0cDovL2NhLndlc3QxNi5jby51ay9vY3NwLzA0
BggrBgEFBQcwAoYoaHR0cDovL2NhLndlc3QxNi5jby51ay93MTZjYWNlcnQucGVtLmNydDA0BgNV
HR8ELTArMCmgJ6AlhiNodHRwOi8vY2Eud2VzdDE2LmNvLnVrL3cxNmNhY3JsLmNybDAMBgNVHRMB
Af8EAjAAMA4GA1UdDwEB/wQEAwIF4DAgBgNVHSUBAf8EFjAUBggrBgEFBQcDAgYIKwYBBQUHAwQw
FAYJYIZIAYb4QgEBAQH/BAQDAgWgMCsGCWCGSAGG+EIBDQQeFhxXZXN0MTYuY28udWsgQ0EgLSBF
bWFpbCBDZXJ0MA0GCSqGSIb3DQEBBQUAA4GBACur56rq0df85N6mNu+AlXBU3BV8r4o4YWjhccTJ
HzwK9KtBvo0yFdKgBffDGPrp/1Xq6oFn2f/5Me/Y05d5lLB1tLu8ktYvE0fa4oLnPP0APLIK9U/M
eLQVDqv/1GU6HbzwpWZbsDg/vcsXUh/FG3yovQ0Wtmx2aVEHxutrM/+tMYID6jCCA+YCAQEwgakw
gaMxCzAJBgNVBAYTAkdCMREwDwYDVQQIEwhDaGVzaGlyZTERMA8GA1UEBxMITmFudHdpY2gxFDAS
BgNVBAoTC1dlc3QgMTYgTHRkMR4wHAYDVQQLExVDZXJ0aWZpY2F0ZSBBdXRob3JpdHkxGDAWBgNV
BAMTD2NhLndlc3QxNi5jby51azEeMBwGCSqGSIb3DQEJARYPY2FAd2VzdDE2LmNvLnVrAgEyMAkG
BSsOAwIaBQCgggKWMBgGCSqGSIb3DQEJAzELBgkqhkiG9w0BBwEwHAYJKoZIhvcNAQkFMQ8XDTA4
MDMyMzE3MTA1NFowIwYJKoZIhvcNAQkEMRYEFHYrNLA7BN83z7a5CcEkXzueJQ3iMFIGCyqGSIb3
DQEJEAIBMUMwQQQdAAAAABAAAADtI59kGSXRRqn/PSrt8JxNAQAAAACAAQAwHTAbgRliZW4uYnV0
bGVyQGMyaW50ZXJuZXQubmV0MGcGCSqGSIb3DQEJDzFaMFgwCgYIKoZIhvcNAwcwDgYIKoZIhvcN
AwICAgCAMA0GCCqGSIb3DQMCAgFAMAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMAcGBSsOAwIaMAoG
CCqGSIb3DQIFMIG6BgkrBgEEAYI3EAQxgawwgakwgaMxCzAJBgNVBAYTAkdCMREwDwYDVQQIEwhD
aGVzaGlyZTERMA8GA1UEBxMITmFudHdpY2gxFDASBgNVBAoTC1dlc3QgMTYgTHRkMR4wHAYDVQQL
ExVDZXJ0aWZpY2F0ZSBBdXRob3JpdHkxGDAWBgNVBAMTD2NhLndlc3QxNi5jby51azEeMBwGCSqG
SIb3DQEJARYPY2FAd2VzdDE2LmNvLnVrAgEyMIG8BgsqhkiG9w0BCRACCzGBrKCBqTCBozELMAkG
A1UEBhMCR0IxETAPBgNVBAgTCENoZXNoaXJlMREwDwYDVQQHEwhOYW50d2ljaDEUMBIGA1UEChML
V2VzdCAxNiBMdGQxHjAcBgNVBAsTFUNlcnRpZmljYXRlIEF1dGhvcml0eTEYMBYGA1UEAxMPY2Eu
d2VzdDE2LmNvLnVrMR4wHAYJKoZIhvcNAQkBFg9jYUB3ZXN0MTYuY28udWsCATIwDQYJKoZIhvcN
AQEBBQAEgYAnqIrEDQoWsGL3LV8OMhvUQhuyistt629JVFHIgi9SrCcSg2qs5sPJcdAXe+jSns/D
AltHqsuKfL32hvpg1oCc03fcWq4w0+c25gRvVYq7fvqHRtFPsCJp4TTutRV1rvmCA2ep2YOw7wNu
3ZGJSBYFqh6n6btzpfy1zTNYGgwFYQAAAAAAAA==
------=_NextPart_000_00F2_01C88D08.DA5CD1B0--
