[103226] in North American Network Operators' Group
RE: rack power question
daemon@ATHENA.MIT.EDU (John Lee)
Sun Mar 23 01:46:45 2008
Date: Sun, 23 Mar 2008 00:34:41 -0500
From: "John Lee" <John@internetassociatesllc.com>
To: "Patrick Giagnocavo" <patrick@zill.net>, <nanog@nanog.org>
Errors-To: owner-nanog@merit.edu
This is a multi-part message in MIME format.
------_=_NextPart_001_01C88CA7.97C4E6F0
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
What is the purpose of the datacenter computing, datacom/telco or both. =
AC or DC power feeds or both, backup power or naked, dual feed from the =
power company with transfer switch or power with generator backup? Are =
you dual feeding the racks? Do you require NEBs compliant racks to make =
it through a shake and bake ( a seismic event).=20
=20
Many centers run DC 50 volt multi hundred amp battery and inverter =
systems. For AC the higher voltage, three phase is more efficient 220, =
440 60 cycle. Power delivery can be at 10 - 12k volts with stepdown =
transformers in the facility. If the building feed is for the entire =
facility than you need home runs from the main power panels to your =
power backup and protection circuits.
=20
When I worked for a certain large fiber backbone based provider of =
circuits and colo for each amount of rack space and racks you would get =
so much DC and AC power and to get more you would pay extra. All of the =
colo power had multi hour battery backup and a generator would kick in. =
They had a DC distribution plant with AC inverters.
=20
Your largest issue may be grounding and the ground plan for the building =
and your datacenter in the building. If the building does not have a =
good ground plane and most do not, you may have to retrofit new =
grounding pads by digging outside the building or through the floor. You =
need to measure the potential to determine if their are any ground loops =
in the building i.e. you want the ground to be the same for all parts of =
the building. You need to put power and transient monitoring equipment =
on your power sources to verify no power spikes or large EMI coming into =
the building. No carpet in the room or you can have bad ESD blowing your =
equipment. Ground the cabinets and have grounding straps for anyone =
working on the equipment.
Check power conditioning equipment for being able to handle brownout =
conditions as well as actual power outages.
=20
In certain areas of the country lightening protection may have to be =
enhanced for the building. Without adequate high voltage and current =
shunts and filters your equipment can be wiped out on a regular basis.
=20
You want to locate the datacenter below ground but not in the basement. =
You want it in the interior of the building for better lightening and =
storm protection. You do not want it in a hundred year flood plane. or =
you may need to seal it against incoming water. You do not want to =
locate it near or below building plumbing. You will want to have non =
reversible drains to drain water but not backup and flood the facility. =
You do not want to locate it close to the elevators, building HVAC or =
other sources of large EM spikes. You may want to add EMI shielding for =
the room to reduce either EMI leaving or entering the room.
=20
If the power requirements are large enough you will need to use a =
chiller system to adequately cool the room. This is putting water in =
your datacenter which is also not a good idea.
=20
During power outages you need to continue to power the HVAC and building =
control or your facility can go down.
=20
You will need to review structural plans for the building to see if the =
floors can handle the extra load or if the floors need to be =
re-enforced.
=20
For security you want re-enforced concrete walls and floors for the =
room. If the current floors and walls are inadequate you may need to =
build a room within the room. If the walls are standard concrete block =
and steel you can run re-enforcing rods and concrete into the blocks. =
You want steel doors with magnetic locks, that can withstand sledge =
hammers and people driving into them. Add video surveillance, biometric =
readers and other sensors for your security systems. =20
=20
Before you can do any construction of course you will need to get the =
appropriate city and county permits and permission from the building =
owners.
=20
If you engineer the facility correctly it can take significant =
investment for a 5, 10 or 15 year investment period.
=20
IMHO make sure you really want to do this.
=20
Good luck
=20
John (ISDN) Lee
=20
________________________________
From: owner-nanog@merit.edu on behalf of Patrick Giagnocavo
Sent: Sat 3/22/2008 10:02 PM
To: nanog@nanog.org
Subject: rack power question
Hopefully this classifies as on-topic...
I am discussing with some investors the possible setup of new datacenter
space.
Obviously we want to be able to fill a rack, and in order to do so, we
need to provide enough power to each rack.
Right now we are in spreadsheet mode evaluating different scenarios.
Are there cases where more than 6000W per rack would be needed?
(We are not worried about cooling due to the special circumstances of
the space.)
Would someone pay extra for > 7KW in a rack? What would be the maximum
you could ever see yourself needing in order to power all 42U ?
Cordially
Patrick Giagnocavo
patrick@zill.net
------_=_NextPart_001_01C88CA7.97C4E6F0
Content-Type: text/html;
charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
<HTML dir=3Dltr><HEAD><TITLE>rack power question</TITLE>=0A=
<META http-equiv=3DContent-Type content=3D"text/html; charset=3Dunicode">=0A=
<META content=3D"MSHTML 6.00.2900.3268" name=3DGENERATOR></HEAD>=0A=
<BODY>=0A=
<DIV id=3DidOWAReplyText45924 dir=3Dltr>=0A=
<DIV dir=3Dltr><FONT face=3DArial color=3D#000000>What is the purpose of =
the datacenter computing, datacom/telco or both. AC or DC power feeds or =
both, backup power or naked, dual feed from the power company with =
transfer switch or power with generator backup? Are you dual feeding the =
racks? Do you require NEBs compliant racks to make it through a shake =
and bake ( a seismic event). </FONT></DIV>=0A=
<DIV dir=3Dltr> </DIV>=0A=
<DIV dir=3Dltr><FONT face=3DArial>Many centers run DC 50 volt multi =
hundred amp battery and inverter systems. For AC the higher voltage, =
three phase is more efficient 220, 440 60 cycle. Power delivery can be =
at 10 - 12k volts with stepdown transformers in the facility. If the =
building feed is for the entire facility than you need home runs from =
the main power panels to your power backup and protection =
circuits.</FONT></DIV>=0A=
<DIV dir=3Dltr><FONT face=3DArial></FONT> </DIV>=0A=
<DIV dir=3Dltr><FONT face=3DArial>When I worked for a certain large =
fiber backbone based provider of circuits and colo for each amount of =
rack space and racks you would get so much DC and AC power and to =
get more you would pay extra. All of the colo power had multi hour =
battery backup and a generator would kick in. They had a DC distribution =
plant with AC inverters.</FONT></DIV>=0A=
<DIV dir=3Dltr><FONT face=3DArial></FONT> </DIV>=0A=
<DIV dir=3Dltr><FONT face=3DArial>Your largest issue may be grounding =
and the ground plan for the building and your datacenter in the =
building. If the building does not have a good ground plane and most do =
not, you may have to retrofit new grounding pads by digging outside the =
building or through the floor. You need to measure the potential to =
determine if their are any ground loops in the building i.e. you want =
the ground to be the same for all parts of the building. You need to put =
power and transient monitoring equipment on your power sources to verify =
no power spikes or large EMI coming into the building. No carpet in the =
room or you can have bad ESD blowing your equipment. Ground the cabinets =
and have grounding straps for anyone working on the =
equipment.</FONT></DIV>=0A=
<DIV dir=3Dltr><FONT face=3DArial>Check power conditioning equipment for =
being able to handle brownout conditions as well as actual power =
outages.</FONT></DIV>=0A=
<DIV dir=3Dltr><FONT face=3DArial></FONT> </DIV>=0A=
<DIV dir=3Dltr><FONT face=3DArial>In certain areas of the country =
lightening protection may have to be enhanced for the building. Without =
adequate high voltage and current shunts and filters your =
equipment can be wiped out on a regular basis.</FONT></DIV>=0A=
<DIV dir=3Dltr><FONT face=3DArial></FONT> </DIV>=0A=
<DIV dir=3Dltr><FONT face=3DArial>You want to locate the datacenter =
below ground but not in the basement. You want it in the interior of the =
building for better lightening and storm protection. You do not want it =
in a hundred year flood plane. or you may need to seal it against =
incoming water. You do not want to locate it near or below building =
plumbing. You will want to have non reversible drains to drain water but =
not backup and flood the facility. You do not want to locate it close to =
the elevators, building HVAC or other sources of large EM spikes. You =
may want to add EMI shielding for the room to reduce either EMI leaving =
or entering the room.</FONT></DIV>=0A=
<DIV dir=3Dltr><FONT face=3DArial></FONT> </DIV>=0A=
<DIV dir=3Dltr><FONT face=3DArial>If the power requirements are large =
enough you will need to use a chiller system to adequately cool the =
room. This is putting water in your datacenter which is also not a good =
idea.</FONT></DIV>=0A=
<DIV dir=3Dltr><FONT face=3DArial></FONT> </DIV>=0A=
<DIV dir=3Dltr><FONT face=3DArial>During power outages you need to =
continue to power the HVAC and building control or your facility can go =
down.</FONT></DIV>=0A=
<DIV dir=3Dltr><FONT face=3DArial></FONT> </DIV>=0A=
<DIV dir=3Dltr><FONT face=3DArial>You will need to review structural =
plans for the building to see if the floors can handle the extra load or =
if the floors need to be re-enforced.</FONT></DIV>=0A=
<DIV dir=3Dltr><FONT face=3DArial></FONT> </DIV>=0A=
<DIV dir=3Dltr><FONT face=3DArial>For security you want re-enforced =
concrete walls and floors for the room. If the current floors and walls =
are inadequate you may need to build a room within the room. If the =
walls are standard concrete block and steel you can run re-enforcing =
rods and concrete into the blocks. You want steel doors with magnetic =
locks, that can withstand sledge hammers and people driving into them. =
Add video surveillance, biometric readers and other sensors for your =
security systems. </FONT></DIV>=0A=
<DIV dir=3Dltr><FONT face=3DArial></FONT> </DIV>=0A=
<DIV dir=3Dltr><FONT face=3DArial>Before you can do any construction of =
course you will need to get the appropriate city and county permits =
and permission from the building owners.</FONT></DIV>=0A=
<DIV dir=3Dltr><FONT face=3DArial></FONT> </DIV>=0A=
<DIV dir=3Dltr><FONT face=3DArial>If you engineer the facility correctly =
it can take significant investment for a 5, 10 or 15 year investment =
period.</FONT></DIV>=0A=
<DIV dir=3Dltr><FONT face=3DArial></FONT> </DIV>=0A=
<DIV dir=3Dltr><FONT face=3DArial>IMHO make sure you really want to do =
this.</FONT></DIV>=0A=
<DIV dir=3Dltr><FONT face=3DArial></FONT> </DIV>=0A=
<DIV dir=3Dltr><FONT face=3DArial>Good luck</FONT></DIV>=0A=
<DIV dir=3Dltr><FONT face=3DArial></FONT> </DIV>=0A=
<DIV dir=3Dltr><FONT face=3DArial>John (ISDN) Lee</FONT></DIV>=0A=
<DIV dir=3Dltr> </DIV>=0A=
<DIV dir=3Dltr>=0A=
<HR tabIndex=3D-1>=0A=
</DIV>=0A=
<DIV dir=3Dltr><FONT face=3DTahoma size=3D2><B>From:</B> =
owner-nanog@merit.edu on behalf of Patrick Giagnocavo<BR><B>Sent:</B> =
Sat 3/22/2008 10:02 PM<BR><B>To:</B> nanog@nanog.org<BR><B>Subject:</B> =
rack power question<BR></FONT><BR></DIV></DIV>=0A=
<DIV><BR>=0A=
<P><FONT size=3D2>Hopefully this classifies as on-topic...<BR><BR>I am =
discussing with some investors the possible setup of new =
datacenter<BR>space.<BR><BR>Obviously we want to be able to fill a rack, =
and in order to do so, we<BR>need to provide enough power to each =
rack.<BR><BR>Right now we are in spreadsheet mode evaluating different =
scenarios.<BR><BR>Are there cases where more than 6000W per rack would =
be needed?<BR><BR>(We are not worried about cooling due to the special =
circumstances of<BR>the space.)<BR><BR>Would someone pay extra for > =
7KW in a rack? What would be the maximum<BR>you could ever see =
yourself needing in order to power all 42U =
?<BR><BR>Cordially<BR><BR>Patrick =
Giagnocavo<BR>patrick@zill.net<BR></FONT></P></DIV></BODY></HTML>
------_=_NextPart_001_01C88CA7.97C4E6F0--
