[177826] in North American Network Operators' Group
Re: Checkpoint IPS
daemon@ATHENA.MIT.EDU (Patrick Tracanelli)
Fri Feb 6 09:52:38 2015
X-Original-To: nanog@nanog.org
From: Patrick Tracanelli <eksffa@freebsdbrasil.com.br>
In-Reply-To: <CALFTrnP-oBBd2fi-GZCLt5PWkqjyH9NfZvx9Btsmbjb-pOL_uA@mail.gmail.com>
Date: Fri, 6 Feb 2015 12:52:35 -0200
To: Ray Soucy <rps@maine.edu>
Cc: NANOG <nanog@nanog.org>
Errors-To: nanog-bounces@nanog.org
Hello,
> On 06/02/2015, at 11:08, Ray Soucy <rps@maine.edu> wrote:
>=20
> An IPS doesn't have to be in line.
AFAIK this is basically what defines an IPS.
> It can be something watching a tap and scripted to use something else
> to block traffic (e.g. hardware filtering options on a router that can
> handle it).
You are naming IPS on what is actually an IDS+Active Response as I =
mentioned before (my #1 option of choice). Not been online won=92t for =
instance prevent the so called single packet attacks and therefore what =
should be the advantage of an IPS over an IDS is just thrown away. =
Which, again, I accept what=92s missed for what I still have. But I =
don=92t think it can be named IPS acting passively+actively, since it=92s =
not actually not preventing.
> An IDS tied into an internal RTBH setup to leverage uRPF filtering in
> hardware can be pretty effective at detecting and blocking the typical
> UDP attacks out there before they reach systems that don't handle that
> as gracefully (e.g. firewalls or host systems).
That=92s exactly the point I agree and adopt. Maybe a first packet will =
pass, but it takes longer anyway to be deterministic whether the traffic =
is common or attack traffic. It=92s an IPs there and exausting/starving =
won=92t cause issues to the network, only to it=92s own capability.
> Even if you keep it
> from being automated and just have it be an IDS that you can have a
> human respond to is pretty valuable.
Not a coincidence that Bejtlich=92s NSM101and TCP/IP Weapons School =
courses are usually sold out on Black Hat. Valuable humans behind the =
tools will always provide better benefits than what vendors may =
generically sell/deliver.=20
>=20
>=20
> On Thu, Feb 5, 2015 at 6:40 PM, Patrick Tracanelli
> <eksffa@freebsdbrasil.com.br> wrote:
>>=20
>>> On 05/02/2015, at 12:31, Terry Baranski =
<terry.baranski.list@gmail.com>
>>> wrote:
>>>=20
>>> On Thu, Feb 5, 2015 at 8:34 AM, Roland Dobbins <rdobbins@arbor.net> =
wrote:
>>>=20
>>>> I've never heard a plausible anecdote, much less seen meaningful
>>>=20
>>> statistics,
>>>>=20
>>>> of these devices actually 'preventing' anything.
>>>=20
>>>=20
>>> People tend to hear what they want to hear. Surely your claim can't =
be
>>> that
>>> an IPS has never, in the history of Earth, prevented an attack or =
exploit.
>>> So it's unclear to me what you're actually trying to say here.
>>>=20
>>>> And the fact that well-known evasion techniques still work against =
these
>>>> devices today, coupled with the undeniable proliferation of =
compromised
>>>> hosts residing within networks supposedly 'protected' by these =
devices,
>>>> militates against your proposition.
>>>=20
>>>=20
>>> Your tendency of making blanket statements is somewhat baffling =
given the
>>> multitude of intricacies, details, and varying circumstances =
involved in a
>>> complex topic like this. To me, it's indicative of an =
overly-simplified
>>> and/or biased way of looking at things.
>>>=20
>>> In any case, go ahead and stick with your router ACLs and =
(stateful!)
>>> proxies. Different strokes.
>>>=20
>>> -Terry
>>=20
>>=20
>> There's room for a good engineered strategy for protection which =
won't turn
>> into a point of failure in the overall networking topology.
>>=20
>> For decades, since first rainbow series books were written and =
military
>> "strategy" started to be added to information security, it's always =
been
>> about defense in depth and layered defense. Today those buzzwords =
became an
>> incredibly "bullshit bingo" on sales force strategy on selling =
magical boxes
>> and people tend to forget the basics. Layers and the "depth" is not a =
theory
>> just to be added to drawings and keynote presentations.
>>=20
>> Considering a simplistic topology for 3-tier (4 if you count T0) =
depth
>> protection strategy:
>>=20
>> (Internet)--[Tier-0]--(Core Router)--[Tier1]--(core
>> switch)--[Tier2]--(DMZ)--[Tier3]--(Golden Secret)
>>=20
>> One security layer (tier, whatever) is there to try to fill the gap =
from the
>> previous one.
>>=20
>> How deep you have to dig depends on who you are. If you are the end
>> organization willing to protect the golden secret, how complex is =
your
>> topology, or if you are the carrier, the telecom the company worried =
only
>> about BGP, PPS, BPS and availability other than the actual value for =
what's
>> the real target for the attack (if not availability)
>>=20
>> In summary, in my experience what will (not) work is:
>>=20
>> 1) Tier 0 & Tier 1
>> On border, core, (Tier0) or on Tier-1 protection layers (typical
>> "firewall/dmz" topological position)
>>=20
>> - Memory and CPU exaustion will shut down your operations (increase =
latency
>> and decrease availability) easily, if you have a Protecting Inbound =
Proxy
>> (Web Application Firewall, for the sales jargon), Stateful Firewall =
or IPS.
>>=20
>> The thing here is, you are insane or naive if you believe a finite =
state
>> machine with finite resources can protect you from a virtually =
infinite
>> (unlimited) source of attacks. No matter how much RAM you have, how =
much CPU
>> cycles you have, they are finite, and since those "amazing stateful =
w/ deep
>> inspection, scrub, normalization and reassembly" features on a =
firewall will
>> demand at least 4K RAM per session and a couple CPU cycles per test, =
you
>> have a whole "line rate" (1.4Mpps / 14Mpps for 1GbE 10GbE ports) =
attack
>> potential, and come on, if a single or a 3-way packets sequence will =
cost
>> you a state, it's an easy math count to find out you are in a bad =
position
>> trying to "secure" those Tier0 and Tier1 topological locations. It's =
just
>> easier and cheaper for the one attacking, than for your amazing =
firewall or
>> IPS.
>>=20
>> So what to do here? Try to get rid of most automated/scripted/simple =
attack
>> you can. You can do ingress filtering, a lot of BCP, protection =
against
>> SNMP/NTP/DNS amplification, verify reverse path, (verrevpath, =
antipsoof,
>> verrevreachability), and many good / effective (but limited) =
protection with
>> ACL, data plane protection mechanisms, BGP blackholing, Null Routing
>> (sending stuff to disc0 on BSD, null0 on Cisco, etc) and Stateles
>> Firewalling.
>>=20
>> Also, an IDS sensor might fit here, because if CPU/Memory starvation =
happens
>> on an IDS sensor, the worst thing will happen is some packets getting =
routed
>> without proper processing. But still, they will get routed.
>>=20
>> Always stateles, always simple tests. No Layer7 inspection of any =
form, no
>> load balancer, WAF, whatever. No regex, hell no! No memory pointers =
that
>> can point to dark processor/memory locations (again, no regex).
>>=20
>> 2) On Tier 2 protection (A defense depth that comes after core =
protection
>> and after Tier-1):
>>=20
>> - Will miserably fail if you use stateful firewall in excess
>> - Will fail even quicker if you use a WAF or IPS
>>=20
>> Many "security engineers" and "security experts" (or worse, security =
tools
>> vendors and developers) excessively use stateful tests on transport
>> protocols that are simply... stateles.
>>=20
>> What good you have on stateful firewalling... ICMP? UDP? DDP? IGMP? =
come on,
>> all those state timers are worthless and your memory limits will =
reach very
>> soon. Remember, in the average, 4K RAM at least, per stateful =
session. Much
>> more resources are needed for IPS, much much more for inbound proxy =
(WAF),
>> not to mention CPU.
>>=20
>> Will you add an IPS here? What for, other than easing putting down =
and
>> making your network and services unavailable?
>>=20
>> Proxy? Mod Security? Hell no! Not here. Did you check how many regex =
and
>> rules you have in the "base_rules" collection for a Top 10 OWASP =
protection
>> on Mod Security? What about vendor specifics rules, or Sans Top 25
>> specifics.
>>=20
>> This is just not the place.
>>=20
>> Also, let's rationale, what is your real benefit on deep inspection =
stateful
>> filtering on those SSH sessions allowing for your trusted locations =
only?
>> Really, what good will it make? Drop it stateles, allow it stateles! =
If you
>> really believe stateful worths something for protocols such as SSH, =
do it
>> somewhere else (Tier3 or host-based).
>>=20
>> Focus on stateful protection only for what really needs some =
protection of
>> this kind. Packets related, fragile services for packets arbitrarily
>> assembled. Maybe SIP (SIP, not RTP), HTTPS, HTTP. Layer3/Layer4 =
fragile and
>> public services, after some stateles inspection was already done on =
Tier1,
>> should deserve stateful protection only. Not your whole network! Not =
your
>> whole set of services or services.
>>=20
>> And no ICMP, no UDP do deserve stateful.
>>=20
>> Also, pick up a good tool for this.
>>=20
>> Most sysadmins, security guys or network operators never notice how =
their
>> tools may betray 'em hardly (by deault).
>>=20
>> For example one use OpenBSD PF firewall, every rule you make will
>> "automagically" become an stateful rule. And this is not good! This =
is
>> terrible! This "auto" behavior makes the security engineer blind, =
since the
>> rules he is writing is not the rules he is adding to kernel.
>>=20
>> OpenBSD's PF has a "no state" option and nobody uses it! It means you =
are
>> doing it wrong... UDP/ICMP/TCP rules always have "keep state" added, =
if you
>> don't explicitly set "no state". Beware!
>>=20
>> The same is valid for Linux Netfilter, and 9 in 10 commercial =
firewalls
>> (checkpoint, mikrotik, fortinet, whatever...).
>>=20
>> FreeBSD's ipfw on the other hand is stateles by default. It means if =
you
>> don't explicitly add "keep-state" there, it's stateles. Which is =
good,
>> unless you explicitly want a rule to be stateful. It's excellent as a
>> default behavior for protection layers not close to your "golden egg
>> provider". On the other hand, ipfw by default has a limit of 4096 =
states
>> which is TOO LOW. Beware too!
>>=20
>> Regarding IPS/WAF/Proxy or "Next Generation" firewalls, this is not =
the
>> place to add it.
>>=20
>> On the other hand you need some level of extra protection firewalls =
will not
>> provide, and some Layer7 inspection on Tier2 will be good.
>>=20
>> But not Proxy! Not mod security! and not IPS! (no WAF)
>>=20
>> IMHO, an IDS will fit good here.
>>=20
>> IDS, not IPS, not IDP. Not a magical solution...
>>=20
>> Why, from my past experiences, an IDS approach here will fit? Due =
it's
>> passive nature. If your IDS (say, Suricata, Bro, keeping on open =
source, or
>> your commercial option of choice) starves on CPU or memory, what's =
the worst
>> thing to happen?
>>=20
>> You will have a high number of PPS on that perimeter port passing =
without
>> getting processed/inspected. Your overall security will decrease, but =
you
>> still have Tier3 (and maybe other tiers) to fulfill the gap! Packets =
that
>> still can be processed should have active response in place taking =
care of a
>> lot of attacks.
>>=20
>> What I mean is, if you have limited (and you do have limited) =
processing
>> and memory power, say you can IDS inspect 400Kpps but a 1Mpps attack =
is
>> ongoing, you have 40% inspection power, but packets not processed =
still get
>> routed! Without latency and without packet loss because it's an IDS =
and not
>> an IPS. It's not inline. It's passive, sitting there. Limited =
resources,
>> priority and lower kernel CPU scheduler relevance.
>>=20
>> And for those 40% (very bad scenario I am drawing here) you still =
have
>> active response in place, rerouting, reconfiguring stateles firewall,
>> stateles ACLs and null routes on upper tiers (tier 0, tier 1, =
routers,
>> switches) and lower tiers (tier 3 reconfigured upon tier 2 decision).
>>=20
>> What you will do is try to fill the gap on next tiers, or increase =
your
>> filtering capabilities that are still stateles or passive.
>>=20
>> For many business, this is the end for added protection layers. A big =
ISP,
>> transport provider, content delivery business... more protection from =
this
>> point is something for the specific end business, and completely =
related to
>> their needs, their business model, process, responsibilities and =
overall
>> evaluated requirements. Not a general model to fit.
>>=20
>> But for everyone else up to this tier you have relevant security =
mechanisms
>> and tecnology added, and still low starvation/exaustion risks due to =
the
>> stateles and passive nature of the approach.
>>=20
>> 3) On Tier 3, a protection strategy for your "golden eggs" provider, =
the
>> golden secret, your business secret and intelligence
>>=20
>> Usually, this protection level is for the corporate strategy. The =
business,
>> not the carrier, the service provider or the network operator. And is =
a
>> business specific requirement. Meaning it may not exist at all!
>>=20
>> Now, for me, here is where you add more stateful inspection (still, =
only
>> what is actually efficient, not that god damn echo reply/request =
wasting
>> memory to be tracked down - useless!).
>>=20
>> Here is a good place for a WAF, after firewall and IDS protection =
took
>> place. WAF is a not a panaceia for anything, it's aimed for specific =
attacks
>> against applications and protocols, is not resilient to coward =
attacks
>> (volumetric, L3/L4 exaustion, etc). And a proxy in the end is always =
a web
>> server, so inherits all it's fragility, and therefore it must be =
protected
>> as well, before it can actually protect anything else.
>>=20
>> Host Intrusion Detection central servers (receiving information =
gathered
>> from HIDS on the actual hosts) also fits Tier3. As well as other =
host-based
>> controls that may add telemetry information to a central location.
>>=20
>> Talking about telemetry, it's important everywhere, and while =
generating
>> flows or snmp info grabbing may impact processing usage on critical =
core
>> devices, those extra added boxes should also passively help =
telemetry, with
>> flow generation or minimum capability for snmp servings.
>>=20
>> Nothing here is new. I am talking about, again, basic stuff discussed =
for
>> decades on colored cover military books, drafts, discussions and BCPs =
and
>> really old stuff discussed for people who may be already dead, =
sometimes
>> (Itojun and other samurais' missed). I'm only mentioning the basic 3 =
tech
>> domains (firewall, ids, proxy), but the other two basic (pen test, =
vuln
>> scan) that are more process than technology are important as well.
>>=20
>> For me, and again this is very personal opinion, I never run an IPS =
unless
>> the customer "really wants to" (or a stupid compliance requirement =
really
>> requires to). An IDS+Active Response is as good as IPS, and the only =
extra
>> benefit an IPS will add compared to IDS is related to "single packet
>> attacks", that ones that will pass quickly enough before the active =
response
>> blocks it. But "single packet" attacks are related to poorly written
>> software (or unpatched / not fixed software) since it's not really an
>> attack, it's a trigger to bad code misbehavior and should be =
addressed on
>> the host.
>>=20
>> This is a very simple model, and easy to understand. However how many
>> situations we've seen big companies getting completely unavailable or =
AAA
>> getting broken because people insist to buy (and sell) "miracle =
boxes" added
>> to core locations, and those miracle boxes will have amazing deep =
inspection
>> firewalls or IPS or DLP (whatever it means, Data Loss, Data Leak, it =
means
>> whatever you want to buy)...
>>=20
>> There may be a place for those stuff, but it's not on core. Nor on =
second
>> level protection layers.
>>=20
>> In the end, ISO27002, PCI-DSS, CIA and AAA triads, what are they =
worth if
>> you add an IPS to your core? When memory is exausted, processing is =
starved,
>> you will have packet loss, latency, or you will be completely =
offline. And
>> what's CIA if you "security features" are breaking Integrity due to =
missing
>> packets, or breaking full Availability at all? What you have from =
CIA? Only
>> confidentiality? Better take that plug off. Same for AAA, if =
Authorization
>> and Authentication are broken or failed due to exaustion/starvation =
what you
>> get? Accounting/Auditing? So you will sit and read the logs to find =
out what
>> went wrong?
>>=20
>> Discouraging firewall/IDS/proxy protection layers is as bad as over
>> leveraging it.
>>=20
>> Dosage is what distinguishes the poison from the vaccine.
>>=20
>> --
>> Patrick Tracanelli
>> FreeBSD Brasil
>> Tel.: (31) 3516-0800
>> http://www.freebsdbrasil.com.br
>> "Long live Hanin Elias, Kim Deal!"
>>=20
>>=20
>=20
>=20
>=20
> --=20
> Ray Patrick Soucy
> Network Engineer
> University of Maine System
>=20
> T: 207-561-3526
> F: 207-561-3531
>=20
> MaineREN, Maine's Research and Education Network
> www.maineren.net
--
Patrick Tracanelli
FreeBSD Brasil LTDA.
Tel.: (31) 3516-0800
316601@sip.freebsdbrasil.com.br
http://www.freebsdbrasil.com.br
"Long live Hanin Elias, Kim Deal!"
