[153394] in North American Network Operators' Group
Re: Penetration Test Assistance
daemon@ATHENA.MIT.EDU (Brett Watson)
Tue Jun 5 15:57:04 2012
From: Brett Watson <brett@the-watsons.org>
In-Reply-To: <4FCE394D.4040102@alter3d.ca>
Date: Tue, 5 Jun 2012 12:48:07 -0700
To: nanog@nanog.org
Errors-To: nanog-bounces+nanog.discuss=bloom-picayune.mit.edu@nanog.org
On Jun 5, 2012, at 9:52 AM, Peter Kristolaitis wrote:
>=20
> As far as horror stories... yeah. My most memorable experience was a =
guy (with a CISSP designation, working for a company who came highly =
recommended) who:
> - Spent a day trying to get his Backtrack CD to "work properly". =
When I looked at it, it was just a color depth issue in X that took =
about 45 seconds from "why is this broken?" to "hey look, I fixed it!".
> - Completely missed the honeypot machine I set up for the test. I =
had logs from the machine showing that his scanning had hit the machine =
and had found several of the vulnerabilities, but the entire machine was =
absent from the report.
> - Called us complaining that a certain behavior that "he'd never =
seen before" was happening when he tried to nmap our network. The =
"certain behavior" was a firewall with some IPS functionality, along =
with him not knowing how to read nmap output.
> - Completely messed up the report -- three times. His report had =
the wrong ports & vulnerabilities listed on the wrong IPs, so according =
to the report, we apparently had FreeBSD boxes running IOS or MS SQL...
> - Stopped taking our calls when we asked why the honeypot machine =
was completely missing from the report.
>=20
> In general, my experience with most "pen testers" is a severe =
disappointment, and isn't anything that couldn't be done in-house by =
taking the person in your department who has the most ingrained =
hacker/geek personality, giving them Nessus/Metasploit/nmap/etc, pizza =
and a big ass pot of coffee, and saying "Find stuff we don't know about. =
Go.". There is the occasional pen tester who is absolutely phenomenal =
and does the job properly (i.e. the guys who actually write their own =
shellcode, etc), but the vast majority of "pen testers" just use =
automated tools and call it a day. Like everything else in IT, security =
has been "commercialized" to the point where finding really good =
vendors/people is hard, because everyone and their mom has CEH, CISSP, =
and whatever other alphabet soup certifications you can imagine.
I agree with a lot of what you've said, but there are absolutely good =
security guys (pen tester, vulnerability assessors, etc) that use both =
open source and commercial automated tools, but still do a fantastic job =
because they understand the underlying technologies and protocols.
I used to do a lot of this in the past, had lots of automated tools, and =
only occasionally wrote some assessment modules or exploit code if =
necessary.
But again, a person in that position has to understand technology =
holistically (network, systems, software, protocols, etc).
-b=
