I gotta know...."Quis custodiet ipsos custodes?" Are there standards?

[Animus]

I'm curious... 
When a client hires my consulting firm to complete a project, there are a few pieces of collateral that I have grown to expect...

• Project Plan (and associated collateral)
• Process Guide
• Process Documentation (usually in UML/XML/BPMN 1.2, etc... though it really depends...)
• Project Notes (including interviews with SMEs, entrance/exit interviews, Post Implementation Reviews, etc...)

So when my buddy's company hires an "Ethical Hacker" to do a security assessment, I'm expecting:

• a list of vulnerabilities (itemized and ranked by priority and criticality/impact)
• the means to exploit them (exploit code location/repository)
• those that were exploited (identified by a unique identifier, like a MAC, IP, name, anything really...)
• those that were not exploited and the reasons why (like it'd bring down X service, etc...)

What I was not expecting was a Word document showing what they scanned and the "possible" risks. With nothing towards remediation... "It's not in the scope of the pen-test. [...] We make recommendations, and they make the changes..."

Wha?

• Is there some "standard" penetration methodology or process out there?
• I'm sure, if it's like any other industry - there's tons of "standards" out there... But which ones are the "biggies" and how would one know if someone did a good job?
• Are there firms that "audit" the pen-testing companies?

I'm thinking there has to be some way to address the age-old question:
"Quis custodiet ipsos custodes?" - Who will watch the watchmen?

[ajohnson]

Some places simply do garbage work. We're often complimented on the detail that goes into our pen tests and vulnerability assessments reports. When I'm writing a report, I explain what the vulnerability is, what the risks are, how it was exploited (or how it was attempted to be exploited), what information/access was obtained, and how to remediate it.

You should review the SoW (statement-of-work) and the contract to make sure expectations are clear, and both parties are on the same page. You should also ask to see sample deliverables.

[sil]

• Is there some "standard" penetration methodology or process out there?
• I'm sure, if it's like any other industry - there's tons of "standards" out there... But which ones are the "biggies" and how would one know if someone did a good job?
• Are there firms that "audit" the pen-testing companies?

There are two well known "pentesting" frameworks. ISSAF (http://www.oissg.org/downloads/issaf-0.2/index.php) and OSSTMM (http://www.isecom.org/osstmm/). Without getting too much into politics, I wouldn't bother with ISSAF since it hasn't been taken serious since 2006 which is a long time for new things to "happen." OSSTMM provides the most information for getting the job done correctly however, it has never really taken off here in the United States.

As for remediation, most companies clarify differences in their SOW's. Some companies steer clear of offering "fixes" for the sake of remaining unbiased in their findings. Some companies offer both a remedy and a cure however, companies that do so run the risk of being viewed as having an agenda. For example, if I told you "I can reach SMB ports, there is a potential for an attack... I can fix it for you for ..." How would you react versus: "It's possible to reach SMB ports" At that point it is at your discretion to act upon it. Validate SMB is vulnerable or go about "business as usual." In the former: "... i can fix it for you" there is a connotation of "slick willie talk" if you ask me.

Anyhow, I'd suggest learning OSSTMM, NSA IAM/IEM methodologies and incorporation them into your own framework. I usually use those to frameworks in a mesh of my own little mess to create my own framework of testing, responses, reporting.

[ajohnson]

Just to be clear, we don't offer remediation for that very reason (remaining unbiased, especially when it comes to IT audits). However, we do make recommendations to give our clients direction for remediation.

For example, in the case of unrestricted zone transfers, I'd inform them that they should limit zone transfers to only the hosts that require them, or disable them entirely if they are not necessary. I don't provide step-by-step instructions for BIND or whatever DNS server they're using, nor do I make the configuration myself. I just try to give them a little push in the right direction.

[Animus]

I'm not exactly expecting people to have some kind of information on a step-by-step remediation... I just want someone to tell me the means of addressing a security vulnerability and some suggestions on how to address it...

Point form, like this:

• Vulnerability identified
• Exploit possible/not possible
• Risk to company
• Possible means of addressing risk
• etc....

I'll take a look at those methodologies and see if there is something in there in the form of some kind of Visio or process diagram. Also, if you could point me to some form of "Checks and Balances" or other "caveats" towards a pen-test... I would be very grateful....

Cheers,
Animus