Who performs the testing? Does your IT systems/network/application operations groups perform, an independent non admin function within security or internal audit department do pentest work?

Where I work I am the sole pentester and am not an administrator within the operations environment nor do I report to the operations group. I'm in a completely different chain of command with separate funding sources. I define controls, and test for controls, manage the compliance programs and monitor for compliance and generate reports and metrics for management but do not manage systems.

How often do you test?

I'm on a planned annual testing cycle for critical systems but my program is in its infancy so have not rotated through the entire workplan yet. The plan is to test critical systems on a yearly basis or when there are major changes in the environment.

What do you test and do you test known vulnerable systems to remind the business about long-term ongoing issues?

I test critical systems but am not currently retesting known vulnerable systems since it seems like a wasted effort. I sometimes wonder if the organization is just blowing me off though and if maybe I should bludgeon them with the high priority issues regardless.

How do you respond to criticism that the system you just pwned does not contain nor is it connected to any system that contains critical or sensitive information? This is more a question about correlation of technical to business risk when you cannot quantify the business risk but the systems are vulnerable.

This is a pain point for me as the best I can come up with is that weak systems identify a process deficiency that is likely present in other critical system components. This way I address the process not the specific systems.

How much do you do internally and what pieces do you outsource? This is especially critical when you have a testing team of only 1 or 2 individuals where expertise may not extend to all systems in use.

I'm currently testing Windows and Linux systems extensively and to a lesser degree web application and plan to outsource some upcoming Oracle and application specific work and potentially some of our embedded systems. I hope to expand on my capabilities and eventually be able to test these other systems but my current skill level prevents me from delivering comprehensive testing.

How do your testing activities feed into remediation efforts? Are you involved with remediation at all?

I generate reports to management but do not remediate or manage systems

What are your favorite tools? I'm really curious here how many small to medium sized businesses are buying the big ticket commercial tools like Core Impact or rely mostly on free tools. How have you been able to quantify the ROI *snicker* for these commercial tools when in many cases open source tools may suffice?

I use a combo of open source tools like Metasploit, Nmap ad Nikto and am planning to purchase NeXpose and Metasploit Pro as well as Maltego and Burp Pro. I'd like to get a dedicated web scanner but need to spend more time with the NeXpose scanner and see how the W3AF partnership plays out before I go spend more money. I have high hopes. I'm also starting to do more Python stuff and am looking forward to learning more about the "Weaponized Python" stuff in the new SANS660 when I take it next year. I have not quantified ROI but billed commercial products under our PCI compliance program as "requirements"

Who performs the testing? Does your IT systems/network/application operations groups perform, an independent non admin function within security or internal audit department do pentest work?

I perform both internal and external testing and have deployed a semi-automatic method of doing so

How often do you test?

Weekly

What do you test and do you test known vulnerable systems to remind the business about long-term ongoing issues?

Weekly and after any significant changes to any systems.

How do you respond to criticism that the system you just pwned does not contain nor is it connected to any system that contains critical or sensitive information? This is more a question about correlation of technical to business risk when you cannot quantify the business risk but the systems are vulnerable.

There is rarely criticism for non-mission critical systems compromised versus mission critical systems. If its internal, it poses a risk period. Management was made aware of "staged" exploits in the sense that, just because there wasn't anything on that particular machine, there is still the potential of using that "non-mission-critical" machine as a pivot/jump-point/escalation server. Whether its via an attacker checking for similar users, installing a sniffer, etc... If it's in-house, we treat ALL systems including development machines as mission critical.

How much do you do internally and what pieces do you outsource? This is especially critical when you have a testing team of only 1 or 2 individuals where expertise may not extend to all systems in use.

Everything is done internally and most business processes, goals and technologies are understood by us (security team) at all points in time.

How do your testing activities feed into remediation efforts? Are you involved with remediation at all?

Most of the times I work with developers and programmers in training them to use better methods in their programs. System administrators, we try to convey the need for RBAC's, separation of duties, realms (VLANs, etc).

What are your favorite tools? I'm really curious here how many small to medium sized businesses are buying the big ticket commercial tools like Core Impact or rely mostly on free tools. How have you been able to quantify the ROI *snicker* for these commercial tools when in many cases open source tools may suffice?

Currently I use a combination of open source and pay-for-play tools. My personal favorites are Scapy for packet manipulation, Canvas, metasploit, AppScan, Acunetix, CAIN, OpenVAS, Paros, Wikto + Nikto (when applicable), custom written tools. I ALWAYS have to fight to get tools and some I can't justify like Core Impact (it lapsed a while back). Because my company offers managed security services, its slightly easier for me to get the things I want/need as long as I make a strong business case for them.

NOW... What *has* happened once or twice that I have gripes with is when I'm forced to do "controlled" testing, where there is a fear that I use a tool that can "take down the house." While those fears may be real in some environments, the fact is, I have never READ, nor SEEN, nor HEARD about someone performing such a severe pentest that it completely rendered an application useless to the point that the application and or service running needed re-installation. Sure there is the risk I can "crash" something, but the actuality of that occurring is low. I try to make management aware that by performing "controlled" tests, they will NOT and NEVER WILL get a real result. The outcome is skewed period.

I try to convey to them there are mechanisms to minimize the potential of "bringing down the house." Since a "real world" attacker isn't going to care whether or not they bring down an application, the same tools should be used to obtain real results.

I would like to expound on this more however, 1) it's Friday 2) I've been in training all day (Acme Packet Security (VoIP)) 3) I'm half-empty/half-full from sleep

@Ziggy, we don't have "informal pentests" for PCI. We have to maintain compliance so I "mop up" before Trustwave validates our findings. (Remember, thou shall not audit thyself!).

As for permission, I dictate the security services that we sell and manage so I bring to management what I'm going to do, how I'm going to do it, why I'm going to do it. I always bring the business facts to the table when asked however, they've learned to trust my judgment. I also try to make sure I always include staff here in my tests. They're the ones that can make sense of things I don't/won't understand. So if I find something I flag as a risk, they can counter on the fly or collectively, we can address compensating controls, etc.