I've seen several recent comments elsewhere questioning the value of IDS.  I'd like to what other people think about it.

I've managed an IDS in the past and conducted some IDS research for a former employer, but it's been several years since I did any hands-on IDS monitoring so I feel like I'm lacking a current perspective.

The argument that I've always accepted is Bejtlich's "prevention eventualy fails."  I still think that argument is valid and can see the value of monitoring systems, logging, keeping session/statistical data both for detection and response, etc.  But, I wonder what value IDS actually gives us. 

Consider Snort, let's say we remove all of the signatures that aren't applicable to our environment (e.g. remove Oracle rules if we don't run Oracle), remove all of the rules that are too out of date to matter (e.g. teardrop), and also remove all of the rules for things that we're blocking anyway.  Once we do that, how much is really left and what are the odds that, if we do undergo a serious attack, that the remaining rules will alert us to it?

Although prevention eventually fails, the detection systems that we put into place is only valuable if they are able to detect malicious activity when prevention fails.  Otherwise, we don't gain any additional security.

My immediate concern isn't managing, tuning, or responding to attacks.  Assuming an organization can handle that, what value does the IDS actually bring?  What is its capability to actually detect attacks?

I've seen numbers for AV (they aren't reassuring), but not IDS.  I'd love to have some hard data or even casual observations from the field as to what various IDS are actually capable of. 

Put another way, I'd like to know how many false negatives there are.  I want to know how much passes "under the radar".  If IDS can only detect 10, 20 or 30% of attacks, then it's not a very valuable tool.  If it can detect 70% or more, the benefits become a lot more significant.

I've brought this up elsewhere and a common response is that the value depends on the person running it, but the person running it is responsible for tuning the IDS for performance and sifting through false positives.  Let's assume tuning is not an issue.  I can enable all the rules I want without running into performance problems and, by magic, the false positive will still be low.  Will the IDS catch the stuff I really care about?  Looking at Snort, it seems to me that a lot of the rules catch older attacks or very generic patterns that are easy to avoid (e.g. 0x90 NOP sleds).

Have you used them to successfully detect any attacks that weren't super-obvious (e.g. a Nessus scan)?  Have you had any slip by (that you know of)?

I'm focused on the existing rule sets, either those available from the IDS vendor or a third-party project such as Emerging Threats.  What you can actually detect with the default rule set?  Most of what I see in the rule sets is old, easy to avoid, or too noisy (e.g. port scans if you looking at Internet-originating traffic).

My concern is that we don't spend much time evaluating how effective IDS actually is.  This has been the case with anti-virus as well although it's easier to find data for that since AV is more widely deployed. 

The ultimate answer may be that IDS is extremely effective, but I don't want to assume that.  I want to challenge our assumptions to get to something more objective. 

I understand the concept of tuning the IDS, turning off rules that generate too many false positives, creating some custom rules to fit local policy, etc.  That's all well and good.  But, underneath that we're depending on someone else to deliver a base set of rules and capabilities that are supposed to detect malicious traffic and I'm not convinced about the effectiveness of what is being delivered.

Well in most cases your IDS and IPS are one in the same.  Only major differences is you tell your IPS side to block specific signatures.  Other than that you tell them both to ignore traffic related to platforms you are not using.  You want them both to log and in some cases you allow certain devices to pass through unhindered.  For instance anything going to your honeypot, you might want to allow through without blocking but you definitely want to log it for analysis. 

Also it is about placement.  My last job we had an IDS/IPS that first was sitting in the rack for like 2 years and all it was doing was passing traffic between the internet and LAN.  I had to call support to get the thing updated since no one bothered doing that for the 2 years.  So once I got that all squared away I turned turned on the logging to get a baseline and disabled the logging for the platforms I know I didn't have.  After that it was a couple weeks of tweaking until I knew I could turn on the IPS part without breaking the network.  I was getting some decent traffic, most of which was valid.  Then some genius decided to have it moved from the main internet line where most of our high traffic devices where to a secondary line where the only thing that existed was email which was going through a filtered service.  Guess what happened.... nothing. 

And I was happy to report the plan that the ISO and network engineer implemented failed miserably.  Oh and yes, we (my boss and I) did recommend alternatives but there was a trust issue with the support staff due to previous members who were no longer there.  I was not sad when they announced they were outsourcing all our jobs, I laughed and gave my notice.  Told them where I was going and almost felt like telling them yeah, that's right bitches, going to move into some real shit

Sorry for the side story

I don't think you're being very fair to IDS.  People criticize IDS because they can be bypassed, but really what preventative measure can't be bypassed?  Intrusion detection is a lot more than an IDS, just like preventing attacks is a lot more than a firewall.  Neither is meant to be the only thing needed for detection/prevention.

1.  You don't have to monitor the entire organization.  You can monitor your most important or most vulnerable systems.
2.  There are around 8000 vulnerabilities found a year, but only around 13 that are commonly exploited [source].  So I think you can get by with a far more conservative ruleset than the default.
3.  You don't have to investigate and respond to every alert in real time.  You could only resond to critical alerts and leave others for forensic purposes.

I wouldn't say an IDS is appropriate for everyone, but an IDS is what you make it.  You can make a firewall a nightmare to manage by using a really conservative ruleset where you have to continually make exceptions.  You could also make an IDS a nightmare to manage by using a liberal ruleset where you get overwhelmed by false positives.  Not many people do the former, but the latter is a common mistake.

Cheap IDS software is not the issue.  It still takes time/money to setup sensors and management consoles (hardware/VMs are cheap at least) and a lot more to staff them.  Even with a low false positive rate, IDS need love and affection. 

IDS may in fact be very valuable.  I'd really like to see some objective evidence that it is valuable rather than make the assumption that it is valuable based on a general notion of defense in depth, prevention fails, etc.  We ought to have some idea of what capabilities IDS actually provides, how well it detects malicious traffic, etc.  What percentage of actual attacks in the wild can IDS detect?  What if we exclude the attacks that are easily prevented?  How do the capabilities of the IDS vary by attack type (e.g. web app vs client-side web vs botnet CnC)?

There has been some research on the effectiveness of anti-virus and it's not encouraging (act surprised).  My impression is that IDS is further behind the curve than AV.

http://blogs.cisco.com/security/the_effectiveness_of_antivirus_on_new_malware_samples/

http://www.cyveillance.com/web/docs/WP_MalwareDetectionRates.pdf