Book Review: Applied Network Security Monitoring

| October 31, 2014 | 1 Comment

As books go, I’m a lifelong reader, so when offered the chance to do more ‘regular’ reviews for The Ethical Hacker Network (EH-Net), I jumped at the opportunity.  The past few weeks, I’ve been buried in a GREAT read.  Applied Network Security Monitoring: Collection, Detection, and Analysis by Chris Sanders and Jason Smith is an extremely informative dive into the realm of network security data collection and analysis.  Fitting for both the offensive and defensive sides of security, the book looks closely at the various concepts, practices and tools that combine to create functional and cost-effective Network Security Monitoring (NSM) solutions for IT environments of all shapes and sizes.  For the offensive-security minded, it gives an insight into the tools and techniques used to monitor the network, and allows one to consider how best to circumvent those methods.  For the defensive-security minded, the authors do a fantastic job of equipping the reader with not only methodologies but also with tools and realistic examples.

Bear with me on this review, as this book at 496 pages is a long one, but in my opinion, an excellent resource.  I’ll do my best to give a thorough overview of the material while keeping things as concise as possible.  Hopefully, you’ll see that it’s a worthwhile read in giving a running start into the world of NSM.

Before I dive into the review, I want to open with a little about myself.  I enjoy both growing my own knowledge-base and being a catalyst to help others, as I believe that in all areas of IT (and the business world, in general), growing a team means contributing to the success of each member in order to benefit the whole.  Ironically, there’s a passage in the book that exemplifies this exact mindset, so I’ll open my review with a quote from the book:

“The premise of servant leadership is that rather than establishing leadership based upon a title or some given authority, servant leaders achieve results by giving priority to the needs of their colleagues. This humble mindset is one in which you look to help others achieve their mission so that the organization will prosper. This has the potential to breed an organization that isn’t anchored by one strong leader, but rather, a group of leaders with different strengths and weaknesses working in harmony to achieve a common mission. Although it sounds like a lofty goal, with the right mindset and buy in from all parties involved, this type of environment can become a reality.” (Applied Network Security Monitoring, page 19)

Chris Sanders, Jason Smith, and the other contributing authors bring a broad experience base to this writing.  Their time in government roles, current positions in reputable security firms, and activities in various projects, specifically related to the NSM field, lend to their credibility as experts in this arena.  As I read the book, it was obvious that they each share a passion for what they do, and their ‘From the Trenches’ notes throughout, allow their perspectives on certain items to be conveyed from folks who have obviously “been there, done that.”

The book is divided into sections then subdivided into Chapters where tools, ideas and methodological approaches are examined in detail as they apply to each topic.  I’ll do my best to give a synopsis of each as the review moves on.  As always, feel free to email me ( or respond in the EH-Net discussion for this review if you have any questions that aren’t covered.

Applied Network Security Monitoring - Book CoverThe Preface is a general overview of NSM as a whole.  The authors lay out the methodology behind NSM beginning with Collection (of data), Detection (of events or items that need to be monitored or addressed), and Analysis (the human process of reviewing what has been collected).  They stress the importance that these processes occur in a circular order, allowing refinement of each stage as previous stages complete, with the goal of continual bettering of each process over time.  They conclude by providing information for the Applied Network Security Monitoring Companion Website, as well as informing the reader that the proceeds from the book will be donated to specific charitable organizations and sites.

In Chapter 1, prior to actually moving into the stages of NSM, the reader is given definitions that are relevant to the practice.  Information is provided in order to solidify the reasons that a good NSM program / solution is put in place, and the authors clarify assumptions that will be followed in further chapters.  The authors define different levels of analysts based on existing knowledge and growth potentials.  They also provide the reason that MOST companies employ many lower-level analysts with a smaller number of truly high-level analysts, whose job is to mentor the lower-level, and help them grow and succeed.  They continue by suggesting a measurement of success in an NSM program, fostering teamwork and encouraging team members to exercise their potentials in order to maximize the impact of an NSM program.  Finally, Security Onion, a Linux distribution containing many of the NSM-related tools is introduced and basic installation instructions are presented.  Throughout the book, various examples are given focusing heavily on Security Onion and the tools both included therein, by default, as well as other freely available tools that are very worthwhile to install on top of it.

At this point, now that the basics have been addressed, and the reader has some idea of the mindset involved in NSM, the remaining chapters are broken down into related sections:  Chapters 2-6 deal with the Collection phase, Chapters 7-12 focus on Detection, and Chapters 13-15 center on Analysis.  I’ll summarize each section and chapter below.

Section 1: Collection

Chapter 2 opens with a discussion about the Applied Collection Framework model, which progresses from Defining Threats to Quantifying Risk, and into Identifying Data Feeds and Narrowing the Focus.  The authors explain that each of these areas require buy-in from various levels to ensure the proper people define the criteria at every step.  They explain the standard methods in NSM for calculating the risk involved from various threats in order to classify and correctly triage (prioritize) varying degrees of threats based on the resources in question and the value of any loss that may ensue, should those resources be compromised.  The reader is then presented with some scenarios, so that they might determine what type of data needs to be gathered and exactly from where it needs to be collected to meet the monitoring expectations of an NSM solution.  The various examples give differing viewpoints as to how data may be accessible and subsequently monitored or protected.

In Chapter 3, different types of data are discussed, such as Full Packet Capture, Session and Statistical Data, Packet String, Log and Alert Data.  The benefits and usefulness of each type are discussed, as well as the impact each might have on throughput and storage retention policies for later analysis.  For instance, they note that Full Packet Capture is obviously the most useful in many cases, because it captures EVERYTHING that crosses the wire.  For those same reasons, it’s also typically the most disk space intensive, and as such, retention policies for FPC Data typically have shorter timespans than other types.  The authors even go into detail with regards to base-lining collected data, in order to try to proactively size the storage and bandwidth needs of the NSM systems.  Next, they discuss sensor placement within the network, focusing on topics such as geographical locations as well as network segments and proximity to critical resources.  They explain the difference between sensors whose job is only to collect data, as well as those with mixed functionality, going as far as sensors that have analysts’ tools on them and how to properly size and spec the sensor hosts.  They continue by stressing the importance of ensuring that the hosts are well secured so as not to become targets or pivot points themselves.

The next chapter focuses on Session Data.  The readers are introduced to different ‘flow types’ which more or less define conversations between hosts in order to be able to later analyze statistical data or monitor specific hosts for malware, illegal access and other malicious activity.  The chapter discusses the difference between hardware and software generated data (for instance, flow records can be generated directly from core switches, while other data may be generated by software on a deployed Security Onion sensor host) and the benefits and drawbacks of each.  Lastly, the SiLK Analysis Toolset and Argus are introduced with real examples of command-line scripts and tool usage to utilize these two tools for both optimal and fine-tuned data collection for specific needs.

Chapter 5 deals directly with Full Packet Capture Data.  Here, tools such as Dumpcap (included with Wireshark) and Daemonlogger are briefly discussed with some command-line usage examples as well.  The authors detail storage considerations and retention for FPC data as well as how certain tools such as Netsniff-NG and IFPPS operate with regards to their benefits for performance and throughput capabilities (for instance, how well each would perform versus other tools in a high-data enterprise deployment).  They conclude the chapter with command line examples of the usage of some of these tools to properly calculate average expected throughput as well as using them to find services and communication streams that may be considered ‘normal’ or unnecessary to be watched as closely. For example, encrypted traffic on port 443 that in many NSM environments isn’t as closely watched, as the time and effort to decrypt the packets often outweighs the value of the data within.  In these cases, Session Data may be more valuable, as it still shows the conversations that occur, and the statistics thereof, without the disk space needed to store all packet contents over a long period of time.

In Packet String Data (Chapter 6), other alternatives to FPC data, above, begin to take shape.  With PSTR Data, the need to analyze ‘byte by byte’ isn’t necessary.  Rather, analysts might simply be looking to gather that a host is communicating over http or other port / protocol (ascertained from packet headers only) to a remote host and how much data was actually sent / received.  In this case, it’s not necessarily the data itself that is watched but the abnormality of what is transacted over the wire.  This data will later be useful, when correlated by analysts to other data in creating a big picture of an anomaly or issue.  The reader is given BASH examples of how they might obtain PSTR Data from live or saved packet captures before being introduced to tools such as URLSnarf (Dsniff), Httpry, and Justifier for gathering some data as well as Logstash (and again, some BASH examples) for parsing and presenting said data.

Section 2: Detection

In Chapter 7, the discussion begins to focus on Detection Mechanisms, Indicators of Compromise and Signatures.  Various host and network indicators are discussed, as well as signatures (such as event signatures, malware signatures, etc.), how they are often formatted / arranged and how they play a role in detecting anomalies and malicious activity from previously collected data.  The authors explain that signatures often need to be maintained and or go through life and revision cycles, as some signatures may be valuable to one organization in one form and another organization in another form.  Case in point, ports may not be the same, so a shared signature may need to be adjusted to accommodate each organization’s specific needs.  The reader is advised of some best practices with maintaining order and signature availability to analysts as well as some common tools and frameworks for doing so such as Mandiant’s OpenIOC, and STIX (developed by MITRE).

The next chapter deals with Reputation-Based Detection, that being identifying hosts and malicious activity based on publicly (or privately) maintained lists of such hosts.  Examples are given for some of the public lists, such as Malware Domain List, Zeus, SpyEye and PhishTank, Tor exit node lists and Spamhaus Block Lists.  There is also a list of a few others for which they simply provide URLs for the reader to research on their own.  The material explains that sometimes public lists can be a bad thing and lead to blocking of legitimate hosts and services, simply because, for instance, a host had once been compromised, even though that activity has since been remediated.  For that very reason the authors stress that lists often require purging to ensure that only valid hosts are kept on the lists.  BASH is used, again, with examples of retrieval and detections from manual lists as well as f from Session Data and FPC Data.  The chapter concluded with examples of the Collective Intelligence Framework (not included with Security Onion, but can be added on), Snort, Suricata and Bro with examples given for each.  Those final three tools, Snort, Suricata and Bro, are taken into further depth, in the next couple of chapters.

Chapter 9, Signature-Based Detection with Snort and Suricata takes a deeper dive into the two well-known IDS tools.  It highlights configuration, strengths of each, similarities and common features between them, and more detailed examples of each.  Discussion ensues regarding Rules, Alerts, Content Matches and MANY other topics, and, while the authors go into significant detail, it’s worth noting that this chapter could easily have been taken and developed into a book all its own.  They put a lot of effort into this chapter, and, for those who haven’t spent much time in these tools, it’s a solid primer.  The chapter concludes with a discussion on visualizing data from Snort using tools such as Snorby and Sguil.

In Chapter 10, the Bro Platform is discussed in more detail.  Examples are provided with regard to file carving and extraction from different data sources using Bro to monitor Darknets (areas on your network where blocks of unused IP addresses sit, for instance, that might be monitored for questionable activity, since any activity there would not be ‘normal’).  The reader is also introduced to using Bro to perform activity such as Emailing, Alarming or Suppressing alerts as well as adding custom data fields to Bro’s logs.  I was impressed by many of the examples in this chapter, as I’m not normally in the NSM business.  By following the examples here, I could easily see a TON of value in efficiently using Bro’s features as part of an NSM deployment.

The following chapter, Chapter 11 – Anomaly-Based Detection with Statistical Data, follows its title accordingly.  Here, the reader revisits SiLK, and how to use it to determine ‘top talkers,’ discover services, and use statistical data for other detection of anomalies.  The authors then move into how to graph / plot / display statistical data in various ways with tools such as Gnuplot, Google Charts and Afterglow.  While not the most exciting chapter in the book, the value of charting was shown in that often times an anomaly is more easily spotted by a sharp spike on a graph, for instance, or statistical data showing a lot of abnormal activity during off-peak hours.

Chapter 12 deals with Canary Honeypots and their use in Detection activity.  A honeypot is used to emulate valid hosts / services in an environment in order to attract evil-doers that may have gained access to the network, in order that their activity can be analyzed.  On one hand, their value lies in the ability to see what an attacker does and how they think, but on the other hand, if not properly secured, they can be pivot points for malicious activity.  The authors are careful to note that Honeypots can be a sore spot in some circles.  While they can certainly provide valuable data in an NSM environment, sometimes they can also bring legal issues such as entrapment and are often not worth the trouble to properly scope and configure in a trade-off of value versus time to engineer and deploy.

Section 3: Analysis

Chapter 13, Packet Analysis, is a dive into truly decoding packets with plenty of screenshots and supporting data to help the reader follow along.  The reader is initiated into understanding bytes in Hex, conversion of Hex to Binary / Decimal, and counting Bytes.  Discussion continues into Tcpdump, Tshark and Wireshark for packet analysis, following streams, analyzing protocols, and graphing or exporting data.  It concludes with a discussion on Berkeley Packet Filters and Wireshark Display Filters.

In the following chapter, Friendly and Threat Intelligence are discussed.  This includes the process of understanding correlated data in order to maintain an awareness of what is legitimate versus what is not in the environment.  The authors provide a clear picture of the NSM Intelligence Cycle, another never-ending, circular process that is refined and modeled continuously as time goes on.  It goes into further detail on Defining Requirements, Planning, Collection, Processing, Analysis and Dissemination, and how each feeds into each other to effectively determine whether or not intelligence is Friendly or Malicious as well as how to manage that data once determinations are made.

Finally, we come to Chapter 15, The Analysis Process.  By now, you’re probably thinking “Tim is a really long-winded reviewer.  I’m so glad we’re almost done.”  But seriously, if you’ve hung in this long, you should realize that this book has a plethora of foundational NSM knowledge and really is a valuable resource.  In this chapter, the reader reaches the realization that, at the end of the day, the Analysis process is the most important piece of NSM. It’s the point where all the data comes together, and the analysts can actually use that data in order to make determinations and take actions that increase the security of the environment.  This chapter explains the way that the actual Threat / Incident investigation processes proceed, and how Analysis plays a key role in handling events.  Best practices are discussed, and how to bring it all together.

The remainder of the book contains some Appendices, containing Security Onion Control Scripts (a lot of useful tidbits here), a listing of important Security Onion files and directories, Packet Header diagrams and a Decimal / Hex / ASCII Conversion chart.

Final Thoughts on Applied Network Security Monitoring

Overall, I really enjoyed the Elsevier Syngress published Applied Network Security Monitoring, and felt that, particularly for those who regularly participate in the NSM cycle / process, as well as those who really want to develop / start one, the material is a very solid step in the right direction.  The experience of the authors and the pure depth of possibilities they’ve presented were well organized and thought through, and I have only positive comments from my study.


Tim Everson, OSCE, OSCP, GPEN, C|EH AKA hayabusa is an avid pentester and security enthusiast / professional who has been involved in IT for nearly 20 years with mixed experiences in pretty much every sector of the industry from SMB to enterprise, manufacturing, education and government. He enjoys reviewing new books and courses to build his knowledgebase and challenge himself as well as to help others find appropriate learning to help them progress in the field. When he’s not tucked behind a computer screen, he’s an avid sport-bike enthusiast, a busy husband and dad, and has a passion for cartoon drawing and computer graphics / animation.

Tags: ,

Category: Book Reviews

Comments (1)

Trackback URL | Comments RSS Feed

  1. azmatt says:

    Great writeup hayabusa, it’s going on the wish list.

Leave a Reply