Typically during penetration tests, scanners are used to detect vulnerabilities. Sometimes security professionals may want to go undetected to test the response of the blue team (aka defensive security) and the security controls of an organization. However, vulnerability scanners are quickly detected due to the amount of network traffic generated by these tools. There are also times that standard, automated scans may miss vulnerabilities. To solve for these issues, manual vulnerability testing is required. Vulnerability scanners should always be used during pentests to ensure that you detect the easy-to-find vulnerabilities quickly and more efficiently, but manual testing should also be done alongside regular scans. Manual vulnerability detection takes more effort and knowledge, but it is a much-needed skill for the advanced pentester. This article will show you how!
All security professionals should be familiar with the Penetration Testing Execution Standard (PTES) as a great set of technical guidelines for performing pentests. It covers the following main sections of a typical pentesting methodology:
- Pre-engagement Interactions
- Intelligence Gathering
- Threat Modeling
- Vulnerability Analysis
- Post Exploitation
This article covers the following bolded sections from PTES: Intelligence Gathering, Vulnerability Analysis, and Exploitations. We won’t go in too deep on the Exploitation step; however, we will show how to determine possible exploits from the data gathered during the Intelligence Gathering and Vulnerability Analysis steps.
The tools you will need to pentest are installed on the Kali Linux distribution, and, if they are not, most are available for install from the Kali Linux repositories. ParrotOS is another great Linux distribution, as is Ubuntu with TrustedSec’s Pen Tester Framework (PTF). PTF is a script that installs the most common pentesting tools on Linux offering similar tools to Kali. If you are a Windows fan, you should checkout FireEye’s Commando VM. Commando VM comes with most of the common pentesting tools installed. You can install any of these as a bare metal install directly on your computer or run them in a VM. I prefer running a Kali VM on a host OS such as MacOS, Windows, or Ubuntu Linux. MacOS or Windows are good, because you can use Microsoft Office to write your pentest reports.
For your lab you will need some VMs to attack. Metasploitable 2 and 3 are good options. Metasploitable 3 can be setup on Windows giving you a Windows VM to attack. VulnHub.com is a great place to find vulnerable VMs for your lab. Some of the VMs on VulnHub were used in CTFs at past conferences. Hack.me is also a great place to practice and share vulnerable web apps.
Intelligence gathering is going to be a lot different between internal and external targets. With external targets OSINT (Open Source Intelligence) tools are more useful. Shodan is like the hacker’s search engine. You can find vulnerable devices and websites. I used Shodan to find an FTP server during a pentest that wasn’t identified in the scope. Shodan found an FTP header with the company name. Recon-ng is another great reconnaissance tool and most useful for external targets. In both scenarios scanning for ports and services are how you collect information on the target. In pentests you would typically use a vulnerability scanner like Nessus or Nexpose. Since the focus of this article is on manual testing, we will not cover vulnerability scanners. Nmap (Network Mapper) is a very popular and powerful port and service scanner. It happens to be one of my favorite tools and the one I use the most.
Network Port and Service Discovery
We will start with nmap to discover network ports and services on our targets. The more you know about your target, the easier time you will have identify vulnerabilities and exploiting them. We are going to touch on the more common nmap options in the table below including my favorites which I use during pentests. I recommend that you spend some time learning nmap. The nmap website is good place to start. It is a very powerful tool and has a powerful arsenal of Nmap Scripting Engine (NSE) scripts.
|-sV||Tests for service and version|
|-sC||Runs default NSE scripts can be obtrusive|
|-A||All (OS versions)|
|-Pn||Treats all host as online and skips probing|
|-iL||Used to scan a hosts from a file (nmap -iL host.txt)|
|-p||Ports, can be specified individually -p80 or multiple -p80,443,8080 or all 65k ports -p1-65535|
|-p-||All ports, equal to -p1-65535|
|-oA||Out puts to nmap file format, greppable format and XML|
|–script=vuln||NSE script to detect vulnerabilities|
In this first example, we are scanning a single host:
This example is scanning a range of hosts:
This example is scanning a 24-bit subnet:
The previous examples were simple scans. We did not specify the ports or port range, so it tests 1000 TCP ports by default. Expanding the scan to include all 65535 ports can find ports and services that might be missed otherwise.
The following nmap scan is a syn scan and set to not probe.
This nmap scan adds the service option. Notice the more detailed information on the target.
This nmap scan adds the default script option. Notice even more information is enumerated.
Vulnerability Detection and Analysis
We were able to enumerate a lot of information by manually customizing our nmap scans. Let’s continue down this same path, and, instead of using canned vulnerability scanners, we can utilize the handy built-in feature known as NSE. This will allow us to extend our custom nmap scans even further to also include vulnerability detection. But first, what exactly is NSE?
Nmap Scripting Engine
According to the Nmap documentation, “The Nmap Scripting Engine (NSE) is one of Nmap’s most powerful and flexible features. It allows users to write (and share) simple scripts to automate a wide variety of networking tasks. Those scripts are then executed in parallel with the speed and efficiency you expect from Nmap.”
NSE scripts are powerful and can be used to get more detailed information on the target. NSE scripts can be used to test certain protocols and for specific vulnerabilities. Peruse all of the available NSE scripts to see how easily nmap can be extended.
The following nmap scan adds the “vuln” NSE script to detect vulnerabilities. Notice the enumerated vulnerabilities with some even including the CVE number.
So we can scan entire subnets, a specified range of hosts and even hone in on a single machine. We were able to see open ports and also what services might be running on those ports. We then were able to see that one such service is vulnerable and were provided with a CVE. But how do we turn that information and venture into the next step in the process?
Searchsploit is a tool that allows you to search for exploits that could be used against vulnerabilities. Searchsploit is included in Kali and searches through a locally stored copy of the Exploit-DB repo of exploits. Exploit-DB is a popular website that stores exploits and is searchable. You can search the Exploit-DB site for vulnerabilities directly, but there may be times on a pentest that content filtering proxies and endpoint protect might prevent you from downloading exploits. So, having an entire copy that is searchable locally prevents us from hitting any roadblocks.
In the following example we use Searchsploit to find one of the detected vulnerabilities from the nmap scan that used the vuln NSE script.
From the picture above, please take note of the path and specific location from our local copy of the repository on my Kali VM. To use this exploit, we execute it from the following path: /usr/share/exploitdb/exploits/multiple/dos/8976.pl
Here is what the same search on the Exploit-DB site looks like. Notice that it is the same Perl script exploit from our local search.
To use the exploit from Exploit-DB, download it and execute the file as above.
In this next example, I focused my scan on port 1524 on the Metasploitable 2 VM.
The nmap scan shows a root shell open on port 1524, so we use a common technique and connect using nc (netcat). You can also see that we are connected as root.
We scanned our network using nmap and used NSE to find vulnerabilities normally associated with network-based hosts. But the hosts we found may also be running some web services. So we should manually use a tool that is more adept than nmap at specifically scanning web apps. Our next tool up to bat is Nikto, a good tool for detecting web-based vulnerabilities. Along with vulnerabilities Nikto lists the Open Source Vulnerability Database (OSVDB) number which can be used to get more information on the vulnerability.
The VM used for this demo was used in a CTF. If we look at the directories discovered by Nikto and browse to the ‘/flag/’ directory, we discover the flag.
This article demonstrates some basics of manual vulnerability detection which is also part of manual pentesting. Even if you use a vulnerability scanner, manual vulnerability detection techniques can help you find vulnerabilities that might be missed. I have done pentests where I discovered vulnerabilities with Nikto or other tools that the Nessus vulnerability scanner missed. Similarly, I have found vulnerabilities with a particular vulnerability scanner that was missed by another popular vulnerability scanner.
We all know the saying, “Trust but verify.” We can trust the results of our tools, but we can further verify the results with multiple tools, non-default usage methods and human intuition. In doing so, you not only become much more aware of the differences in various tools but also learn more of their capabilities. But most importantly, your penetration tests become much more thorough.
Phillip Wylie is a Principal InfoSec Engineer on the Assessment Services Penetration Testing Team in the financial sector. Phillip is also an Adjunct Instructor at Richland College teaching Ethical Hacking and System Defense, a Bugcrowd Ambassador and the founder of The Pwn School Project. Phillip has over 21 years of experience in InfoSec and IT and has performed pentests on networks, wireless networks, applications including thick client, web application and mobile. Phillip has a passion for sharing, mentoring and educating. This passion was his motivation to start teaching and founding The Pwn School Project, a free monthly educational meetup with a focus on hacking. Phillip holds the following certifications; CISSP, NSA-IAM, OSCP, GWAPT. Follow him @PhillipWylieTags: exploit nikto nmap nse pentest vulnerability