Here's what I was looking for in this challenge, which was based on a real-world incident involving one of my clients:
1) Which process was most unusual and therefore most likely to be the backdoor planted on the machine?
The most unusual process was the second smss.exe process (with PID 1384). Windows should only have a single process with this name. Smss is the Session Manager, the first user-mode process to run on a Windows machine during boot. The kernel activates smss.exe, which in turn, invokes all other user mode processes, acting rather like the init daemon in UNIX. There should never be more than one smss.exe processes.
Another hint that this was the evil process was the fact that Nigel couldn't kill it with Task Manager. When he tried to kill it, he got that warning message: "This is a critical system process. Task Manager cannot end this process." Of the processes included in Nigel's list, only processes named csrss.exe, lsass.exe, and smss.exe exhibit this behavior. But having one csrss.exe, one lsass.exe, and one smss.exe is totally normal. It's the extra smss.exe that stands out like a sore thumb.
I did throw in a couple of red herrings to keep you guys on your toes.
Many people suggested that the answer was the three cmd.exe processes. Sorry, but Nigel was using these to analyze the box, and they certainly are easily killable with the appropriate permissions in Task Manager.
Others thought that the renegade process was Explorer.EXE due to its strange capitalization. This is just my Windows GUI, nothing special.
A couple of your entries zoomed in on RunDll32.exe. That's a process that can take a DLL and run it like a stand-alone executable. Some legitimate tools invoke it on a machine, so it wasn't the offending process. Also, it is killable with Task Manager and administrative privileges.
Some folks pointed out the s__please_buy_ and s_my_new_book_ entries. This was just a little subliminal (or not-so-subliminal) advertising by me. These processes certainly can be killed by Task Manager.
Also, many observant folks pointed out the ibmpmsvc.exe. Yup, that's my Thinkpad's power management service. I threw that one in for those who have seen my old, beat-up Thinkpad in class. Still, that process can be killed by Task Manager as well.
By far, the most common incorrect answer was CMLUC.exe. Many of you knew that this process is associated with Orinoco wireless cards. You correctly pointed out how unusual it is to have this on a web server. I agree. However, CMLUC.exe can, in fact, easily be killed with Task Manager when logged in as Administrator, as Nigel clearly was. So, you might ask, why did I put this process there? Why, it's based on the movie, of course. It's the wireless interface for Nigel's electric guitar. He's upgraded his radio frequency guitar-to-amplifier interface to wi-fi. To economize, he just used the web server to hold the wireless card. As they say in the movie, "There's a fine line between clever and stupid!" ; )
By the way... what was the second smss.exe process really? It wasn't a worm or virus-infected application. Nope. It was just a plain ol' Netcat listener implementing a backdoor. I renamed nc.exe to smss.exe and ran it. Voila!
2) How could Nigel determine whether this process was listening on a TCP or UDP port, the user name it was running under, and the file that was executed to invoke the process? Please list any built-in or third-party tools you would use to answer this question.
I received a lot of great answers to this question. Some of my favorite tools in this genre are:
Fport by Foundstone
TCPView by Sysinternals
Some other tools that folks mentioned to use for getting more detailed information about running processes included:
Pulist from the Windows 2000 Resource Kit (now available at no extra charge from Microsoft!)
Active Ports from Smartline, Inc
Prcview from Igor Nys
Process Explorer by Sysinternals
A lot of folks mentioned using netstat, with the -nao flags. That's ok, but remember that we have a Windows 2000 server here. In Windows 2000, netstat doesn't have a -o flag, so it won't give us the PID. Even on Windows XP, when Microsoft finally threw in the -o flag for PID, I thought, "Well, thanks, but we need a lot more information, such as user name." It makes you wonder whether Microsoft really understands what people really need to secure their boxes in a real-world production environment. Oh well.... Also, please note that Windows 2000 Task Manager doesn't have a column for user name, which was (thankfully) added to Windows XP.
3) Why couldn't Nigel kill this process using the "End Process" button in the Windows Task Manager?
Quite a few folks thought Nigel couldn't kill the process because it was running with SYSTEM privileges. Actually, it was not. The second smss.exe process (that is, my alternative Netcat) was running with very limited privileges. Nigel couldn't kill it solely because of its name, due to a quirk in Windows Task Manager.
As I say in Chapter 6 of my Malware book: "If an attacker gives a backdoor a name [such as winlogon.exe, csrss.exe, and smss.exe], the Task Manager will refuse to kill it. The system gets confused, believing the backdoor process is really the vital system process. The system is over-protective. To prevent a user from accidentally killing a vital process and making the system unstable, Windows goes overboard by preventing users from killing any process with such a name... You might think that Windows would be smart enough to differentiate vital system processes from imposters by looking at the file on the hard drive the process was started from, or even its process ID number. However, Windows doesn't do this, and just assumes that any process named "winlogon.exe" or "smss.exe" must be OK." The book also includes a figure showing the error message displayed when you try to kill such a process, which is exactly what Nigel witnessed.
Enough plugs for the book (for now!)
4) How could Nigel actually kill the attacker's process without rebooting the box?
I got a lot of good suggestions here, too.
One of my favorite tools for this is TCPView, which not only shows running process that are listening on TCP and UDP ports, but also lets you kill such processes without regard to their name.
Others had suggested the following tools, all of which would work:
Prcview from Igor Nys
kill from Windows NT Resource Kit (also available at no extra charge from Microsoft)
tskill from Windows 2000 Terminal Services Session Management Tools (also available at no extra charge
pskill from Sysinternals
TaskMan+ from DiamondCS
So, let's get to the winners list. Instead of just giving out one copy of the book (as originally promised in the challenge), I'll give books to the top four winners. Hey, it's the holiday season, after all.
These four folks had the best of the best answers:
T. Brian Granier... Nice use of plot... great technical analysis. His answer is here .
Arun Darlie Koshy... Brevity is the soul of wit. These answers were brief, and exactly right. Also, good use of references. His answer is here.
Raul Siles... Very detailed analysis and good use of references. Good info about the differences between Windows 2000 and Windows XP. His answer is here.
Jay Swofford... Great logic flow... nice research. His answer is here.
Congrats, congrats, congrats!! You guys rock.
Other correct answers (sorry, no free book for you guys!) and honorable mentions (almost correct answers, still no free book) include the following brilliant folks:
Thanks again to everyone who played.