I got a call from a friend, whom we’ll call him Dave, who was having some account lock out problems on his Windows domain. He was a bit spooked as the accounts were service accounts for backup, monitoring and SQL and they all locked out within minutes of each other. This stopped two major production systems in the middle of the working day and was causing him a stream of abusive phone calls on why “his” IT systems weren’t working and what the heck was going on.
Given Dave’s network was pretty well patched, they used a good account naming standards (so not accounts named backup, admin, etc) and every device had a decent AV on it, his admin senses were warning him something hoaky was going on. I directed him to look on his domain controller for the account lock out event id’s 539. Event ID 539 has the machine name, which means you can trace back the machine causing the lock outs. The machine name came back as XP, which was odd as the company machines are named after their asset numbers for accounting purposes. Pinging the machine’s name returned its IP address but no echo replies, so it appeared to be down. As I was asking questions, three more service accounts got locked out, all from workstation XP.
On a hunch, I asked him to check his DCHP server, look for the IP address there and tell me what the MAC address was. He found it quickly and noted that it looked different from the Dell machines MAC address details.
A quick look up on http://www.coffer.com/mac_find/ confirmed with was a VMware machine. Dave was somewhat disturbed by this as they are a windows only shop, including any virtualisation software. Dave’s LAN covers several floors and a large warehouse and has 16 switches, so we went with the old trick of searching the switches’ CAM tables for the MAC address to located which switch and port this IP address was connected to. As it happens the wiring diagrams were well maintained; the switch and offending port the MAC address had been connected was down one flight of stairs. Another three accounts locked out by the same machine name while we were doing this. A somewhat angry and confused Dave popped down stairs to see what was going on.
A now very excited Dave call back telling that a guy he’d never seen before was sitting at a spare desk, plugged in to the “evil” port, working on a shiny Mac laptop. He was on the way down to the warehouse to gather up some of the bigger lads to act as back up when he confronted the intruder.
Dave’s a big fan of the show 24, and imagination had got carried away with him. I told him to turn around, call his boss and check with reception if they knew this person was first. Reception proved to be more helpful than his boss, who had no idea. The “nice young man”, according to the receptionist, was a guest of the chief finance officer. Dave ran up the stairs to the top floor, burst in to the CFO’s office demanding to know who this guy was and why he was plugged in to Dave’s network.
The shocked then annoyed CFO stared at the wild-eyed, panting Dave and calmly replied he was a consultant doing some testing highlighted in the last audit. Dave, bravely attempted to not get himself fired on the spot, blurted out that consultant was the one who’d just stopped the inventory database critical to the warehouse operations. The CFO brain connected that this was something bad and, with Dave following, rushed down to the consultant’s desk and order him to cease all activity.
Dave had all the accounts unlocked and all the services running again within the next hour. He was asked to sit in a meeting with the CFO, his boss, the consultant and the consultant’s boss. The angry CFO was demanding answers on what had happened. The two from the consultancy exchanged looks and hand over the penetration testing contract and scope of work. All signed and agreed by the CFO.
To cut a long and embarrassing (for the CFO) story short, new auditors had recommended a penetration test be carried out to assess the internal systems as IT systems where now critical to the business. The CFO decided not to involve or inform anyone from IT, against the penetration testing company’s objections, as he always dealt with these types of financial matters. He ordered an extensive testing program including password attacks, all to be carried out on site and during office hours. Dave looked over the scope of work and was horrified what the CFO had chosen, the document clearly stated some of these tests carried risk, but had been signed off on.
The outcome from this was
1) The business had a three hour outage in the middle of its working day when the consultant was in attacking mode by attempting password brute forcing on service accounts
2) The business estimated losing half a million dollars during those three hours
3) The two day pentest engagement was cancelled and a more sensible scope of work was drawn up
4) IT was included in on any and all audit requirements
5) The CFO got his wrist slapped.
Draw what you will from this tale, but always, always have written permission from someone in the business authorized to give it before even think about pen-testing a system.