An Insider's Look at the Social-Engineer.Org SE CtF at DEFCON

By Chris Hadnagy

dc-18-logo_smsq.pngI want you to picture this scene:  It is a warm day in sunny Maryland, my phone rings.  I answer it.

Me – “Chris speaking…”
Voice – “Hello Sir, this is Special Agent Smith (name changed) from the FBI, I would like to speak to you about this social engineering contest…”
Me – “Nice Dave, not falling for it.  Good try sucker!”
Voice – “Sir, I already mentioned my name is Special Agent Smith, not Dave.  It is important that we…
Me – “Blah, Blah Blah.. right Dave.  You are always trying to get me.  Nice one, almost sounds real.  Later loser…”
Moments after the phone was hung up it rings again…
Me – “Hello?”
Voice – “I would ask that you listen sir and do not hang up.  Call me back at this number… And ask for Special Agent Smith.”

This was the birth of the very first Social-Engineer.Org’s Social Engineering Capture the Flag Contest (SE CtF) at DEFCON over 2 years ago.

twitter-icon.png delicious.png

Discuss in Forums {mos_smf_discuss:Hadnagy}


Maybe we didn’t really know what it was we were getting into back then, but we were aware that this was going to be a hot topic.  That first year kicked off with amazing success with hundreds of people packed into a tiny room.  We broke DEFCON history and won a Black Badge for a first year contest.
All that is pretty cool, but can we actually learn anything from a game?  If so, what?  And how should it affect you?
The Structure
Let me first tell you how we structure the SE CtF.  The idea of a CtF is that contestants race against a clock to capture a flag or set of flags for points, and the one with the most flags or most points wins (depending on the value of each flag).

Doing this with social engineering proved to be no easy task.  We had a goal of making sure that our contest and contestants did nothing to embarrass a company by releasing private information about them.  We wanted to make sure that a company’s details were not the target.  We chose flags that would provide small bits of innocuous information like who handles dumpster removal, who manages the cafeteria and things like that.  The goal again was to demonstrate if companies who spend millions on security and have large security teams are training their employees on how to spot and mitigate against the simplest of social engineering attacks.

We assign each contestant a target company.  They are given two weeks to write a professional report based on OPEN SOURCE information gathering on their target.  They are looking for information leakage to build attack vectors and at the same time build their knowledge-base for the future calls during the live contest.

During DEFCON each contestant is given 25 minutes to make a call to their target company and attempt to elicit the flags from them using one or a series of pretexts.  This data is collected, scores are given, and the winner is the one with the most points  Afterwards, a report is written just as it would be after a professional penetration test.

Defcon 18 Social-Engineer.Org SECTF Report

Defcon 19 Social-Engineer.Org SECTF Report

The Year of the Hack

Then in 2011 something happened.  Hacktivism groups sprouted up and began to lay waste to major companies all over the world, exposing their deepest, darkest secrets for all to read.  In one interview with a member of the most prominent group, she said they “used social engineering in every attack” they launched throughout that year.

This put the spotlight on social engineering like never before.  But it also added relevancy to the CtF results from the previous years.  Here we have major Fortune 1000 companies claiming to perform consistent security awareness programs, and yet these very companies are failing the SE CtF each year.  And all of this was accomplished with mostly untrained and unskilled social engineers.

What would happen if skilled social engineers were to try? 2011 answered that, and the answer wasn’t good.

What Can We Learn?

The SE CTF is broken down into many categories that helps us to learn.  We break down the info into a business sector, type of flags, who answers the phone, pretexts used as well as other details.

In one case during Defcon 18 we had a contestant pull an amazing pretext.  He was an employee for a small engineering firm that focused on social media.  He was tasked with doing competitive quotes on some technology they wanted to buy.  The sales guy tried his hardest to make the contestant email him and wait, but our contestant would have none of that.  Instead, he used artificial time constraints, the plea for help and showing interest in the target helped build rapport.  The rapport must have been strong, as with very little effort the contestant had the target answering every question he asked. 

This kind of story is not uncommon.  Contestant after contestant seemed to have amazing success in getting the targets to answer every question asked.  By the end of the first year we were sure that the state of security in American Big Business was in poor straights.

By year two, we were hopeful that companies took to heart much of the previous year’s lessons as well as the increase in hacking activity.  But the Defcon 19 event proved to be even scarier than the first year.  Contestants were able to obtain more information with less time and less sophisticated pretexts.

What were some of the lessons learned from all this?

Overwhelmingly our first observations are that the larger the company and the more money spent does NOT automatically equate to better security.  Next, the more technical the company does NOT mean they will be more secure.

The datasets we have at our disposal also show that the level of skill of the social engineer didn’t matter much in regards to overall success but only in the speed of that success.  People who either have phone skills or previous experience in SE seem to get the data faster, but even people who are doing this for the first time have surprisingly great success of getting the flags over the phone.

One silver lining in this dark cloud is that retail companies providing regular and consistent Security Awareness Training seemed to have better results.  Not only were they harder to infiltrate with pretexts, but the training also seemed to be working in the sense that they were able to ward off the attacks many times.  What this statistic shows even better was that the majority of those who were security conscious were women.

If we compile all of these statistics, we can basically boil it down to a few simple facts:

1. Companies that have consistent security awareness training fared better
2. Quantity of dollars spent does NOT equate to quality of security
3. Quality, effective Security Awareness Training is lacking in most industries called by the contestants

The reports that we produce each year can help your company compare in order to see where you may fare in real-life circumstances.  Many a large company have called us and asked if they can use the report as part of their own Security Awareness Training programs as well in their internal auditing for ideas.

As software vendors get better at creating hardened software, and hardware vendors are creating new and improved devices that make the perimeter even more difficult to penetrate, social engineering will remain the easiest way into a company.  Humans will always be the biggest weakness, and because of that they need constant training, reminders and education to stay secure.


If you are interested in seeing how this all operates, be sure to check out the very special DEFCON 20 Social-Engineer.Org SE CtF.  It is entitled “Battle of the SExes” as each target will have a male and female caller.  Not only will we be able to capture the high-end data as in previous years, but also this year will prove if there is an advantage between the genders.

kids-ctf.jpgOne final invite, if you are coming to DEFCON in Las Vegas and want to bring the kids but afraid they will be bored, check out the Social Engineering CtF for Kids.  We have developed an intense track of social engineering skills, puzzles, ciphers, lock picking and more that will challenge your kids and help them acquire essential skills to enhance their education and personal security in the future.  You can register here.

Till next month!

If you have comments or questions – please feel free to reach out to me at  

Chris Hadnagy, aka loganWHD, has been involved with computers and technology for over 14 years. Presently his focus is on the "human" aspect of technology such as social engineering and physical security. Chris has spent time in providing training in many topics around the globe and also has had many articles published in local, national and international magazines and journals. He is also the lead developer of Social-Engineer.Org as well as the author of the best-selling book, Social Engineering: The Art of Human Hacking.

He has launched a line of professional social engineering training and pen testing services at Social-Engineer.Com. His goal is to help companies remain secure by educating them on the methods the "bad guys" use. Analyzing, studying, dissecting then performing the very same attacks used by malicious hackers on some of the most recent attacks (i.e. Sony, HB Gary, LockHeed Martin, etc), Chris is able to help companies stay educated and secure. Chris can be found online at, and twitter as @humanhacker.

Copyright ©2021 Caendra, Inc.

Contact Us

Thoughts, suggestions, issues? Send us an email, and we'll get back to you.


Sign in with Caendra

Forgot password?Sign up

Forgot your details?