Advice for running a security competition

By Cameron Clark

As the field of computing security continues to grow, events and organizations are beginning to be spawned within the field. This can take the form of school clubs like HackUCF from University of Central Florida, SPARSA from Rochester Institute of Technology; to organizations like the Collegiate Cyber Defense Competition which is run by Raytheon.

One of the unique advantages our field has is the ability to practice our skills in minimal environments. Between virtualization and the low cost of computing power, the barrier to entry is quite low. With the ability to practice your skills, naturally a competitive aspect is introduced. Granted, competition in security has been there forever, a constant battle of us out thinking and preparing for attackers. However a more gamified version of it is has been gaining popularity over the years. It started to gain recognition with the Collegiate Cyber Defense Competition. Now clubs within the previously mentioned schools and more are starting to create their own iterations of these security competitions.

This year I had the privilege of running SPARSA’s Information Security Talent Search. A competition where we have multiple Blue teams which defend an enterprise like networks against our Red Team, a group of hand picked people from the industry which perform multiple attack techniques against the Blue teams. Another twist that we’ve added is that the Blue teams are also allowed to attack each other. I learned a lot from running this competition and wanted to provide some advice to people who wish to put together a competition of their own like this.

First and foremost, make the network realistic. A large goal of the competition is for the competitors to learn skills that can help them in the real world. If you fill your network with End of Life systems like Windows XP or Solaris 8, you’re going to handicap your competitors and their going to learn things that won’t be of use to them down the road. Your best bet is to use newer versions of systems or ones that have been released within the last 10 years. It will make your life easier as setting it up won’t be as problematic and it will be a huge benefit for the competitors.

A second thought is not to restrict your red team too heavily. They’re here on their own time doing this for fun, so let them have some fun. The obvious rule you have to put forward is that they make their attacks a learning experience. If they were to just delete the competitors box, there’s not a lot to be learned from that. But if the red team is able to leverage some interesting attacks and leave a trail of their actions, the competitor can gain a lot from that experience. At the end of the day the Red Team members know what they’re doing, so you have to place some faith into them and let them do their thing.

One of the most important pieces of advice, automate deployment and testing. A huge advantage NECCDC had this year was their utilization of devops tools like Ansible to automate deployment of the systems within each team’s network. This allows for constancy across all the networks, and easy redeployment if a team’s system was to be corrupted or lost.

Make sure you have clear, defined rules for what happens if a team breaks said rules. Like if a team breaks rule X then they will lose Y amount of points or they’ll be suspended for an hour, some punishment that you see fit, just make sure it’s clearly written somewhere and all teams are made aware of it. This year we had a situation where a team broke one of our rules, but we didn’t have a clear description of what the punishment would be for the discrepancy. So we were stuck with trying to decide on a punishment that was suitable but wouldn’t cause issues with the rest of the competition, a situation you don’t want to be in.

Document everything. When making the systems for the competition, have a list of every command that was ran and all the application settings, go through and put your notes down of why you ran said command or why you needed those settings. This will help for during the competition when trying to troubleshoot, you’ll have all the information you need about what’s running on that system. It will also be useful down the road. If someone wants to recreate your environment or make their own similar version, it will be extremely useful to have a well documented working version to refer to.

The final piece of advice is to make sure you have multiple people who have a full understanding of how the topology works. Having only one mission critical person can cause a lot of problems. If they were to suddenly disappear and something went wrong, it would cause a significant hitch in the competition. Having multiple people with a full knowledge will allow you to be able to fix multiple problems at once and people can leave and go without causing too many problems.

These are just some things that I learned the hard way from running a security competition. They are by no means a guideline on how to run your competition, just things to consider as each competition will be different at its core. Hopefully interest in these competitions continue to grow and more professional organizations pursue them so we can have some solid examples of how to run a proper competition.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s