Riley "Caezar" Eller: Capture the Flag Games: Measuring Skill with Hacking Contests (English)

Black Hat Briefings, Japan 2004 [Audio] Presentations from the security conference

"With the cost of security experts increasing each year, it is expensive to audit critical systems as often as is needed. Worse yet, it is difficult to know how much to trust the reports since the worst consultants give the most positive answers. In order to address this problem, Caezar proposes a system for ranking the merit of security experts alone or in teams. Based on his years of experience with DEFCON Capture the Flag games, Caezar shows the difficulty of solving this problem in a fair and reliable manner. He will demonstrate the state-of-the-art technique used in this year's game and show insight into the eight teams, concentrating on the top two. Then, beginning with discussion of the failings of the International Chess Federation's (FIDE) ""Elo"" rating scheme, Caezar will propose a complete framework for ranking security professionals. By night, he donates his time and energy to the Ghetto Hackers organization; a group best known for dominating the annual DEFCON Capture the Flag contest. As the creator of the Root Fu scoring system, Caezar is the foremost authority on security contest scoring. By day, Mr. Eller has extensive experience in Internet embedded devices and protocol security. He invented fully automatic web vulnerability analysis and ASCII-armored stack overflow exploits, and contributed to several other inventions including a pattern language for describing network attacks. Mr. Eller's credits include the Black Hat Security Briefings and Training series, the Meet the Enemy seminars, ""Hack Proofing Your Network: Internet Tradecraft"", and the ""Caezar's Challenge"" think tank."