Fuzzing for Software Security Testing and Quality Assurance Ari Takanen, CTO, Codenomicon

36 Pages·2009·1.85 MB·English

by  Ari Takanen

Europe’s Premier Software Testing Event
Stockholmsmässan, Sweden
“Testing For Real, Testing For Now”
Fuzzing For Software
Security Testing & Quality
Assurance
Ari Takanen,
Codenomicon,
Finland
WWW.EUROSTARCONFERENCES.COM
Fuzzing for Software Security
Testing and Quality Assurance
Ari Takanen, CTO, Codenomicon
December 3th, 2009
EuroSTAR Conferences
About Ari Takanen
• The Past: Researcher and Lecturer
– 1998-2002
– University of Oulu
– OUSPG/PROTOS research group
– Software Quality Assurance related lectures
• The Present: Entrepreneur and Preacher
– 2001-today
– CTO of Codenomicon
– Evangelist: 10+ conference talks every year
– Author of two books:
• VoIP Security
• Fuzzing
3
About Codenomicon
• Founded in 2001, after five years of research in
product security at University of Oulu (1996-2001)
• Solutions for Global Security Challenges
– Superior security testing technologies
– Expertise in complex communication protocols, security,
and integration glue to other systems
– Collaborative security services
• Customers include:
– Manufacturers
– Defense
– Finance
– Leading Enterprises
4
Security View: Window of Vulnerability
TIME
SW - after product release
BUG APPEARS
Zero RELEASE
Exposure BUG FOUND
SW - under vulnerability analysis
VULN FOUND
Limited VULN REPORT
Exposure VULN FIX AVAIL.
SW - after the vulnerability process
PATCH RELEASE
ADVISORY RELEASE
Public
PATCH INSTALL
Exposure
5
Fuzzing - The Original Definition
• http://en.wikipedia.org/wiki/Fuzz_testing
• “Fuzz testing or fuzzing is a software testing
technique that provides random data ("fuzz") to
the inputs of a program. If the program fails (for
example, by crashing, or by failing built-in code
assertions), the defects can be noted.”
6
Fuzz by Barton Miller et al.
• http://pages.cs.wisc.edu/~bart/fuzz/fuzz.html
• “Fuzz testing a simple technique for feeding
random input to applications. While random
testing is a time-honored technique, our approach
has three characteristics that, when taken together,
makes it somewhat different from other
approaches.”
• Key difference with fuzzing was that it used testing
practices for finding security problems
• The main reasons for its success were these three
factors that simplified the approach
7
Original Fuzz Was Random
• “The input is random. We do not use any model of
program behavior, application type, or system
description. *...+”
• The goal was to test hundreds of applications, each
with a multitude of inputs
• Although the “API” (i.e. command line) was quite
similar, there was no one single definition for the
possible parameters
• They did not even try to define the parameters for
each application
• But they still do have “a model”!
8
Pass/Fail Verdict for Fuzz Was Simple
• “Our reliability criteria is simple: if the application
crashes or hangs, it is considered to fail the test,
otherwise it passes. Note that the application does
not have to respond in a sensible manner to the
input, and it can even quietly exit.”
• Very little monitoring of the IUT was used
• Any means of trying to hide the failures would have
left the problems undiscovered
• No memory leaks or heap overflows were caught
(unless they resulted in a crash)
• But there still was monitoring!
9
Fuzz Was Entirely Automated
• “As a result of the first two characteristics, fuzz
testing can be automated to a high degree and
results can be compared across applications,
operating systems, and vendors.”
• Fully automated
• General purpose tool
• Extremely easy to use
10