Fuzzing for Software Security Testing and Quality Assurance

345 Pages·2018·24.12 MB·English

by  Ari Takanen

Fuzzing for Software
Security Testing and
Quality Assurance
Second Edition
6760 Book.indb 1 12/22/17 10:50 AM
For a listing of recent titles in the
Artech House Information Security and Privacy Library,
turn to the back of this book.
6760 Book.indb 2 12/22/17 10:50 AM
Fuzzing for Software
Security Testing and
Quality Assurance
Second Edition
Ari Takanen
Jared DeMott
Charlie Miller
Atte Kettunen
artechhouse.com
6760 Book.indb 3 12/22/17 10:50 AM
Library of Congress Cataloging-in-Publication Data
A catalog record for this book is available from the U.S. Library of Congress
British Library Cataloguing in Publication Data
A catalog record for this book is available from the British Library.
ISBN 13: 978-1-60807-850-9
Cover design by John Gomes
© 2018 Artech House
685 Canton Street
Norwood, MA
All rights reserved. Printed and bound in the United States of America. No part of
this book may be reproduced or utilized in any form or by any means, electronic or
mechanical, including photocopying, recording, or by any information storage and
retrieval system, without permission in writing from the publisher.
All terms mentioned in this book that are known to be trademarks or service marks
have been appropriately capitalized. Artech House cannot attest to the accuracy of this
information. Use of a term in this book should not be regarded as affecting the validity
of any trademark or service mark.
10 9 8 7 6 5 4 3 2 1
6760 Book.indb 4 12/22/17 10:50 AM
This book is dedicated to our families and friends . . .
. . . and also to all quality assurance specialists and security experts
who are willing to share their knowledge and expertise
to enable others to learn and improve their skills.
6760 Book.indb 5 12/22/17 10:50 AM
6760 Book.indb 6 12/22/17 10:50 AM
Contents
Foreword from the First Edition xv
Foreword to the Second Edition xix
Preface from the First Edition xxiii
Preface to the Second Edition xxv
ChApTEr 1
Introduction 1
1.1 Software Security 2
1.1.1 Security Incident 4
1.1.2 Disclosure Processes 5
1.1.3 Attack Surfaces and Attack Vectors 6
1.1.4 Reasons Behind Security Mistakes 9
1.1.5 Proactive Security 10
1.1.6 Security Requirements 12
1.2 Software Quality 13
1.2.1 Cost-Benefit of Quality 14
1.2.2 Target of Test 16
1.2.3 Testing Purposes and Test Verdicts 16
1.2.4 Structural Testing 20
1.2.5 Functional Testing 21
1.2.6 Code Auditing 21
1.3 Introduction to Fuzzing 22
1.3.1 Brief History of Fuzzing 22
1.3.2 Fuzzing Overview 26
1.3.3 Vulnerabilities Found with Fuzzing 27
1.3.4 Fuzzer Types 28
1.3.5 Logical Structure of a Fuzzer 31
1.3.6 Fuzzing Process 32
1.3.7 Fuzzing Frameworks and Test Suites 33
1.3.8 Fuzzing and the Enterprise 34
1.4 Book Goals and Layout 35
ChApTEr 2
Software Vulnerability Analysis 37
2.1 Purpose of Vulnerability Analysis 38
vii
6760 Book.indb 7 12/22/17 10:50 AM
viii Contents
2.1.1 Security and Vulnerability Scanners 38
2.2 People Conducting Vulnerability Analysis 40
2.2.1 Hackers 42
2.2.2 Vulnerability Analysts or Security Researchers 42
2.2.3 Penetration Testers 43
2.2.4 Software Security Testers 43
2.2.5 IT Security Engineers 43
2.3 Target Software 44
2.4 Basic Bug Categories 44
2.4.1 Memory Corruption Errors 44
2.4.2 Web Applications 52
2.4.3 Brute Force Login 55
2.4.4 Race Condition 55
2.4.5 Denial of Service 56
2.4.6 Session Hijacking 56
2.4.7 Man in the Middle 56
2.4.8 Cryptographic Attacks 57
2.5 Bug Hunting Techniques 57
2.5.1 Reverse Engineering 57
2.5.2 Source Code Auditing 60
2.6 Fuzzing 61
2.6.1 Basic Terms 62
2.6.2 Hostile Data 62
2.6.3 Number of Tests 64
2.7 Defenses 64
2.7.1 Why Fuzzing Works 65
2.7.2 Defensive Coding 65
2.7.3 Input Verification 66
2.7.4 Hardware Overflow Protection 67
2.7.5 Software Overflow Protection 68
2.8 Summary 71
ChApTEr 3
Quality Assurance and Testing 73
3.1 Quality Assurance and Security 73
3.1.1 Security in Software Development 74
3.1.2 Security Defects 75
3.2 Measuring Quality 75
3.2.1 Quality Is About Validation of Features 75
3.2.2 Quality Is About Finding Defects 78
3.2.3 Quality Is a Feedback Loop to Development 78
3.2.4 Quality Brings Visibility to the Development Process 79
3.2.5 End Users’ Perspective 79
3.3 Testing for Quality 79
3.3.1 V-Model 79
3.3.2 Testing on the Developer’s Desktop 80
6760 Book.indb 8 12/22/17 10:50 AM
Contents ix
3.3.3 Testing the Design 81
3.4 Main Categories of Testing 81
3.4.1 Validation Testing versus Defect Testing 81
3.4.2 Structural versus Functional Testing 81
3.5 White-Box Testing 82
3.5.1 Making the Code Readable 82
3.5.2 Inspections and Reviews 82
3.5.3 Code Auditing 82
3.6 Black-Box Testing 85
3.6.1 Software Interfaces 85
3.6.2 Test Targets 86
3.6.3 Fuzz Testing as a Profession 86
3.7 Purposes of Black-Box Testing 88
3.7.1 Conformance Testing 89
3.7.2 Functional Security Testing 89
3.7.3 Functional Safety Testing 89
3.7.4 Interoperability Testing 90
3.7.5 Performance Testing 90
3.7.6 Robustness Testing 90
3.8 Testing Metrics 91
3.8.1 Specification Coverage 91
3.8.2 Input Space Coverage 91
3.8.3 Interface Coverage 91
3.8.4 Code Coverage 92
3.9 Black-Box Testing Techniques for Security 92
3.9.1 Load Testing 92
3.9.2 Stress Testing 92
3.9.3 Security Scanners 93
3.9.4 Unit Testing 93
3.9.5 Fault Injection 93
3.9.6 Syntax Testing 94
3.9.7 Negative Testing 96
3.9.8 Regression Testing 98
3.10 Testing in Continuous Integration 99
3.11 Summary 100
ChApTEr 4
Fuzzing Metrics 101
4.1 Threat Analysis and Risk-Based Testing 103
4.1.1 Threat Trees 104
4.1.2 Threat Databases 105
4.1.3 Ad Hoc Threat Analysis 105
4.2 Transition to Proactive Security 107
4.2.1 Cost of Discovery 108
4.2.2 Cost of Remediation 114
4.2.3 Cost of Security Compromises 115
6760 Book.indb 9 12/22/17 10:50 AM