Software Test Attacks to Break Mobile and Embedded Devices

364 Pages·2013·19.902 MB·English

by  Jon Duncan Hagar

Software Engineering
CHAPMAN & HALL/CRC INNOVATIONS IN
SOFTWARE ENGINEERING AND SOFTWARE DEVELOPMENT
Software Test Attacks
Software Test Attacks
M
to Break Mobile and S
o
o
ft
b
Embedded Devices to Break Mobile and
iw
l
e
a

ar
e
n Embedded Devices

dT
“Using the framework of attacks popularized by James Whittaker’s books, Jon Duncan Ee
s
Hagar describes those that are relevant here and extends the approach with new attacks mt
specifically for mobile and embedded systems. He provides detailed information and A Jon Duncan Hagar
b
guidance on how to test more effectively and efficiently in the mobile and embedded tt
e
world.” da
—From the Foreword by Dorothy Graham, Software Testing Consultant dc
k
e
“Every tester who wants to keep current needs to read this book, and you can read with s
d

confidence knowing you are being guided by the best in this business.” t
Do
—From the Foreword by Lisa Crispin, Agile Testing Coach and Practitioner
e
B
Using a mix-and-match approach, Software Test Attacks to Break Mobile and v
r
Embedded Devices presents an attack basis for testing mobile and embedded systems. ice
The book focuses on attack-based testing that can be used by individuals and teams. ea
sk
The numerous test attacks show you when a software product does not work (i.e., has

bugs) and provide you with information about the software product under test. The book
guides you step by step starting with the basics. It explains patterns and techniques
ranging from simple mind mapping to sophisticated test labs.
Features
• Provides over 30 specific software test attacks needed to find bugs/errors in today’s
mobile and smart devices
• Explains how to create test labs, facilities, and tools to aid in conducting the attacks
• Addresses white and black box test approaches as well as risk-based testing, H
a
model-driven tests, agile testing, and mathematical approaches
g
• Details the error taxonomy upon which the attacks are based a
r
• Presents checklists for user interface and game evaluations
• Includes practice exercises in each chapter
K16700
K16700_Cover.indd 1 8/8/13 12:48 PM
Software Test Attacks to Break
Mobile and Embedded Devices
Chapman & Hall/CRC Innovations in Software Engineering
and Software Development
Series Editor
Richard LeBlanc
Chair, Department of Computer Science and Software Engineering, Seattle University
AIMS AND SCOPE
This series covers all aspects of software engineering and software development. Books
in the series will be innovative reference books, research monographs, and textbooks at
the undergraduate and graduate level. Coverage will include traditional subject matter,
cutting-edge research, and current industry practice, such as agile software development
methods and service-oriented architectures. We also welcome proposals for books that
capture the latest results on the domains and conditions in which practices are most ef-
fective.
PUBLISHED TITLES
Software Development: An Open Source Approach
Allen Tucker, Ralph Morelli, and Chamindra de Silva
Building Enterprise Systems with ODP: An Introduction to Open
Distributed Processing
Peter F. Linington, Zoran Milosevic, Akira Tanaka, and Antonio Vallecillo
Software Engineering: The Current Practice
Václav Rajlich
Fundamentals of Dependable Computing for Software Engineers
John Knight
Introduction to Combinatorial Testing
D. Richard Kuhn, Raghu N. Kacker, and Yu Lei
Software Test Attacks to Break Mobile and Embedded Devices
Jon Duncan Hagar
Software Designers in Action: A Human-Centric Look at Design Work
André van der Hoek and Marian Petre
CHAPMAN & HALL/CRC INNOVATIONS IN
SOFTWARE ENGINEERING AND SOFTWARE DEVELOPMENT
Software Test Attacks to Break
Mobile and Embedded Devices
Jon Duncan Hagar
MATLAB® is a trademark of The MathWorks, Inc. and is used with permission. The MathWorks does not warrant the
accuracy of the text or exercises in this book. This book’s use or discussion of MATLAB® software or related products
does not constitute endorsement or sponsorship by The MathWorks of a particular pedagogical approach or particular
use of the MATLAB® software.
CRC Press
Taylor & Francis Group
6000 Broken Sound Parkway NW, Suite 300
Boca Raton, FL 33487-2742
© 2014 by Taylor & Francis Group, LLC
CRC Press is an imprint of Taylor & Francis Group, an Informa business
No claim to original U.S. Government works
Version Date: 20130611
International Standard Book Number-13: 978-1-4665-7531-8 (eBook - PDF)
This book contains information obtained from authentic and highly regarded sources. Reasonable efforts have been
made to publish reliable data and information, but the author and publisher cannot assume responsibility for the valid-
ity of all materials or the consequences of their use. The authors and publishers have attempted to trace the copyright
holders of all material reproduced in this publication and apologize to copyright holders if permission to publish in this
form has not been obtained. If any copyright material has not been acknowledged please write and let us know so we may
rectify in any future reprint.
Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or uti-
lized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopy-
ing, microfilming, and recording, or in any information storage or retrieval system, without written permission from the
publishers.
For permission to photocopy or use material electronically from this work, please access www.copyright.com (http://
www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA 01923,
978-750-8400. CCC is a not-for-profit organization that provides licenses and registration for a variety of users. For
organizations that have been granted a photocopy license by the CCC, a separate system of payment has been arranged.
Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for
identification and explanation without intent to infringe.
Visit the Taylor & Francis Web site at
http://www.taylorandfrancis.com
and the CRC Press Web site at
http://www.crcpress.com
Contents
Foreword by Dorothy Graham, xi
Foreword by Lisa Crispin, xiii
Preface, xv
Acknowledgments, xvii
Copyright and Trademarks Declaration Page, xix
Introduction, xxi
Author, xxxiii
Chapter 1 ◾ Setting the Mobile and Embedded Framework 1
OBJECTIVES OF TESTING MOBILE AND EMBEDDED SOFTWARE SYSTEMS 1
WHAT IS EMBEDDED SOFTWARE? 2
WHAT ARE “SMART” HANDHELD AND MOBILE SYSTEMS? 3
WHY MOBILE AND EMBEDDED ATTACKS? 5
FRAMEWORK FOR ATTACKS 6
BEGINNING YOUR TEST STRATEGY 6
ATTACKS ON MOBILE AND EMBEDDED SOFTWARE 8
IF YOU ARE NEW TO TESTING 9
AN ENLIGHTENED TESTER MAKES A BETTER TESTER 10
Chapter 2 ◾ Developer Attacks: Taking the Code Head On 13
ATTACK 1: STATIC CODE ANALYSIS 14
ATTACK 2: FINDING WHITE-BOX DATA COMPUTATION BUGS 21
ATTACK 3: WHITE-BOX STRUCTURAL LOGIC FLOW COVERAGE 25
TEST COVERAGE CONCEPTS FOR WHITE-BOX STRUCTURAL TESTING 28
NOTE OF CONCERN IN MOBILE AND EMBEDDED ENVIRONMENTS 29
v
vi   ◾   Contents
Chapter 3 ◾ Control System Attacks 33
ATTACK 4: FINDING HARDWARE–SYSTEM UNHANDLED
USES IN SOFTWARE 33
ATTACK 5: HARDWARE-TO-SOFTWARE AND
SOFTWARE-TO-HARDWARE SIGNAL INTERFACE BUGS 39
ATTACK 6: LONG-DURATION CONTROL ATTACK RUNS 45
ATTACK 7: BREAKING SOFTWARE LOGIC AND/OR CONTROL LAWS 49
ATTACK 8: FORCING THE UNUSUAL BUG CASES 54
Chapter 4 ◾ Hardware Software Attacks 59
ATTACK 9: BREAKING SOFTWARE WITH HARDWARE
AND SYSTEM OPERATIONS 59
SUB-ATTACK 9.1: BREAKING BATTERY POWER 65
ATTACK 10: FINDING BUGS IN HARDWARE–SOFTWARE
COMMUNICATIONS 66
ATTACK 11: BREAKING SOFTWARE ERROR RECOVERY 69
ATTACK 12: INTERFACE AND INTEGRATION TESTING 74
SUB-ATTACK 12.1: CONFIGURATION INTEGRATION EVALUATION 80
ATTACK 13: FINDING PROBLEMS IN SOFTWARE–SYSTEM
FAULT TOLERANCE 80
Chapter 5 ◾ Mobile and Embedded Software Attacks 89
ATTACK 14: BREAKING DIGITAL SOFTWARE COMMUNICATIONS 89
ATTACK 15: FINDING BUGS IN THE DATA 94
ATTACK 16: BUGS IN SYSTEM-SOFTWARE COMPUTATION 97
ATTACK 17: USING SIMULATION AND STIMULATION TO DRIVE
SOFTWARE ATTACKS 101
Chapter 6 ◾ Time Attacks: “It’s about Time” 107
ATTACK 18: BUGS IN TIMING INTERRUPTS
AND PRIORITY INVERSIONS 108
STATE MODELING EXAMPLE 114
ATTACK 19: FINDING TIME-RELATED BUGS 116
ATTACK 20: TIME-RELATED SCENARIOS, STORIES, AND TOURS 121
ATTACK 21: PERFORMANCE TESTING INTRODUCTION 125
SUPPORTING CONCEPTS 139
COMPLETING AND REPORTING THE PERFORMANCE ATTACK 140
WRAPPING UP 140
Contents   ◾   vii
Chapter 7 ◾ H uman User Interface Attacks: “The Limited
(and Unlimited) User Interface” 143
HOW TO GET STARTED—THE UI 144
ATTACK 22: FINDING SUPPORTING (USER)
DOCUMENTATION PROBLEMS 146
SUB-ATTACK 22.1: CONFIRMING INSTALL-ABILITY 149
ATTACK 23: FINDING MISSING OR WRONG ALARMS 149
ATTACK 24: FINDING BUGS IN HELP FILES 153
Chapter 8 ◾ Smart and/or Mobile Phone Attacks 159
GENERAL NOTES AND ATTACK CONCEPTS APPLICABLE TO MOST
MOBILE–EMBEDDED DEVICES 159
ATTACK 25: FINDING BUGS IN APPS 161
ATTACK 26: TESTING MOBILE AND EMBEDDED GAMES 165
ATTACK 27: ATTACKING APP–CLOUD DEPENDENCIES 170
Chapter 9 ◾ Mobile/Embedded Security 177
THE CURRENT SITUATION 178
REUSING SECURITY ATTACKS 178
ATTACK 28: PENETRATION ATTACK TEST 180
ATTACK 28.1: PENETRATION SUB-ATTACKS:
AUTHENTICATION—PASSWORD ATTACK 186
ATTACK 28.2: SUB-ATTACK FUZZ TEST 188
ATTACK 29: INFORMATION THEFT—STEALING DEVICE DATA 189
ATTACK 29.1: SUB-ATTACK—IDENTITY SOCIAL ENGINEERING 193
ATTACK 30: SPOOFING ATTACKS 194
ATTACK 30.1: LOCATION AND/OR USER PROFILE SPOOF SUB-ATTACK 199
ATTACK 30.2: GPS SPOOF SUB-ATTACK 200
ATTACK 31: ATTACKING VIRUSES ON THE RUN IN FACTORIES OR PLCS 201
Chapter 10 ◾ Generic Attacks 209
ATTACK 32: USING COMBINATORIAL TESTS 209
ATTACK 33: ATTACKING FUNCTIONAL BUGS 215
Chapter 11 ◾ Mobile and Embedded System Labs 221
INTRODUCTION TO LABS 221
TO START 222
TEST FACILITIES 223
viii   ◾   Contents
WHY SHOULD A TESTER CARE? 224
WHAT PROBLEM DOES A TEST LAB SOLVE? 225
STAGED EVOLUTION OF A TEST LAB 227
SIMULATION ENVIRONMENTS 227
PROTOTYPE AND EARLY DEVELOPMENT LABS 228
DEVELOPMENT SUPPORT TEST LABS 228
INTEGRATION LABS 230
PRE-PRODUCT AND PRODUCT RELEASE (FULL TEST LAB) 230
FIELD LABS 230
OTHER PLACES LABS CAN BE REALIZED 232
DEVELOPING LABS: A PROJECT INSIDE OF A PROJECT 233
PLANNING LABS 233
REQUIREMENT CONSIDERATIONS FOR LABS 234
FUNCTIONAL ELEMENTS FOR A DEVELOPER SUPPORT LAB 234
FUNCTIONAL ELEMENTS FOR A SOFTWARE TEST LAB 235
TEST LAB DESIGN FACTORS 236
LAB IMPLEMENTATION 238
LAB CERTIFICATION 238
OPERATIONS AND MAINTENANCE IN THE LAB 239
LAB LESSONS LEARNED 240
AUTOMATION CONCEPTS FOR TEST LABS 241
TOOLING TO SUPPORT LAB WORK 241
TEST DATA SET-UP 243
TEST EXECUTION: FOR DEVELOPER TESTING 244
TEST EXECUTION: GENERAL 245
PRODUCT AND SECURITY ANALYSIS TOOLS 247
TOOLS FOR THE LAB TEST RESULTS RECORDING 247
PERFORMANCE ATTACK TOOLING 248
BASIC AND GENERIC TEST SUPPORT TOOLS 250
AUTOMATION: TEST ORACLES FOR THE LAB USING
MODELING TOOLS 251
SIMULATION, STIMULATION, AND MODELING IN THE LAB TEST BED 253
CONTINUOUS REAL-TIME, CLOSED-LOOP SIMULATIONS TO SUPPORT
LAB TEST ENVIRONMENTS 256
KEYWORD-DRIVEN TEST MODELS AND ENVIRONMENTS 259
Contents   ◾   ix
DATA COLLECTION, ANALYSIS, AND REPORTING 260
POSTTEST DATA ANALYSIS 262
POSTTEST DATA REPORTING 265
WRAP UP: N-VERSION TESTING PROBLEMS IN LABS AND MODELING 267
FINAL THOUGHTS: INDEPENDENCE, BLIND SPOTS,
AND TEST LAB STAFFING 268
Chapter 12 ◾ Some Parting Advice 273
ARE WE THERE YET? 273
WILL YOU GET STARTED TODAY? 273
ADVICE FOR THE “NEVER EVER” TESTER 273
BUG DATABASE, TAXONOMIES, AND LEARNING FROM YOUR HISTORY 274
LESSONS LEARNED AND RETROSPECTIVES 275
IMPLEMENTING SOFTWARE ATTACK PLANNING 275
REGRESSION AND RETEST 277
WHERE DO YOU GO FROM HERE? 278
APPENDIX A: M OBILE AND EMBEDDED ERROR TAXONOMY: A SOFTWARE
ERROR TAXONOMY (FOR TESTERS), 279
APPENDIX B: MOBILE AND EMBEDDED CODING RULES, 289
APPENDIX C: Q UALITY FIRST: “DEFENDING THE SOURCE CODE SO THAT
ATTACKS ARE NOT SO EASY,” 293
APPENDIX D: BASIC TIMING CONCEPTS, 299
APPENDIX E: DETAILED MAPPING OF ATTACKS, 303
APPENDIX F: UI/GUI AND GAME EVALUATION CHECKLIST, 307
APPENDIX G: RISK ANALYSIS, FMEA, AND BRAINSTORMING, 313
REFERENCES 319
GLOSSARY, 323