Table Of ContentHacking Artificial
Intelligence
Hacking Artificial
Intelligence
A Leader’s Guide from Deepfakes
to Breaking Deep Learning
Davey Gibian
ROWMAN & LITTLEFIELD
Lanham • Boulder • New York • London
Published by Rowman & Littlefield
An imprint of The Rowman & Littlefield Publishing Group, Inc.
4501 Forbes Boulevard, Suite 200, Lanham, Maryland 20706
www.rowman.com
86-90 Paul Street, London EC2A 4NE
Copyright © 2022 by Davey Gibian
All rights reserved. No part of this book may be reproduced in any form or by any
electronic or mechanical means, including information storage and retrieval systems,
without written permission from the publisher, except by a reviewer who may quote
passages in a review.
British Library Cataloguing in Publication Information available
Library of Congress Cataloging-in-Publication Data
Names: Gibian, Davey, 1988– author.
Title: Hacking artificial intelligence : a leader’s guide from deepfakes to breaking deep
learning / Davey Gibian.
Description: Lanham : Rowman & Littlefield, [2022] | Includes bibliographical
references and index.
Identifiers: LCCN 2021062446 (print) | LCCN 2021062447 (ebook) | ISBN
9781538155080 (cloth) | ISBN 9781538155097 (epub)
Subjects: LCSH: Artificial intelligence. | Information storage and retrieval systems—
Risk management. | Information technology—Management. | Data privacy. | Data
protection.
Classification: LCC Q335 .G53 2022 (print) | LCC Q335 (ebook) | DDC 006.3—
dc23/eng20220223
LC record available at https://lccn.loc.gov/2021062446
LC ebook record available at https://lccn.loc.gov/2021062447
The paper used in this publication meets the minimum requirements of American
National Standard for Information Sciences—Permanence of Paper for Printed Library
Materials, ANSI/NISO Z39.48-1992.
To all the amazing women in my life
Contents
Introduction: Hacking Facial Recognition. . . . . . . . . . . . 1
AI Risks
Chapter 1: A Brief Overview of Artificial Intelligence . . . 11
Artifical Intelligence: A History . . . . . . . . . . . . . . . . 11
Can We Start with Some Definitions?. . . . . . . . . . . . . 17
Chapter 2: How AI Is Different from Traditional Software . 19
One of These Things Is Not Like the Other . . . . . . . . . . . 19
Stress Testing AI . . . . . . . . . . . . . . . . . . . . . . . 24
Chapter 3: Data Bias. . . . . . . . . . . . . . . . . . . . . 25
Sexist Machines. . . . . . . . . . . . . . . . . . . . . . . . 25
So What Is AI Bias? . . . . . . . . . . . . . . . . . . . . . 27
Why Is Bias an AI Risk? . . . . . . . . . . . . . . . . . . . 28
What Everyone Gets Wrong About Data . . . . . . . . . . . 31
How to Limit Data Bias . . . . . . . . . . . . . . . . . . . 35
Synthetic Futures . . . . . . . . . . . . . . . . . . . . . . . 36
Counterfactual Fairness. . . . . . . . . . . . . . . . . . . . 37
Combating Bias Is an Ethical Issue . . . . . . . . . . . . . . 38
Chapter 4: Hacking AI Systems . . . . . . . . . . . . . . . 41
Want to See a Tank Disappear? . . . . . . . . . . . . . . . . 41
The AI Kill Chain. . . . . . . . . . . . . . . . . . . . . . . 44
Hacking a Car Through a Stop Sign. . . . . . . . . . . . . . 46
Chapter 5: Evasion Attacks . . . . . . . . . . . . . . . . . 49
Scaring the Sh*t Out of a CISO. . . . . . . . . . . . . . . . 49
What Is an Evasion Attack? . . . . . . . . . . . . . . . . . 52
vii
Contents
The Science of Adversarial Examples. . . . . . . . . . . . . . 54
Types of Evasion Attacks . . . . . . . . . . . . . . . . . . . 56
WhiteBox. . . . . . . . . . . . . . . . . . . . . . . . . 57
Glasses That Hack . . . . . . . . . . . . . . . . . . . . 58
GreyBox Attacks . . . . . . . . . . . . . . . . . . . . . 61
BlackBox Attacks. . . . . . . . . . . . . . . . . . . . . 63
Transfer and Surrogate Attacks. . . . . . . . . . . . . . 64
Chapter 6: Data Poisoning. . . . . . . . . . . . . . . . . . 71
What Is Data Poisoning? . . . . . . . . . . . . . . . . . . . 71
Availability Attacks: AI Learning Gone Wrong . . . . . . . . 73
Tay Becomes Racist . . . . . . . . . . . . . . . . . . . . . . 75
Integrity Attacks: Back Doors in Your AI . . . . . . . . . . . 76
A Spy Slips Past . . . . . . . . . . . . . . . . . . . . . . . 78
Attacker Strengths . . . . . . . . . . . . . . . . . . . . . . 80
Logic Corruption. . . . . . . . . . . . . . . . . . . . . 81
Data Modification . . . . . . . . . . . . . . . . . . . . 81
Data Injection. . . . . . . . . . . . . . . . . . . . . . . 81
Transfer Attacks . . . . . . . . . . . . . . . . . . . . . 82
Defenses Against Data Poisoning Attacks . . . . . . . . . . . 82
Chapter 7: Model Inversion (“Privacy”) Attacks. . . . . . . 85
Stealing Data. . . . . . . . . . . . . . . . . . . . . . . . . 86
Stealing a Model . . . . . . . . . . . . . . . . . . . . . . . 90
Interpreting AI Through Hacking . . . . . . . . . . . . . . . 93
Chapter 8: Obfuscation Attacks . . . . . . . . . . . . . . . 95
Chapter 9: Talking to AI: Model Interpretability . . . . . . 99
Chapter 10: Machine versus Machine . . . . . . . . . . . 109
What Is a Deepfake? . . . . . . . . . . . . . . . . . . . . 111
Chapter 11: Will Someone Hack My AI?. . . . . . . . . 115
Same Threats, New Tech . . . . . . . . . . . . . . . . . . 115
Secure AI Life Cycle. . . . . . . . . . . . . . . . . . . . . 116
Why. . . . . . . . . . . . . . . . . . . . . . . . . . . 119
How. . . . . . . . . . . . . . . . . . . . . . . . . . . 122
Who . . . . . . . . . . . . . . . . . . . . . . . . . . 125
Impact . . . . . . . . . . . . . . . . . . . . . . . . . 125
viii
Contents
Data. . . . . . . . . . . . . . . . . . . . . . . . . . . 126
Defenses . . . . . . . . . . . . . . . . . . . . . . . . 126
Monitoring . . . . . . . . . . . . . . . . . . . . . . . 127
Chapter 12: The Machine Told Us to Do It . . . . . . . . 129
Our Current Tools Are Not Enough. . . . . . . . . . . . . 129
Asking the Right Questions . . . . . . . . . . . . . . . . . 138
Quality, Performance, and Traceability . . . . . . . . . . . 141
Security and Drift . . . . . . . . . . . . . . . . . . . . . 143
Ethics, Legalities, Risk, and Compliance. . . . . . . . . . . 149
Notes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . 159
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
About the Author . . . . . . . . . . . . . . . . . . . . . . 179
ix
