Learning in Adversarial Environments

81 Pages·2012·7.11 MB·English

Learning in Adversarial Environments
Blaine Nelson1 Pavel Laskov1
1Cognitive Systems Group
Wilhelm SchickardInstitute for Computer Science
Universit¨at Tu¨bingen, Germany
Advanced Topics in Machine Learning, 2012
B.NelsonandP.Laskov (Tu¨bingen) AdversarialLearning May22,2012 1/65
Introduction & Motivation
Benefits of machine learning (ML)
Rapid adaptability to changing trends
Scalability to large, diverse data
Statistically sound decision-making
Domains of Interest
Spam / virus / intrusion / fraud detection
Ranking, recommendation systems, performance modeling, advertising
B.NelsonandP.Laskov (Tu¨bingen) AdversarialLearning May22,2012 2/65
What can go wrong?
Known attack applications
Potentially new attack applications
B.NelsonandP.Laskov (Tu¨bingen) AdversarialLearning May22,2012 3/65
Outline
1 Overview of Adversarial Learning
2 Attacks against the SpamBayes Spam Filter
3 Analysis Frameworks for Adversarial Learning
4 Analysis of Shifting a Centroid-Based Anomaly Detector
5 Poisoning Attacks Against Support Vector Machines
6 Additional Material
Single-byte anomaly detection
N-gram anomaly detection
B.NelsonandP.Laskov (Tu¨bingen) AdversarialLearning May22,2012 4/65
Part I
Overview of Adversarial Learning
B.NelsonandP.Laskov (Tu¨bingen) AdversarialLearning May22,2012 5/65
Securing machine learning applications
Can we prove that a machine learning algorithm is secure?
B.NelsonandP.Laskov (Tu¨bingen) AdversarialLearning May22,2012 6/65
Securing machine learning applications
Can we prove that a machine learning algorithm is secure?
Can we prove that some algorithm secure?
B.NelsonandP.Laskov (Tu¨bingen) AdversarialLearning May22,2012 6/65
Securing machine learning applications
Can we prove that a machine learning algorithm is secure?
Can we prove that some algorithm secure?
No! Even strong cryptographic algorithms can be broken by brute force.
B.NelsonandP.Laskov (Tu¨bingen) AdversarialLearning May22,2012 6/65
Securing machine learning applications
Can we prove that a machine learning algorithm is secure?
Can we prove that some algorithm secure?
No! Even strong cryptographic algorithms can be broken by brute force.
Security objective of machine learning:
Show that a specific algorithm under specific conditions can be
broken only with an infeasible amount of work.
B.NelsonandP.Laskov (Tu¨bingen) AdversarialLearning May22,2012 6/65
Security Concerns / Approach
Concern: data non-stationarity during learning
Adversary adapts to evade learner
Learning manipulated by adversary
Security Principles:
Proactive defenses
Avoid security through obscurity
B.NelsonandP.Laskov (Tu¨bingen) AdversarialLearning May22,2012 7/65