An Introduction to Machine Learning with Kernels - Alex Smola

241 Pages·2004·4.38 MB·English

by  Alexander J. Smola

An Introduction to Machine Learning
with Kernels
Lecture 1
Alexander J. Smola
[e-mail]
Statistical Machine Learning Program
National ICT Australia, Canberra
AlexanderJ.Smola:AnIntroductiontoMachineLearningwithKernels,Page1
Day 1
Machine learning and probability theory
Introduction to pattern recognition, classification, regres-
sion, novelty detection, probability theory, Bayes rule, in-
ference
Density estimation and Parzen windows
Kernels and density estimation, Silverman’s rule, Wat-
son Nadaraya estimator, crossvalidation
Perceptron and kernels
Hebb’s rule, perceptron algorithm, convergence, feature
maps, kernel trick, examples
Support Vector classification
Geometrical view, dual problem, convex optimization,
kernels and SVM
AlexanderJ.Smola:AnIntroductiontoMachineLearningwithKernels,Page2
Day 2
Text analysis and bioinformatics
Text categorization, biological sequences, kernels on
strings, efficient computation, examples
Optimization
Sequential minimal optimization, convex subproblems,
convergence, SVMLight, SimpleSVM
Regression and novelty detection
SVM regression, regularized least mean squares, adap-
tive margin width, novel observations
Practical tricks
Crossvalidation, ν-trick, median trick, data scaling,
smoothness and kernels
AlexanderJ.Smola:AnIntroductiontoMachineLearningwithKernels,Page3
L1 Introduction to Machine Learning
Data
Texts, images, vectors, graphs
What to do with data
Unsupervised learning (clustering, embedding, etc.)
Classification, sequence annotation
Regression, autoregressive models, time series
Novelty detection
What is not machine learning
Artificial intelligence
Rule based inference
Statistics and probability theory
Probability of an event
Dependence, independence, conditional probability
Bayes rule, Hypothesis testing
AlexanderJ.Smola:AnIntroductiontoMachineLearningwithKernels,Page4
Data
Vectors
Collections of features (e.g. height, weight, blood
pressure, age, . . . )
Can map categorical variables into vectors (useful for
mixed objects)
Matrices
Images, Movies
Remote sensing and satellite data (multispectral)
Strings
Documents
Gene sequences
Structured Objects
XML documents
Graphs
AlexanderJ.Smola:AnIntroductiontoMachineLearningwithKernels,Page5
Optical Character Recognition
AlexanderJ.Smola:AnIntroductiontoMachineLearningwithKernels,Page6
Reuters Database
AlexanderJ.Smola:AnIntroductiontoMachineLearningwithKernels,Page7
Faces
AlexanderJ.Smola:AnIntroductiontoMachineLearningwithKernels,Page8
More Faces
AlexanderJ.Smola:AnIntroductiontoMachineLearningwithKernels,Page9
Microarray Data
AlexanderJ.Smola:AnIntroductiontoMachineLearningwithKernels,Page10