Introduction to Machine Learning
Association Analysis
Supervised (inductive) learning
Training data includes desired outputs
Classification
Regression/Prediction
Unsupervised learning
Training data does not include desired outputs
Semi-supervised learning
Training data includes a few desired outputs
Reinforcement learning
Rewards from sequence of actions
3. ML in a Nutshell
• 10s of 1000s of machine learning algorithms
• 100s new every year
• Every machine learning algorithm has 3
components:
◦ Representation
◦ Evaluation
◦ Optimization
3
4. Representation
• Sets of rules / Logic programs
• Decision trees
• Instances
• Graphical models (Bayes/Markov nets)
• Neural networks
• Support Vector Machines (SVM)
• Model ensembles
4
7. Types of Learning
• Association Analysis
• Supervised (inductive) learning
• Training data includes desired outputs
• Classification
• Regression/Prediction
• Unsupervised learning
• Training data does not include desired outputs
• Semi-supervised learning
• Training data includes a few desired outputs
• Reinforcement learning
• Rewards from sequence of actions
7
8. Learning Associations
Basket analysis:
P (Y | X ) probability that somebody who buys X also buys Y
where X and Y are products/services.
Example: P ( milk | beer ) = 0.66
8
Market-Basket transactions
TID Items
1 Bread, Milk
2 Bread, Diaper, Beer, Eggs
3 Milk, Diaper, Beer, Coke
4 Bread, Milk, Diaper, Beer
5 Bread, Milk, Diaper, Coke
9. Inductive Learning
Given examples of a function (X, F(X))
Predict function F(X) for new examples X
◦ Discrete F(X): Classification
◦ Continuous F(X): Regression
◦ F(X) = Probability(X): Probability estimation
9
10. Classification
10
Example: Credit
scoring
Differentiating
between low-risk and
high-risk customers
from their income and
savings
Discriminant: IF income > θ1 AND savings > θ2
THEN low-risk ELSE high-risk
Model
14. R Examples
• Support Vector Machines
◦ Supervised learning methods
◦ Used for classification & regression tasks
◦ Generates non-overlapping partitions & usually employs
all attributes
• Decision tree
◦ Random forest
14