We use association rule methods to find the relationships between the data attributes. We will see 2 methods: apriori and eclat. We will also use 2 libraries : ML-extend and fim library. Association rule learning can be for example used to identify the items that are more often bought together. [notebook](https://colab.research.google.com/drive/1gABWBOK176R0q9HRYrDVpYREWKPKfbLK)

1. Unsupervised Learning:
Association Rule Learning
AAA-Python Edition

2. Plan
●
1- Association Rules Learning
●
2- Apriori
●
3- Apriori illustration
●
4- Apriori Application
●
5- Eclat
●
6- fim librraryr

3. 3
1-Association
RuleLearning
[Byr Amina Delali]
Concept and terminologyrConcept and terminologyr
●
The learning concerns identifyring associations bretween data
attributes.
●
This association is expressed bryr : associations rules in the
form:
➔
If X then Y Or in the form: X ==> Y
➢
Where X and Y are subrsets of attribrutes. These attribrutes are
generallyr called: items. And the subrsets of attribrutes are called:
itemsets.
➢
The data samples describred bryr these attribrutes are called
transactions.
●
These rules are obrtained bryr identifyring the frequent itemsets.
●
The rules permit to predict the presence of items knowing the
existence of other ones.

4. 4
1-Association
RuleLearning
[Byr Amina Delali]
MeasuresMeasures
●
The following measures are used in the association rules
identification process: (where X and Y are itemsets from the
transactions set T, and form the rule X ==> Y)
●
Support_count (X):
●
Support (X):
●
Support (X,Y) =
●
Confidence( X==YY):
●
Lift :
=
frequency of X in T
size of T
=
frequency of X an Y together in T
frequency of X
=frequency of X in T =Number of occurences of X in T
=
frequency of X and Y together in T
size of T
=Support (X ==>Y )
=
Support(X,Y)
Support (X)×Support(Y )

5. 5
1-Association
RuleLearning
[Byr Amina Delali]
Frequent itemsets and association rulesFrequent itemsets and association rules
●
An itemset A is frequent if :
➢
Support(A) >= minimum support threshold
●
An association rule (X ==> Y) is generated as follow:
➢
Select All itemsets that are frequents
➢
Split each frequent itemset in all possibrle subrsets:X and Y
that satisfyr the condition:
➔
Confidence (X ==Y) >= minimum confidence threshold
●
In association rules learning, we applyr specific algorithms on
the transactions dataset (the training data) to identifyr the
frequent itemsets in order to generate the association rules.
●
Some of these algorithms:
➢
Apriori ( Breath First Search )
➢
FP-growth (Frequent Pattern Growth)
➢
Eclat ( Depth First Search )

6. 6
2-Apriori
[Byr Amina Delali]
Naive Apriori: frequent itemsetsNaive Apriori: frequent itemsets
●
There is the “naive” approach, which describre the original apriori
algorithm. Some improvements were introduced to this algorithm,
which lead to diferent versions. We are going to describre the steps
of the naive algorithm describred in [Agrawal et al., 1994]
●
The steps of frequent itemsets generation:
➢ Define the L1
={frequent 1-itemset} (k-itemset = itemset with k
items).
➢
For (k=2, , k++)
➔ Ck
= from Lk-1
generate-all candidates k-itemsets (using onlyr
frequent itemsets)
➔
For all transactions (T is the set of all transactions)
● Increment the count of all itemsets in Ck
and contained in t
➔ Lk
= frequent itemsets in Ck
( itemsets with count >= min
support threshold)
➢
The final frequent itemsets
Lk−1≠∅
t∈T
is ∪k Lk

7. 7
2-Apriori
[Byr Amina Delali]
Naive apriori: rules generationNaive apriori: rules generation
●
The steps of rules generation are:
➢ For all frequent itemsets lk
, k>=2
➔
Generate all valid rules for each
a valid rule is the one that have confidence Y= min
confidence threshold
●
To generate all valid rules for each
➢ 1- Set am
= lk
➢ 2- A = all am-1
itemsets that are subrsets of am
➢
3- For each
➔ Compute confidence of the rule r= (am-1
==> lk
- am-1
) ==
support (lk
) /support(am-1
)
➔
If (confidence (r) Y= min confidence threshold) then
●
select r as a valid rule
● If (m-1 >1) set am
= am-1
and go to 2.
¯a→(lk−¯a) ¯a⊂lk
¯a→(lk−¯a) ¯a⊂lk¯a⊂lk
am−1∈ A

8. 8
2-Apriori
[Byr Amina Delali]
RemarksRemarks
● When we generate the Ck
candidates, we eliminate all the ones
created bryr subrsets that are not frequent. We call it the prune
step.
●
As an improvement, we can consider onlyr transactions that
contain frequent itemsets
●
The min support and min confidence thresholds must bre chosen
wiselyr:
➢
A small threshold will lead to more iterations of the algorithm
➢
A high threshold can eliminate rare items.

9. 9
3-Aprioriillustration
[Byr Amina Delali]
The dataThe data
●
We will run the algorithm on the example cited in
[Gollapudi, 2016] (after correction)
●
We suppose we have the dataset T of transactions that represent
the items brought together in each purchase.
T=
1 A, B,E
2 B, D
3 B, C
4 A, B , D
5 A, D
6 B, C
7 A, D
8 A, B, C, E
9 A, B,C
●
Where each letter represents an item:
●
A = iPad
●
B = iPad case
●
C = iPad scratch guard
●
D = Apple care
●
E = iPhone
●
The numbrers in left column represent
the TID: transaction identifier

10. 10
3-Apriori:illustration
[Byr Amina Delali]
Frequent itemsetsFrequent itemsets
●
We suppose that the minimum support count =2,(min shupport
threshold (2/9 )
Itemset Support
count >= 2
A 6
B 7
C 4
D 4
E 2
1 A, B,E
2 B, D
3 B, C
4 A, B , D
5 A, D
6 B, C
7 A, D
8 A, B, C, E
9 A, B,C
Itemset Support
count
A,B 4
A,C 2
A,D 3
A,E 2
B,C 4
B,D 2
B,E 2
C,D 0
C,E 1
D,E 0
T
C2
L1
Itemset Support
count >= 2
A,B 4
A,C 2
A,D 3
A,E 2
B,C 4
B,D 2
B,E 2
L2

11. 11
3-Apriori:illustration
[Byr Amina Delali]
Frequent items sets: prune stepsFrequent items sets: prune steps
● L2
C3 Itemset Subsets
A,B,C AB, AC, BD
A,B,D AB, AD, BD
A,B,E AB, AE, BE
A,C,D AC, AD, CD
A,C,E AC, AE, CE
A,D,E AD, AE, DE
B,C,D BC, BD, BE
B,C,E BC, BE, CE
B,D,E BD, BE, DE
Itemset SC>=2
A,B 4
A,C 2
A,D 3
A,E 2
B,C 4
B,D 2
B,E 2
C3
Prune
Itemset Support
count
A,B,C 2
A,B,D 1
A,B,E 2
B,C,D 0
Not found in L2
L3
Itemset SC>=2
A,B,C 2
A,B,E 2
We stop here, because if we
want to prune C4,
we will
eliminate the generated
subsets (ABCD, ABCE) since
they contain subsets of 3 items
that are not in L3
C4
Itemset SC
L4
Itemset SC

12. 12
4-AprioriApplication
[Byr Amina Delali]
Ml-xtend The dataMl-xtend The data
●
mlxtend librraryr implments the apriori algorithm.
●
But, brefore applyring the algorithm on the previous example, we
have to create the corresponding data.

13. 13
4-AprioriApplication
[Byr Amina Delali]
Ml-extend frequent itemsetsMl-extend frequent itemsets
●
The result is the union of all previous Li
we
found ( )L1∪L2∪L3

14. 14
4-AprioriApplication
[Byr Amina Delali]
Mlxtend: generated rulesMlxtend: generated rules
●
we will generate the association rules corresponding to the
found frequent itemsets
Min
confidence
threshold
Correspond to
the rule: E→B , A

15. 15
5-Eclat
[Byr Amina Delali]
ConceptConcept
●
It it is brased on the relationship bretween subrset inclusion and
support values.
●
In this algorithm, an item is represented bryr the list of transactions
it brelongs to. Theyr are the TidLists
●
The support is computed from the intersection of these TidLists.
●
If N is the size of T (the transactions dataset), then:
●
It also relies on the concept that:
➢
If
➢
If
support(X ,Y )=
|Tid(X)∩Tid(Y )|
N
=
frequency of X and Y together
N
X⊆Y ,and support (Y )=S⇒ support (X)≥s
Y ⊆X ,and support (X)≤min_s⇒ support (Y )<min_s

16. 16
5-Eclat
[Byr Amina Delali]
AlgorithmAlgorithm
●
The algorithm is defined bryr a recursive function eclat
●
A recursive function is a function that calls itself directlyr or
indirectlyr. It stops when a certain condition is met.
●
The steps are:
➢
set p= {}, Items ={all items}
➢
Call Eclat(P,Items)
●
Eclat (P,I) definition:
➢ F = {}, Cit
={}
➢
If Items ={} return F
➢
else
➔
C = for each i in Items and not in P generate (P U i, i ) tuples
➔
Filter out C so it will contain onlyr frequent P U i itemsets
➔ For each (P U i, i) in c add i to Cit
➔
For each (X, i) in C:
● Cit
= Cit
- {i}
● F = F U X U Eclat(X, Cit
)
➔
Return F

17. 17
5-Eclat
[Byr Amina Delali]
IllustrationIllustration
●
We will run the algorithm on the example cited in [Eclat] ( in the
reference it is actuallyr run for threshold=2 and not 3):
●
I = {a,c,br,e,d,f}, min_support_count_threshold= 3
●
T = [[a,br,c],[a,c,d,e,f], [a,br,c],[d,e]]
●
P ={} , Items= {a,br,c,d,e}
●
Eclat(P= {}, Items={a,br,c,d,e}) --------------(1)
➢
Eclat(P= {}, Items={a,br,c,d,e}) (from 1)
➔ F= {}; Cit
={}
➔
C= {(a,a),(br,br),(c,c),(d,d),(e,e),(f,f)}
➔ C = {(a,a),(c,c)}, Cit
={a,c}
●
1) X=a, i= a
● Cit
={c}
●
F = {a} U Eclat(P={a}, Items={c}) ------- (11)
●
Eclat(P={a}, Items= {c}) (from 11)
● F= {}, Cit
={}
●
C = {(ac,c)}

18. 18
5-Eclat
[Byr Amina Delali]
Illustration (suite)Illustration (suite)
● C = {(ac,c)}, Cit
= {c}
●
X={ac}, i= c
●
Items={}
●
F = {ac } U Eclat (P={ac}, Items={})---- (111)
●
Eclat (P={ac}, Items={}) (from 111)
- F= {}, Cit
={}
- Items =={}, return F (return to 111)
●
F = {ac}
●
Return F (return to ---- (11)
●
F ={a, ac}
●
2) X={c}, i = c
● Cit
={a}
●
F = {a,ac,c} U Eclat(P={c}, Items={a}) ------- (12)
●
Eclat(P={c}, Items= {a}) (from 1)

19. 19
5-Eclat
[Byr Amina Delali]
Illustration (suite)Illustration (suite)
●
Eclat(P={c}, Items= a}) (from 12)
● F= {},Cit=
{}
●
C = {(ca,a)}
● Cit
= {a}
●
X= {ca} ,i=a
● Cit
= {}
●
F = {ca } U Eclat (P={ca}, Items={})---- (121)
●
Eclat (P={ca}, Items={}) (from 121)
- F= {}, Cit
={}
- Items =={}, return F (return to 121)
●
F = {ca}
●
Return F (return to ---- (12))
●
F ={a,ac,c}
➔
Return F (return to (1))
●
Final F = {a, ac, c}

20. 20
6-fimlibrary
[Byr Amina Delali]
Librraryr and dataLibrraryr and data
●
fim is a librraryr comprised of a module that implements a set of
functions dedicated to frequent itemset mining.
●
The functions are related to the algorithm theyr implement.
●
For example, the librraryr implements “Eclat”, “apriori”, et
“fpgrowth” functions.
●
To have a list of all the the represented algorithms, take a look at
its homepage (PyrFIM - Frequent Item Set Mining for Pyrthon).
●
To install the librraryr, just use the pip command:
●
Concerning the data, we do not need to do anyr transformation. So,
we will use the transactions of the example, as we defined them
(list of lists):

21. 21
6-fimlibrary
[Byr Amina Delali]
Eclat applicationEclat application
●
●
Supp =75
means
Support =
0.75 (¾)
●
By default it
prints
support_coun
t values
“S” to print the support as
fractions
The same frequent itemsets we found
earlier in the illustration

22. 22
6-fimlibrary
[Byr Amina Delali]
Apriori applicationApriori application
●
We will use the “apriori” function on the previous example we saw
in apriori section.
Support_cou
nt = 2 is
equivalent to
support = 2/9

23. References
●
[Agrawal et al., 1994] Agrawal, R., Srikant, R., et al. (1994). Fast
algorithms for mining association rules. In Proc. 20th int. conf. veryr
large data brases, VLDB, volume 1215, pages 487–499.
●
[Alexeyr, 2014] Alexeyr, G. (2014). Eclat. On-line at http://mlwiki.org/
index.php/Eclat. Accessed on 30-12-2018.
●
[Gollapudi, 2016] Gollapudi, S. (2016). Practical Machine Learning.
Communityr experience distilled. Packt Pubrlishing.
●
[Sebrastian, ] Sebrastian, R. mlxtend’s documentation. On-line at
http://rasbrt.githubr.io/mlxtend/. Accessed on 30-12-2018.

24. Thank
you!
FOR ALL YOUR TIME