‭This session aims to provide participants with a comprehensive understanding of‬ decision-making fundamentals in AI/ML, covering key concepts like reinforcement learning,‬ different representations, and an exploration of current state-of-the-art methodologies.‬

1. Fundamentals of Decision Making
AI and ML Team
Shriram (Head)
Samyuktaa
Sanjai Balajee
Shreyas Sai
Sushmithaa P
Reinforcement Learning, Knowledge
Representations and many more

2. Behavioural Conditioning- Pavlov’s Dog

3. Sutton and Barto’s First Paper

4. What is it?
→ Learning from outcomes of doing something (Reward or Punishment)
→ Exploration Based
→ Learns some kind of a Policy - Association between actions and input, usually delayed.
→ Learning usually in a World which is random.
Reinforcement Learning

5. Markov Decision Process
→ Hear the word Markov: It’s all about the states (present,past, future)
→ Conditional Probability
→ MDP: Math used to model a decision-making setup.
→ Important Terms: State, Action, Reward, Transition Probability and
Policy.
→The ‘Agent’ in an ‘environment’ follows a policy to move from one
‘state’ to another (basically does an ‘action’) with a certain ‘Transition
Probability’ and gets a ‘reward’ for the outcomes. The goal of the agent
is to keep mending it’s ways to maximize the long term reward.

7. Example

8. RL Algorithms or Frameworks
→ Basic - Q learning and SARSA
→ Key Difference in the Policy it chooses
→ Q learning converges faster

9. Monte Carlo Tree Search

10. Resources
To Learn RL:
- Sutton and Barto (the OG)
- Introduction to RL, David Silver (Recommended for an easier understanding)
To try out:
- OpenAI Gym
- W&B
- AWS DeepRacer

11. To begin with let us try defining knowledge and reasoning.
Knowledge - Knowledge is awareness or familiarity gained by
experiences of facts, data, and situations.
Reasoning - A way to infer from existing data.
Knowledge representation in AI

12. What is a knowledge based agent?

13. Knowledge representations allows a KBA or AI agent in
general to answer questions intelligently (really?) and
make deductions about real world facts.
Consider this to be an art of portraying information to a
computer so that the machine can take decisions that are
required.
Now what is Knowledge
representation?

14. 1) Structural
2) Declarative
3) Meta
4) Heuristic
5) Procedural
Types of knowledge to represent

15. 1) Logical
2) Semantic Networks
3) Frames
4) Production rules
Here we’ll focus on Logical and Semantic representations of
knowledge.
Types of Knowledge Representation

16. Semantic networks are a graphical representation of
knowledge or concepts, where nodes represent concepts, and
edges represent relationships between these concepts.
Semantic networks provide to us a structured way to
represent knowledge and also make complex relations between
entities easy to comprehend.
Semantic Representation

18. 1) Knowledge-Graphs
2) Ontologies (Semantic Web)
Some popular network methods

19. Cycle of Knowledge Representation

20. Example: Search Problem- Ask Minimum Cost Path
Inference
Learning
Answer
Model
Data
Question

21. Types of Models
Inference
Learning
Answer
Model
Data
Question
● State based Models (states, actions and
costs)
● Variable based Models (variables and
factors)
● Logic based Models (Formulas and
inference)

22. ● Knowledge based model
● Use rules to draw conclusions
● Used for
● Logical Reasoning
● Theorm proving
● Verification
Propositional Logic and First-Order Logic
Logic Based Model

23. Motivation: Smart Assistants
Tell Information Ask Information
It has to digest the given information and reason deeply based on that
information
siri

24. How do you represent the information?
Natural Language is slippery!
● A penny is better than nothing
● Nothing is better than happiness
● Therefore penny is better than happiness?
We use formal language (Logic)
Example,
First-Order Logic: ∀x. Even(x) → Divides(x,2)
∀x(P(x)→Q(x))

25. What is logic made of?
Ingredient 1: Syntax
● defines a set of valid formulas
● Example: Rain ∧ Wet
Ingredient 2: Semantics
● set of assignments and configuration for a
formula
● The meaning
Ingredient 3: Inference Rules
● defines a set of operations that can be
performed

26. Syntax and Semantics
Syntax: What are the valid expressions in this language?
Semantics: What do these expressions really mean?
Different syntax,same semantics
2+3 ⇔ 3+2
Same syntax,different semantics
3/2 (Python2) ⇎ 3/2 (Python3)

27. Propositional Logic

28. Propositional Logic
Syntax
● Propositional Symbols (Atomic Formulas): A,B,C
● Logical Connectives: ¬ ∧ ∨ → ⇔
So, If f and g are formulas then the following holds good
Negation: ¬f
Conjunction: f ∧ g
Disjunction: f ∨ g
Implication: f → g
Biconditional: f ⇔ g

29. Propositional Logic
Examples of PL
P means “It is hot”
Q means “It is humid”
R means “It is Raining”
(P ∧ Q) -> R
“If it is hot and humid, then it is raining”
Q -> P
“If it is humid, then it is hot”

30. Propositional Logic
Logical Connectives

31. Model
A Model in propositional logic is assignment of truth
values to the symbols
Consider 3 Propositional Symbols say, P,Q,R
There are 8 possible models w:
P: 0, Q: 0, R: 0
P: 0, Q: 0, R: 1
P: 0, Q: 1, R: 0
P: 0, Q: 1, R: 1
P: 1, Q: 0, R: 0
P: 1, Q: 0, R: 1
P: 1, Q: 1, R: 0
P: 1, Q: 1, R: 1
Propositional Logic

32. Interpretation function
Let f be a formula
Let w be the model
An interpretation function I(f, w) returns,
● 1 (true) if w satisfies f
● 0 (false) if w does not satisfy f
Propositional Logic

33. Example
Propositional Logic

34. Inference Rules
Example of making a inference
● The lights are off. (LightsOFF)
● If the lights are off, then the room is dark (LightsOFF-> Dark)
Therefore, It is dark. (Dark)
LightsOFF, LightsOFF -> Dark (Premises)
Dark (Conclusion)
Propositional Logic

35. Inference Rules
Modus Ponens
For any propositional symbol P and Q
P, P -> Q (Premise)
Q (Conclusion)
Propositional Logic

36. Inference Algorithm
Input: set of rules
repeat until no changes to KB:
choose f1,f2,f3………fk from KB
if matching rule f1,f2,f3………fk exists
g
add g to KB
Propositional Logic

37. Example inference using Modus Ponens
Starting point:
KB= {Rain, Rain -> Wet, Wet -> Slippery}
Applying Modus Ponens to Rain and Rain -> Wet
KB= {Rain, Rain -> Wet, Wet -> Slippery, Wet}
Applying Modus Ponens to Wet and Wet -> Wet -> Slippery
KB= {Rain, Rain -> Wet, Wet -> Slippery, Wet, Slippery}
Propositional Logic

38. some more rules of inference
Propositional Logic

39. All Students know arithmetic
SanjjitIsAStudent -> SanjjitKnowsArithmetic
SanjaiIsAStudent -> SanjaiKnowsArithmetic
RamIsAStudent -> RamKnowsArithmetic
and so on
SO CLUNKY!!!!
Propositional Logic

40. A WEAK LANGUAGE!!
● Cant talk about the properties of the individuals or
relations between individuals (Example, “Bill is tall”)
● Generalisations cant be easily represented (Example, “All
triangles have 3 sides”)
Solution???
First Order Logic
● FOL adds relations, variables and quantifiers
Example: “All elephants are gray”
∀x (Elephant(x) → Gray(x))
Propositional Logic

41. First Order Logic models the world in terms of
● Objects, which are things with individual identities
○ Example: Students, cars, companies
● Properties of things which distinguish them from other objects
○ Example: blue, square, even
● Relations between objects
○ Example: has-shape, has-color
● Functions, which is a subset of relations where there is only
one “value” for any given “input”
○ Example: fatherOf, BrotherOf
First Order Logic

42. Constant Symbols, which represent individuals in the real world
● Sanjjit
● 3
● Yellow
Function Symbols, which map individuals to individuals
● friendOf(Sanjjit)=SaiRahul
● colorOf(sky)=blue
Predicate Symbols, which map individuals to truth values
● even(4)
● greater(5,4)
How do we denote FOL?

43. Variable Symbols
Example: x,y
Connectives
Same as in PL, ¬ ∧ ∨ → ⇔
Quantifiers
Universal ∀x and Existential ∃x
First Order Logic

44. Quantifiers
Universal Quantification
(∀x)P(x) means that P holds for all values of x in the domain
associated with that variable
Example: (∀x) dolphin(x) → mammal(x)
Existential quantification
(∃ x)P(x) means that P holds for some value of x in the domain
associated with that variable
Example: (∃ x) mammal(x) ∧ lays-eggs(x)
First Order Logic

45. Examples:
Every gardener likes the sun.
∀x gardener(x) → likes(x,Sun)
Alice and Bob both know arithmetic.
Knows(alice, arithmetic) ∧ Knows(bob, arithmetic)
First Order Logic

46. More Examples:
All students know arithmetic.
∀x Student(x) → Knows(x, arithmetic)
Some student knows arithmetic.
∃x Student(x)∧Knows(x, arithmetic)
There is some course that every student has taken.
∃y Course(y) ∧ [∀x Student(x) → Takes(x, y)]
First Order Logic

47. Thank you!!!

48. Feedback