formal languages time complexity

1. Formal Languages
Time Complexity
Hinrich Schütze
IMS, Uni Stuttgart, WS 2006/07
Slides based on RPI CSCI 2400
Thanks to Costas Busch

2. M
L
Consider a deterministic Turing Machine
which decides a language

3. For any string the computation of
terminates in a finite amount of transitions
w M

Accept
or Reject w
Initial
state

4. 
Accept
or Reject w
Decision Time = #transitions
Initial
state

5. Consider now all strings of length n
)
(n
TM
= maximum time required to decide
any string of length n

6. 
)
(n
TM
Max time to accept a string of length n
1 2 3 4 n 
STRING LENGTH
TIME

7. Time Complexity Class: ))
(
( n
T
TIME
All Languages decidable by a
deterministic Turing Machine
in time
1
L 2
L
3
L
))
(
( n
T
O

8. Example: }
0
:
{
1 
 n
b
a
L n

9. Example: }
0
:
{
1 
 n
b
a
L n
This can be decided in time
)
(n
O
)
(n
TIME
}
0
:
{
1 
 n
b
a
L n

10. )
(n
TIME
}
0
:
{
1 
 n
b
a
L n
}
0
,
:
{ 
k
n
aba
abn
}
even
is
:
{ n
bn
Other example problems in the same class
}
3
:
{ k
n
bn


11. )
( 2
n
TIME
}
0
:
{ 
n
b
a n
n
Examples in class:
}}
,
{
:
{ b
a
w
ww R

}}
,
{
:
{ b
a
w
ww 

12. )
( 3
n
TIME
Examples in class:
}
grammar
free
-
context
by
generated
is
:
,
{
2
G
w
w
G
L 
}
and
matrices
:
,
,
{
3
2
1
3
2
1
3
M
M
M
n
n
M
M
M
L




CYK algorithm
Matrix multiplication

13. Polynomial time algorithms: )
( k
n
TIME
Represents tractable algorithms:
for small we can decide
the result fast
k
constant 0

k

14. )
( k
n
TIME
)
( 1

k
n
TIME
)
(
)
( 1 k
k
n
TIME
n
TIME 

It can be shown:

15. 
0
)
(


k
k
n
TIME
P
The Time Complexity Class P
•“tractable” problems
•polynomial time algorithms
Represents:

16. P
CYK-algorithm
}
{ n
n
b
a }
{ww
Class
Matrix multiplication
}
{ b
an

17. Exponential time algorithms: )
2
(
k
n
TIME
Represent intractable algorithms:
Some problem instances
may take centuries to solve

18. Example: the Hamiltonian Path Problem
Question: is there a Hamiltonian path
from s to t?
s t

19. s t
YES!

20. Time?

21. )
2
(
)
!
(
k
n
TIME
n
TIME
L 

Exponential time
Intractable problem
A solution: search exhaustively all paths
L = {<G,s,t>: there is a Hamiltonian path
in G from s to t}

22. The clique problem
Does there exist a clique of size 5?

23. The clique problem
Does there exist a clique of size 5?

24. Example: The Satisfiability Problem
Boolean expressions in
Conjunctive Normal Form:
k
t
t
t
t 


 
3
2
1
p
i x
x
x
x
t 



 
3
2
1
Variables
Question: is the expression satisfiable?
clauses

25. )
(
)
( 3
1
2
1 x
x
x
x 


Satisfiable?
Example:

26. )
(
)
( 3
1
2
1 x
x
x
x 


Satisfiable: 1
,
1
,
0 3
2
1 

 x
x
x
1
)
(
)
( 3
1
2
1 


 x
x
x
x
Example:

27. 2
1
2
1 )
( x
x
x
x 


Not satisfiable
Example:

28. e}
satisfiabl
is
expression
:
{ w
w
L 
)
2
(
k
n
TIME
L 
Algorithm?
exponential

29. e}
satisfiabl
is
expression
:
{ w
w
L 
)
2
(
k
n
TIME
L 
Algorithm:
search exhaustively all the possible
binary values of the variables
exponential

30. Non-Determinism
Language class: ))
(
( n
T
NTIME
A Non-Deterministic Turing Machine
decides each string of length
in time
1
L
2
L
3
L
))
(
( n
T
O
n

31. Non-Deterministic Polynomial time algorithms:
)
( k
n
NTIME
L

32. 
0
)
(


k
k
n
NTIME
NP
The class NP
Non-Deterministic Polynomial time

33. Example: The satisfiability problem
Non-Deterministic algorithm?
e}
satisfiabl
is
expression
:
{ w
w
L 

34. Example: The satisfiability problem
Non-Deterministic algorithm:
•Guess an assignment of the variables
e}
satisfiabl
is
expression
:
{ w
w
L 
•Check if this is a satisfying assignment

35. Time for variables:
n
)
(n
O
e}
satisfiabl
is
expression
:
{ w
w
L 
Total time:
•Guess an assignment of the variables
•Check if this is a satisfying assignment )
(n
O
)
(n
O

36. e}
satisfiabl
is
expression
:
{ w
w
L 
NP
L
The satisfiability problem is an - Problem
NP

37. Observation:
NP
P 
Deterministic
Polynomial
Non-Deterministic
Polynomial

38. Open Problem: ?
NP
P 
WE DO NOT KNOW THE ANSWER

39. Example: Does the Satisfiability problem
have a polynomial time
deterministic algorithm?
WE DO NOT KNOW THE ANSWER
Open Problem: ?
NP
P 

40. NP-Completeness
A problem is NP-complete if:
•It is in NP
•Every NP problem is reduced to it
(in polynomial time)

41. Observation:
If we can solve any NP-complete problem
in Deterministic Polynomial Time (P time)
then we know:
NP
P 

42. Observation:
If we prove that
we cannot solve an NP-complete problem
in Deterministic Polynomial Time (P time)
then we know:
NP
P 

43. Cook’s Theorem:
The satisfiability problem is NP-complete
Proof:
Convert a Non-Deterministic Turing Machine
to a Boolean expression
in conjunctive normal form

44. Other NP-Complete Problems:
•The Traveling Salesperson Problem
•Vertex cover
•Hamiltonian Path
All the above are reduced
to the satisfiability problem

45. Observations:
It is unlikely that NP-complete
problems are in P
The NP-complete problems have
exponential time algorithms
Approximations of these problems
are in P