Computer Science Evolutionary Computation Genetic Algorithms Memetic Algorithms Complex Networks Complex Systems Self-* Properties

1. 11th International Symposium on Intelligent Distributed Computing
A Performance Analysis of Self-? Evolutionary
Algorithms on Networks with Correlated Failures
Rafael Nogueras Carlos Cotta
Departamento de Lenguajes y Ciencias de la Computación
Universidad de Málaga, Spain
IDC 2017, Belgrade, 11-13 October 2017
Self-? EAs on Complex Networks Universidad de Málaga 1 / 18

2. Introduction Model Description Experimental Analysis Conclusions
Parallel Computing & EAs
Use of parallel and distributed
models of EAs (GAs, MAs,
MMAs, etc.) to improve solution
quality and reduce computational
times.
The island model spatially
organizes populations into
partially isolated panmictic
demes.
island1
island2
island3
island4
migrants
Self-? EAs on Complex Networks Universidad de Málaga 2 / 18

3. Introduction Model Description Experimental Analysis Conclusions
Emergent Paralell Environments
Two emergent computational environments are offering new
opportunities to EAs:
I P2P networks: Equally privileged computing nodes carry out a
distributed computation without need for central coordination.
I Desktop Grids: Distributed networks of heterogeneous systems
which typically contribute computing cycles while they are
inactive (volunteer computing platforms).
Churn
The combined effect of multiple computing nodes leaving and
entering the system along time.
Self-? EAs on Complex Networks Universidad de Málaga 3 / 18

4. Introduction Model Description Experimental Analysis Conclusions
Scope
Analysis of island-based EAs with self-? properties on unstable
networks with correlated failures.
1. Sandpile model to induce the correlated failures.
2. Self-scaling and self-sampling to mitigate the degradation of
performance in a basic island-based EA.
Goal
Compare EAs running on unstable computational environments
with both correlated and non-correlated failures:
I Study by different volatility rates (churn rates).
I Use of self-? properties to increase the resilience of the EA.
I Impact on performance.
Self-? EAs on Complex Networks Universidad de Málaga 4 / 18

5. Introduction Model Description Experimental Analysis Conclusions
Network Topology
Regular lattice with von Neumann connectivity (virtual topology
for migration in the island model) overlaid on a scale-free network
(underlying topology for failure correlation).
Scale-free networks are commonly observed in many natural
phenomena. They feature a power-law distribution in node degrees.
This kind of networks is often the result of processes driven by
preferential attachment ⇒ Barabási-Albert model.
1
2 3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18 19
20
21
22
23
24
25
26
27
28
29
30
31
32
10
0
10
1
10
−2
10
−1
10
0
10
1
α=−1.9346
degree
P(degree)
Cumulative distribution function
Self-? EAs on Complex Networks Universidad de Málaga 5 / 18

6. Introduction Model Description Experimental Analysis Conclusions
Network Topology
Algorithm 1: Barabási-Albert Model
function BA-Model (↓ m, n : N) : Network
m0 ← min(n, m);
net ← CreateClique(m0);
δ[1 . . . m0] ← m0;
for i ← m0 + 1 to n do
net ← AddNode(net);
for j ← 1 to m do
k ← Pick(δ) // Sampling w/o replacement ∝ δ
δ[k] ← δ[k] + 1;
net ← AddLink(net, i, k);
end
δ[i] ← m;
end
return net
Self-? EAs on Complex Networks Universidad de Málaga 6 / 18

7. Introduction Model Description Experimental Analysis Conclusions
Instability: Basic Failure Model
Each node can switch from active to inactive or vice versa
independently of other nodes with some probability p(t) that
depends on the time it has been in its current state.
0 100 200 300 400 500 600 700 800 900 1000
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
time
survival
probability
k=1
k=2
k=5
k=10
k=20
We assume node availability
follows a Weibull distributiona:
p(t1 | t0) = e
−
h
t1
β
η
−
t0
β
ηi
If the shape parameter η 1 the
hazard rate increases with time.
a
J Grid Comput, doi: 10.1007/s10723-014-9315-6, 2015
Self-? EAs on Complex Networks Universidad de Málaga 7 / 18

8. Introduction Model Description Experimental Analysis Conclusions
Instability: Correlated Failure Model
Node failures can be influenced by neighboring nodes:
I Sandpile model variant to induce cascading failures.
I each node has an associated threshold value θi .
I a micro-failure event happens with probability p(t): when the
number of such micro-failures equals θi , the node i is
disconnected from the system and each of its active neighbors
receives an additional micro-failure event.
I if any of these neighbors accumulated a number of
micro-failures equal to its own threshold, it would go down as
well, propagating in turn another micro-failure to its active
neighbors ⇒ cascading failures.
I reactivation doesn’t change.
Self-? EAs on Complex Networks Universidad de Málaga 8 / 18

9. Introduction Model Description Experimental Analysis Conclusions
Instability: Example of Cascading Failure
a
0
b
5
c
0
d
0
e
1
g 0 i
0
f
h
a
1
c
1
d
1
e
2
g 1 i
0
f
h
b
6
c
1
d
1
g 1 i
1
b
f
h
a
1
e
2
c
1
d
1
g 1 i
1
a b
e f
h
Self-? EAs on Complex Networks Universidad de Málaga 9 / 18

10. Introduction Model Description Experimental Analysis Conclusions
Coping with Instability
Computing in an unstable environment requires fault-tolerance.
Classical approaches are redundancy or checkpointing. Note that:
I These strategies require access to external safe storage and
possibly some central monitoring.
I As nodes go up and down, overall population size will
fluctuate.
An alternative strategy is used1:
I No central command required: decision making and
information exchange is done locally among neighboring
islands.
I Qualitative exchange of information among islands.
1
Comput Appl Math, doi:10.1016/j.cam.2015.03.047, 2015.
Self-? EAs on Complex Networks Universidad de Málaga 10 / 18

11. Introduction Model Description Experimental Analysis Conclusions
Self-scaling
The algorithm changes its structure in response to variations in the
problem or environment. Islands dynamically change their size in
the presence of churn:
I if a neighboring island goes down, an island increases its size.
I active islands exchange individuals in order to balance their
sizes (self-balancing).
b
10 16
a
22 16 18 17
c
20 18
d
16 17
← ping
→ ping
→ ping
→ pong
← status?
→ h10, 4i
← push(6)
← pong
→ status?
← h20, 3i
→ request(2)
← push(2)
← pong
→ status?
← h16, 5i
→ push(1)
b a
17 21
← ping
[timeout]
Compensate the loss of islands and balance population sizes2.
2
Comput Appl Math, doi:10.1016/j.cam.2015.03.047, 2015.
Self-? EAs on Complex Networks Universidad de Málaga 11 / 18

12. Introduction Model Description Experimental Analysis Conclusions
Self-sampling
Self-balancing only captures the quantitative aspect of resizing:
I New solutions are randomly constructed from scratch.
I This method introduces diversity but does not keep up the
momentum of the search.
Improvement by using smart strategies:
1. Probabilistic model to estimate the population of each island
to be enlarged.
2. New individuals are generated by sampling from previous
model.
3. Diversity is still introduced since new individuals can be
different.
We use a tree-like bivariate probabilistic model (COMIT).
Self-? EAs on Complex Networks Universidad de Málaga 12 / 18

13. Introduction Model Description Experimental Analysis Conclusions
Benchmark and Settings
Parameters for island-based model:
I nι = 64 islands and µ = 32 individuals (at the beginning).
I m = 2 (BA model).
Node deactivation/reactivation:
I shape parameter η = 1.5.
I scale parameters β = −1/ log(p) for p = 1 − (knι)−1,
k ∈ {1, 2, 5, 10, 20, ∞}.
Problems used:
I Deb’s trap function (concatenating 32 four-bit traps).
I HIFF function (using 128 bits).
I MMDP (using 24 six-bit blocks).
25 runs @ 250,000 evaluations are performed for each problem and
algorithm.
Self-? EAs on Complex Networks Universidad de Málaga 13 / 18

14. Introduction Model Description Experimental Analysis Conclusions
Numerical Results
Approximation to the Optimum
Deviation from the optimum as a function of the churn rate.
(Left) EA and (Right) Self-? EA.
0.2 0.4 0.6 0.8 1
0
10
20
30
40
50
60
70
1/k
deviation
from
optimum
(%)
non−correlated
correlated
0.2 0.4 0.6 0.8 1
0
5
10
15
20
25
30
35
40
45
50
1/k
deviation
from
optimum
(%)
non−correlated
correlated
Performance degrades with increasing churn rates but not in the
same way for the different strategies.
Self-? EAs on Complex Networks Universidad de Málaga 14 / 18

15. Introduction Model Description Experimental Analysis Conclusions
Numerical Results
Evolution of Genetic Diversity: EA
Population entropy is an indicator of algorithmic convergence.
(Left) Non-correlated and (Right) Correlated.
0.5 1 1.5 2 2.5
x 10
5
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
evaluations
entropy
k = ∞
k = 20
k = 10
k = 5
k = 2
k = 1
0.5 1 1.5 2 2.5
x 10
5
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
evaluations
entropy
k = ∞
k = 20
k = 10
k = 5
k = 2
k = 1
EA faces increasingly large difficulties to converge even for
low-volatility settings in the correlated scenario.
Self-? EAs on Complex Networks Universidad de Málaga 15 / 18

16. Introduction Model Description Experimental Analysis Conclusions
Numerical Results
Evolution of Genetic Diversity: Self-? EA
(Left) Non-correlated and (Right) Correlated.
0.5 1 1.5 2 2.5
x 10
5
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
evaluations
entropy
k = ∞
k = 20
k = 10
k = 5
k = 2
k = 1
0.5 1 1.5 2 2.5
x 10
5
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
evaluations
entropy
k = ∞
k = 20
k = 10
k = 5
k = 2
k = 1
Self-? EA can maintain a better focus on the search even in the
worst scenarios.
Self-? EAs on Complex Networks Universidad de Málaga 16 / 18

17. Introduction Model Description Experimental Analysis Conclusions
Conclusions
Correlated failures constitute a major threat to the robustness of
computing networks.
Resilience is a key feature on unstable computational environments.
EA with self-? properties increase its resilience and make it able to
withstand from low up to moderately high volatility.
Future work:
I consider other topologies.
I extend to different models of correlated failures or other
alternative models.
I use in a related problem as the optimization of the network
itself to cope with this kind of failures.
Self-? EAs on Complex Networks Universidad de Málaga 17 / 18

18. Introduction Model Description Experimental Analysis Conclusions
Thank You!
EphemeCH
Project
Please find us in Facebook
http://facebook.com/EphemechProject
and in Twitter
@EphemeCH
Self-? EAs on Complex Networks Universidad de Málaga 18 / 18