Reference Based NSGA-II

1. Reference Point Based Multi-objective
Optimization Using Evolutionary Algorithms
K. Deb and J. Sundar
GECCO 2006
Reviewed by
Paskorn Champrasert
paskorn@cs.umb.edu
http://dssg.cs.umb.edu

2. Outline
• Introduction
• Reference Based EMO Approaches
• Proposed Reference Point Based EMO Approach
R-NSGA-II: Reference Pointed Based NSGA-II
• Simulation Results
– Two-objective Problem
– Three-objective Problem
– Five-objective Problem
– 10-objective Problem
– Engineering Design Problem (weld-beam problem)
• Conclusions
December 2, 08 DSSG Group Meeting 2/31

3. NSGA-II Problems
• NSGA-II
– elitism Non-dominated sorting genetic algorithms-II
was proposed by K.Deb in 2002
– NSGA-II has problems in solving problems with a
large number of objectives*
• Problems with a large number of objectives, the most of
individuals in the population are non-dominated solutions
-> the individuals (solutions) may not move towards the Pareto-
optimal region.
-> the size of population may be increased to overcome this issue
but
1) this makes the algorithm work very slow
2) how large the population size should be?
* NSGA-II paper in 2002
December 2, 08 DSSG Group Meeting 3/31

4. Observation
The nature of decision makers: DM (--who wants to find the
optimal solutions for their problems)
• In large-objective problem solving, decision makers have some
clues in their mind.
• For example,
– In the problem of maximizing throughput and minimizing latency, DM may
have a clue that throughput should be about 99.9%
• The EMOA (evolutionary multi-objective optimization algorithms)
can provide the Pareto-optimal solutions close to the point that the
throughput is 99.9% instead of the entire frontier.
• The DM can concentrate on only the regions on the Pareto-optimal
frontier which are of interest to her/him.
December 2, 08 DSSG Group Meeting 4/31

5. Research Approach
• In large-objective problem solving, if DM
provide a clue, EMOA can be put to benefit in
finding a preferred and smaller set of Pareto-
optimal solutions instead of the entire frontier.
• The proposed algorithm uses the concept of
reference point methodology
– DM provides a clue as some points on objective
domain that the DM interests
• The proposed algorithm attempts to find a set of
preferred Pareto-optimal solutions near the
reference points called regions of interest to a
decision maker.
December 2, 08 DSSG Group Meeting 5/31

6. Design Principles
1) Multiple preference conditions can be specified
simultaneously
2) For each provided reference point, a set of Pareto-
optimal solutions close to the provided reference
point is the target set of solutions
3) The algorithm can be used for any shape of Pareto
optimal frontier (e.g., convex, non-convex,
continuous, discrete, connected, disconnected.)
4) The algorithm can be used to a large number of
objectives (e.g., 10 or more objectives), a large
number of variables and linear or non-linear
constraints
December 2, 08 DSSG Group Meeting 6/31

7. R-NSGA-II
The proposed reference point-based NSGA-II
• R-NSGA-II provides a set of
Pareto optimal solutions
near a provided set of
reference points. So that, the
DM can have an idea about
the regions that the DM
interests.
• R-NSGA-II is implemented
based on NSGA-II
– DM provides one or more
reference points.
December 2, 08 DSSG Group Meeting 7/31

8. NSGA2: Mainloop Step 1: Tournament
Each individual is compared with another randomly selected individual.
Pt: Selected Parents at generation t
(niche comparison)
Qt: the offspring that are generated The copy of the winner is placed in the mating pool
from Pt
Step 2: Apply crossover rate for each
Rank 1 Individual 1 individual in a mating pool, and select
Rank 2 tournament Individual 2 a parent (s). Two parents perform
crossover Qt crossover and generate two offspring.
Rank 3 Two offspring will be placed in the
Individual N offspring population Qt+1
Rank 4 Pt1
Population
Mating pool
size = N
Step 3: Apply non-dominated sorting to
Rt population. All non-dominated Step 4: Stop adding the individuals in
fronts of Pt+Qt are copied to the parent the rank when the size of parent
population rank by rank. population is larger than the population
size (N)
Rank 1 Individuals in the last accepted rank,
Non- Crowding
Rank 2 that make the parent population size
Pt dominated distance larger than N (in example, rank 4), are
sorting Rank 3 sorting sorted by crowding distance sorting.
Rank 4 Rank 4 rejected
Qt
Rt Pt+1: The parent population
that will generate offspring
to the next generation
December 2, 08 DSSG Group Meeting 8/31

9. Crowding distance Assignment
Individuals are sorted in each objective domain.
The first individual and the last individual in the rank are assigned
the crowding distance = infinity.
For other individuals, the crowding distance is calculated by the
different of the objective value of two closet neighbors.
Example (objective F1)
Crowding Distance ∞ 5-1 8-2 20 - 5 ∞
8 20 F1
Objective value 1 2 5
December 2, 08 DSSG Group Meeting 9/31

10. Niche comparison
Between two solutions with differing non-domination ranks
the solution with the better rank is preferred.
Otherwise if both solutions belong to the same front then
the solution which is located in lesser crowded region (has
lower crowding distance value) is preferred.
December 2, 08 DSSG Group Meeting 10/31

11. R-NSGA-II
The proposed reference point-based NSGA-II
• The crowding distance in NSGA-II is modified
in R-NSGA-II.
• In R-NSGA-II, crowding distance represents
how the solutions are close to the reference
points.
December 2, 08 DSSG Group Meeting 11/31

12. Crowding Distance in R-NSGA-II
1) For each reference point, the normalization
Euclidean distance (dIR) of each solution of the
front is calculated and the solutions are sorted in
ascending order of distance.
q PM fi (x)−Ri
dIR = ( f max −f min )2
i=1 i i
dIR : the normalization Euclidean distance from individual I to reference R
M is the number of objectives
fimax and fimin are the population maximum and minimum objective value of i-th
objective
Normalization is used to avoid the problem that the objectives are in the
different scale. (e.g., one objective value is ~1000 and another objective is
~0.01).
December 2, 08 DSSG Group Meeting 12/31

13. • This way, the solution closest to the reference
point is assigned a best rank for the reference
point.
Min F2
1 3 Rank to reference
point R1
1 3
Rank to reference
R1 R2 point R2
2 2
3 1
4 1
Min F1
December 2, 08 DSSG Group Meeting 13/31

14. Crowding Distance in R-NSGA-II
2) After such computations (distance to reference
points ranking) are performed, the minimum of
the assigned ranks is assigned as the crowding
distance to a solution.
Min F2
1 Rank to reference
point R1
1
Rank to reference
R1 R2 point R2
2
1
1
Min F1
The solutions with a smaller crowding distance are preferred.
This is used in binary tournament. If the two randomly selected solutions are in
the same front, the one that has smaller crowding distance is the winner.
December 2, 08 DSSG Group Meeting 14/31

15. Crowding Distance in R-NSGA-II
3) To control the number of the solutions, all solutions having a sum of normalized difference
in objective values (Dxy) of ε or less between them are grouped.
A randomly picked solution from each group is retained and rest all group members are
assigned a large crowding distance in order to discourage them to remain in the
population. q PM fi (x)−fi (y) 2
Dxy = i=1; ( fi
max −f min )
i
E.g., in one objective problem, ε = 2/8
Objective Value 2 3 9 10
2 0 (3-2)/8 = 1/8 (9-2)/8 = 7/8 (10-2)/8 = 1
3 (3-2)/8 = 1/8 0 (9-3)/8 = 6/8 (10-3)/8=7/8
9 (9-2)/8 = 7/8 (9-3)/8 = 6/8 0 (10-9)/8 =1/8
10 (10-2)/8 = 1 (10-3)/8 = 7/8 (10-9)/8 =1/8 0
2 3 9 10
2/8 2/8
December 2, 08 DSSG Group Meeting 15/31

16. ε ε
December 2, 08 DSSG Group Meeting 16/31

17. R-NSGA-II
• If the decision maker is interested in biasing
some objectives more than others, a suitable
weight vector can be used with each reference
point.
• The solutions with a shortest weighted Euclidean
distance from the reference point can be
emphasized.
qP
M fi (x)−Ri
dIR = i=1 wi ( f max −f min )2
i i
wi : the weight value for i-th objective
December 2, 08 DSSG Group Meeting 17/31

18. Simulation Results
• Two to 10 objectives optimization problems
– ZDT1, ZDT2, ZDT3, DTLZ2
• Weld Beam Problem
– 2 Objectives
• R-NSGA-II
– SBX with nc = 10
– Polynomial mutation nm =20
• Population size = 100
• Max generations = 500 generations
December 2, 08 DSSG Group Meeting 18/31

19. ZDT1
• 30 varaible
• f1 in [0,1]
• f2 = 1 – sqrt(f1)
When ε is large, the
range of obtained
solutions is also
large
December 2, 08 DSSG Group Meeting 19/31

20. • When some of the
reference points are
infeasible, the
obtained solutions are
on the Pareto front and
close to the reference
points.
December 2, 08 DSSG Group Meeting 20/31

21. With three different
weight vectors,
Weight vector =
(0.2,0.8) (more
emphasize on f2)
the obtained
solutions are close
to f2
December 2, 08 DSSG Group Meeting 21/31

22. ZDT2
• 30 varaible
• f1 and f2 in [0,1]
• f2 = 1 – f12
• Non-convex Pareto
front.
The result is similar to
the convex Pareto
front
December 2, 08 DSSG Group Meeting 22/31

23. ZDT3
• 30 variable
• Disconnected set of
Pareto fronts
• Problem:
Point A is not on Pareto
front but it is not
dominated by any
solutions.
A is obtained!
December 2, 08 DSSG Group Meeting 23/31

24. DTLZ2
• 14 variable
• Three objectives
A good distribution
of solutions near
the two reference
points are obtained.
December 2, 08 DSSG Group Meeting 24/31

25. DTLZ2 five objective
• 14 variable problem
• Five objectives
• Two reference points
(0.5,0.5,0.5,0.5,0.5)
(0.2,0.2,0.2,0.2,0.8)
PM
i=1 fi2 = 1
Results the obtained solutions are in
[1.000,1.044]
Solutions are very close to the true
Pareto optimal front
December 2, 08 DSSG Group Meeting 25/31

26. 10-Objective DTLZ2 Problem
• 19 variable
• Reference point
fi = 0.25 for all i
The obtained results
concentrates near fi =
1/sqrt(10) = 0.316
P10
i=1 fi2 = 1
For all obtained solutions.
Results are on true Pareto
front
December 2, 08 DSSG Group Meeting 26/31

27. The welded beam design Problem
December 2, 08 DSSG Group Meeting 27/31

28. • Three reference points
• The result shows that a given
reference point is not an optimal
solution (i.e., 20,0.002) but the
obtained results are on the Pareto
Front.
• Thus, if the DM is interested in
knowing trade-off optimal solutions
in three major areas (min cost,
intermediate cost and deflection, and
min deflection), the proposed
algorithm is able to find solutions
near the given reference points
instead of finding solution on the
entire Pareto-optimal fornt, thereby
allowing the DM to consider only a
few solutions that lie on the regions
of her/his interest.
December 2, 08 DSSG Group Meeting 28/31

29. Conclusions
• The paper proposed R-NSGA-II
– R-NSGA-II applied preference based strategy to
obtain a preferred set solutions near the reference
points.
– R-NSGA-II is designed on NSGA-II by changing
crowding distance calculation.
– R-NSGA-II works well on many-objective
optimization problem.
– R-NSGA-II provide the decision-maker with a set of
solutions near her/his preference so that a better and
more reliable decision can be made.
December 2, 08 DSSG Group Meeting 29/31