Survey on Evolutionary Computation of Computer Vision.

1. A SURVEY ON
EVOLUTIONARY
COMPUTATION FOR
COMPUTER VISION
AND IMAGE
ANALYSIS.
Past, Present, and
Future Trends
- Nisha Mohan Devadiga

2. INTRODUCTION
• In Computer Vision (CV), one of the most important tasks is image analysis, which
aims to extract and analyze meaningful information from images.
• Evolutionary computation (EC) techniques are evolutionary algorithms, swarm
intelligence and others.
• Many problems in engineering, management, business ,, remote sensing, biology,
finance, manufacturing, and medicine have been successfully solved using EC
techniques.
• EC solves functions/parameter optimization, classification, regression, and
clustering, among other optimization and learning-related problems.

3. ADVANTAGES
• ECs that can be used for a lot of CV and image analysis tasks and are effective
without requiring a lot of domain knowledge.
• By producing a set of non-dominated solutions, EC techniques are well suited for
solving problems with multiple competing goals. Multi-objective optimization
problems are those that are difficult to solve with exact techniques.
• EC methods have been effectively applied to CV and image analysis and hence such
area of research is known as ECV, evolutionary computer vision.

4. TAXONOMY AND SCOPE
EC for Images Analysis
Task Type
Edge
detection
Image
segmentation
Image feature
analysis
Object
detection
Image
classification
Others
Solution
Representation
Fixed length
Variable
length
Method Type
Evolutionary
algorithms
Swarm
Intelligence
No of Objective
Single-
objective
methods
Multi-
objective
methods
Role of the
method
Optimization
in specific
solutions
Learning
solution from
scratch
Application Domain
Facial images
Biomedical
images
Satellite
images
Others
Fig 1. Categories of existing EC methods for image analysis.
Image Source - https://ieeexplore.ieee.org/document/9943992

5. EDGE DETECTION
• Finding or detecting discontinuities in the image's pixel values is the goal of edge
detection.
• To approximate the pixel value gradient, a straightforward solution is to compare the
value of the current pixel to the values of its neighboring pixels from various directions.
The current pixel may become an edge pixel if it significantly changes.
• Utilized in image segmentation and object detection.
However, edge detection is difficult because of the
noise and complex background in images. As can be
seen in figure, EC methods have been utilized
extensively for edge detection.
Image Source - https://ieeexplore.ieee.org/document/9943992

6. EDGE DETECTION - ANT COLONY
OPTIMIZATION (ACO)
• The most popular approach for graph-based edge detection is "Ant Colony
Optimization (ACO)."
• A graph in which each node represents a pixel can be used to represent an image.
Ants can move from one node to another and mark each node by increasing the cell
in a "pheromone" matrix that corresponds to it.
• By imposing a threshold on the ant pheromone, edges can be identified.
• The primary operations of the ACO edge detection techniques are the ant
movement from one node to another and the update of the pheromone matrix.
• Image enhancement, filtering, and edge detection are the three phases of the edge
detection procedure.

7. EDGE DETECTION - ANT COLONY
OPTIMIZATION (ACO)
Benefits:-
• ACO is used to locate the image's edge points.
• During the search, each ant can move to four adjacent directions.
• Additionally, ACO has been utilized to enhance the outcomes of conventional edge
detection techniques.
• Two conventional edge detectors' results can be improved by ACO by repairing
broken edges.
• In terms of improving edge detection performance, it is both efficient and effective.

8. EDGE DETECTION - GENETIC PROGRAMMING
(GP)
• "Genetic Programming" (GP) models that automatically categorize pixels into non-
edge or edge groups or identify edge pixels from images are built.
• W. Fu and co. in order to create natural image-based tree-based edge detectors,
create a new GP method with a new function set and program structure.
• Based on a predetermined threshold, the evolved edge detectors classify a pixel as
an edge or non-edge point. The outcomes demonstrate that the leaf nodes in the
edge detectors are crucial and that the automatically constructed edge detectors
are very impressive.

9. EDGE DETECTION - OTHER EC METHODS
• Other EC methods are used to optimize edge detection process.
• The optimal coefficients of the cloning template in a cellular neural network for
edge detection can be found using Differential Evolution (DE).
• As inputs to the generator of the generative adversarial network (GAN) for edge
detection, an improved DE method is proposed for producing better images.
• The loss function derived from the GAN's discriminator is DE's fitness function. On
two benchmark datasets, this method performs at the cutting edge. However, it
takes a long time and several real images to train the models.
• In conclusion, EC techniques can automatically evolve edge detectors, construct
graphs for edge detection, optimize traditional detector results, and optimize
existing algorithm parameters when used for edge detection.

10. IMAGE SEGMENTATION
• The goal of image segmentation is to split an image into multiple uniform and
uncorrelated regions.
• It is a crucial step in image analysis, and it is frequently required to complete
higher-level tasks like object detection and image classification.
• It is a challenging endeavor that may necessitate extensive domain knowledge, high
computational costs, and complex or large search spaces.
• Due to its powerful global search capability and low demand for domain expertise,
EC has found widespread application in image segmentation.
• Classified into five main categories, including region-based, classification-based,
clustering-based, thresholding-based, and other types. Graph-based, edge-
detection-based and model-based image segmentation techniques are included in
this category.

11. IMAGE SEGMENTATION
Image Source - https://ieeexplore.ieee.org/document/9943992

12. IMAGE SEGMENTATION
A. Threshold-based Image Segmentation Methods
 most common and straightforward methods for segmenting images.
 The basic notion is to determine an image's histogram or other statistical value,
then divide the image by comparing it to the histogram or statistical value and
grouping pixels that are similar.
Multiple threshold values are frequently required to achieve good segmentation
performance for complex images.
 Achieved by - Optimize threshold values & Optimize other parameters

13. IMAGE SEGMENTATION
B. Region-based Image Segmentation Methods
 By preserving the spatial relationships between pixels, region-based image
segmentation techniques typically locate regions with similar pixel values that
comply with pre-determined rules.
Region-based strategies can be broken down into two categories: region-growing
strategies and region-merging strategies.
A bottom-up method known as a region-growing method uses some pixels as
seed pixels and grows the region by checking pixels that are nearby. The
neighboring pixels will be placed in the same region as the seed pixel if they
comply with predetermined rules.
Instead, the region merging technique is a top-down approach that divides an
image into smaller sections and then merges such sections in accordance with
established guidelines.
EC techniques is used to optimize both region-growing and region-merging
methods.

14. IMAGE SEGMENTATION
C. Clustering-based Image Segmentation Methods
 To achieve image segmentation, clustering-based techniques focuses to combine
pixels into distinct clusters with low inter-cluster and high intra-cluster similarity.
 EC strategies have been utilized to track down great beginning bunch centroids for
picture division.
 The number of clusters and the initial cluster centroids are evolved using EC
methods.
 Optimizing clustering-based image segmentation techniques is a popular
application of EC techniques, such as GAs, PSO, DE, and EMO. The number of
clusters, initial cluster centers, or both can be maximized using these strategies. In
these strategies, a genuine worth vector-based portrayal is ordinarily used to
address the arrangements.
***genetic algorithms (GAs);differential evolution (DE) ;particle swarm optimization (PSO);ant colony optimization (ACO) ;genetic programming (GP) ;evolutionary multi-objective optimization (EMO).

15. IMAGE SEGMENTATION
D. Classification-based Image Segmentation Methods
 For image segmentation, classification-based methods typically create models or classifiers
that can associate each pixel in an image with distinct classes.
 The two main types of EC-based image segmentation techniques are those that use EC
(mostly GP) to automatically evolve/construct classifiers/models or those that use EC to
improve existing classification algorithms.
For image segmentation, GP is the primary method that does not rely on any other
segmentation or learning techniques to automatically evolve models or classifiers. superior
to the other four approaches.
Classification-based approaches typically make use of EC-based approaches to optimize
CNN-based models and use GP to automatically evolve models and classifiers.
Most of the time, GP-based methods find tree-based models with functions and terminals or
features that group pixels into different classes and are easy to understand.
The goal of the EC-based CNN approaches is to locate promising CNNs for effective image
segmentation. The limitations of the existing CNN methods are addressed by these
methods.

16. IMAGE FEATURE ANALYSIS
• Numerous tasks involving images rely on image feature analysis. Roasters of raw
pixels, which may contain information that is useful or meaningful, are typically
used to represent images.
• The primary focus of image feature analysis is the examination of images'
meaningful features and information. Typically, it consists of feature selection,
feature extraction, feature construction, or a more general step known as feature
learning.
Image Source - https://ieeexplore.ieee.org/document/9943992

17. IMAGE FEATURE ANALYSIS
• A necessary step in image analysis is feature selection, which selects the most
significant features from a set of features that have already been extracted. When
there are a lot of features and complicated interactions between them, feature
selection becomes difficult.
 GA-based Methods
 PSO-based Methods
 ACO-based Methods
 EMO-based Methods
 Hybrid Methods
 Other Methods

18. IMAGE FEATURE ANALYSIS
• Feature Extraction and Learning - ex interactions between features.
Optimization of Existing Feature Extraction Methods
Automatic Evolution of Models/Descriptors for Feature Extraction

19. IMAGE CLASSIFICATION
In CV and machine learning, image classification is a central and crucial task. Based on the content of
the image, it aims to classify it into one of several predetermined categories.
A. EC for Evolving NNs for Image Classification:
DNNs, or deep CNNs, for image classification have recently been developed using EC methods with
success.
 Evolving NN Weights and/or Architectures:
 Suganuma and co propose the CGP-CNN approach, which uses ResNet blocks or standard
convolutional operators to create a tree-based representation of NAS.
 Low model complexity and high classification performance are achieved by this approach.
 Li introduces a binary encoding to evolve deep CNNs using quantum-behaving PSO.
 Gong et al. suggests to combine backpropagation with a co-evolutionary approach to search
the parameters of DNN models for image classification.

20. IMAGE CLASSIFICATION
When one method is unable to optimize the training objective function, the weights
are learned using co-evolution and backpropagation.
For efficient search, the parameter learning task in co-evolution is broken down into
numerous smaller tasks. The conventional DNN parameter learning methods are
inferior to this one.
 EMO for Evolving NNs:
In federated learning, Zhu and Jin propound an EMO-based strategy for optimizing
NN structures.

21. IMAGE CLASSIFICATION
Multilayer perceptron and CNN models can be optimized using this
approach at lower communication costs.
Under the real-time federated learning framework, Zhu and Jin develop an
EMO-based NAS method with the objectives of minimizing local payload
and maximizing model performance. Using double-sampling, this method
can effectively cut costs associated with computation and communication.
Wang and co. present a PSO-based multi-objective NAS strategy that
simultaneously maximizes two goals, namely the number of floating-point
operations and classification accuracy.

22. IMAGE CLASSIFICATION
 Computation Efficiency in EC for Evolving NN:
Because it requires numerous fitness evaluations, utilizing EC to optimize NNs is
computationally costly. As a result, latest research focuses on developing search
efficiency-enhancing evaluation strategies that are computationally inexpensive.
Zhang et al. propose a node inheritance strategy and a small sample of training data
in an evolutionary NAS method to lower the cost of fitness evaluation computations.
 Lu, et al. propose NSGANetV2, which makes use of surrogates to predict how CNN
models will perform over time. To naturally select one of four surrogate models for
fitness prediction, a method of selection known as adaptive switching is proposed.

23. IMAGE CLASSIFICATION
B. EC for Evolving Non-NN-based Methods:
Evolving classifiers from Image Features:
 For multi-class object classification using domain-independent features, Zhang and Smart
propose a GP method with a fitness function that calculates the overlap of the distributions of
all possible pairs of classes.
 Multiple GP classifiers and the normal probability density function serve as the foundation for
the classification decision. Three datasets show good results for this method.
Evolving Classifiers from Raw Pixels:
 To classify images, the most common approach employs GP to create tree-like models from
raw pixels. In GP, binary classifiers based on raw pixels have been developed using multi-tier
or multi-layer tree structures.
 Al-Sahaf and other present a two-tier GP method with a classification and an aggregation tier:
The image classifier is the first, which focuses on identifying small image regions and
extracting features.

24. IMAGE CLASSIFICATION
 On four datasets, this method achieves high accuracy, and the learned classifiers and models
are easy to understand.
Feature Learning and Emerging Topics:
 With only a few training images, GP-based methods have demonstrated tremendous potential
for image classification.
 Al Sahaf and other propose a compound GP for region detection and a one-shot GP for
developing classifiers directly from images to make feature extraction for binary image
classification easier.
 Only a few training images are needed for this method, which performs better than traditional
methods.
 Bi and co. In order to enhance GP's generalizability for few-shot image classification, we
propose a dual-tree-based GP approach and develop an effective fitness function.
 On nine image datasets, this method outperforms popular few-shot learning techniques in the
3-shot and 5-shot scenarios.

25. OBJECT DETECTION
• In CV and image analysis, object detection is a fundamental function. It's not just
about finding objects, but also where they are in the images.
• Object detection has been made possible by using EC methods with success. Using
EC methods to naturally derive detectors from scratch and using EC methods to
optimize an object detection system are two broad categories of existing methods.
A. Optimizing Object Detection Systems –
 EC methods can be used to improve the features, parameters, and positions of objects
in an object detection system, among other things.

26. OBJECT DETECTION
 An entropic binary PSO method for enhancing an ear detection system's entropy map. Based
on the optimized entropy map, classification is carried out using a threshold. On four face
image datasets, the performance of this method is promising.
 Eye tracking and detection based on PSO. The deformable multiple template matching
algorithm's center point and scaling factor are optimized using PSO in this approach.
 Notable item identification is an exceptional instance of item recognition that just recognizes
the main item in a picture and overlooks other superfluous ones. Singh and co. propose a
PSO approach to obtaining a saliency map for object detection and the best weights for
three features. The PSO search is guided by an evaluation of the background and attention
pixels by a fitness function. Different performance metrics show that this method performs
better than other methods.
 Moghaddam and co propose a GP-based technique to develop undeniable level highlights
from low - level saliency highlights for remarkable item recognition. A GP method is used to
construct features from each of the four subsets of the feature sets, which are then combined
to produce the final saliency map.
 GP-based methods combine the saliency feature maps in non-linear ways, in contrast to
PSO-based methods.

27. OBJECT DETECTION
B. Automatically Evolving Detectors –
Automatically evolving image object detectors have also utilized EC techniques. Based
on GP, the primary methods for evolving detectors.
Zhang investigates three feature sets, creates a novel fitness function, and develops a
two-phase learning strategy to propose a novel GP-based object detection method.
Using a small portion of the training samples, GP-based detectors are developed
and evaluated in the first phase with the goal of increasing classification accuracy.
Using all of the training samples, the evolved detectors are further refined in the
second phase with the goal of increasing detection accuracy.
For object detection, the findings demonstrate that the two-stage GP method is
more reliable and efficient than the single-stage GP method.
Object detection is typically more challenging than these other tasks because it requires
object localization and classification. However, EC techniques have demonstrated a lot of
promise for both automatically evolving object detectors and improving existing object
detection systems.

28. OTHER TASKS
• Other image-related tasks, such as interest point detection, image registration, remote-sensing image
classification, and object tracking, have demonstrated significant potential for EC methods.
 Interest Point Detection –
 The detection of interesting points, such as corners, blobs, and edges, that convey relevant or
interesting visual information is referred to as interest points detection.
 GP is used to develop interest point detectors in images automatically. The majority of the
works are by Gustavo Olague's group. In order to construct detectors from images, Trujillo and
Olague devise a GP method that incorporates arithmetic functions and image operators like
histogram normalization and Gaussian derivatives.
 Point dispersion, stability,and information content are few of the many possible competing
goals in interest point detection.
 For the purpose of interest point detection, a multi-objective GP method with the goals of
stability and point dispersion optimization is investigated.

29. OTHER TASKS
Image Registration –
 In many applications, like medical imaging, image registration is an important preprocessing
step that aims to align two or more images by finding the best transformation of the images
in geometric space.
 Image registration, specifically locating the best transformation, has been made possible by
the development of EC methods.
 In order to simultaneously look for control parameters and image registration solution, a self-
adaptive evolutionary optimization (SaEvO) algorithm is presented.
 because the EC-based applications to image registration have not received a lot of attention
in recent years, even though EC techniques have powerful search capabilities and flexible
representations.

30. OTHER TASKS
Remote Sensing Image Classification –
Typically, the goal of remote sensing or hyperspectral image classification is to
classify individual pixels within images rather than the entire image. In this context,
numerous EC-based methods have also been developed to carry out tasks like
parameter optimization, feature subset or band selection, and locating the best
cluster centers for the clustering-based methods.
The interval type-2 semi-supervised possibilistic fuzzy C-means clustering method
for satellite image classification employs a PSO-based method to optimize the
cluster centers and parameters. Most of the methods compared to this one achieve
better accuracy.
The ACO method employs a binary encoding to select a subset of the 76 original
features, including the Gabor wavelet, the grey-level co-occurrence matrix (GLCM),
and histogram statistics, while simultaneously locating the best SVM parameters.
The performance of this approach is superior to that of other EC-based approaches.
For cropland field classification on very high-resolution images, use GP to develop
classifiers based on spectral, shape, and texture features. On two datasets, this
approach outperforms five conventional classifiers.

31. OTHER TASKS
Object Tracking–
 The goal of object tracking is to locate the positions of objects in a series of images or frames
in order to follow them. Typically, a target object is presented, and the objective is to locate it
across multiple frames.
 Half and half gravitational pursuit calculation (HGSA) to look for the areas of items by
expanding a cosine-closeness capability that assesses the comparability between the objective
items and likely articles in the picture utilizing highlights extricated by CNNs. A cutting-edge
SI-based object tracker becomes more accurate as a result of this approach.
 GP method for developing detectors that can track moving targets against a background that
is stationary. This GP strategy involves pixels in a sliding window as terminals and a few
administrators/capabilities to develop the finders/classifiers. Under various conditions, such as
noise, the proposed method seems to be working.
 an EC-based NAS method for one-time object tracking that uses a pretrained supernet to find
the best trackers. Compared to the earlier methods that did not include NAS search, this one
is more effective and efficient.

32. APPLICATIONS
Facial image analysis Biomedical image analysis
Remote sensing image
analysis
Image analysis related to
humans
Agricultural image analysis
Others. e.g., motor imagery
classification , art classification
, digit recognition , kinship
verification , fish classification,
and vehicle detection

33. CHALLENGES
Scalability Representation Search Mechanisms Interpretability Computational Cost Recognition by the Main
CV Community and
Publishing Papers in
Major CV and AI
Conferences

34. TRENDS
Evolutionary Deep Learning (EDL).
Computationally Efficient ECV Methods.
EMO for CV and Image Analysis.
EC with Transfer Learning for CV and Image
Analysis.
ECV Using Small-Scale Training Data.
EC Methods for Interpretable CV and Image
Analysis.

35. CONCLUSIONS
Even though EC-based methods frequently perform well, they still have to deal with a number of
problems and limitations regarding scalability, representations, search mechanisms, computational cost,
and interpretability.
For CNNs, EC-based approaches may be a good option because they don't require a lot of expertise in
the NN domain, require expensive GPU computing resources, and have poor interpretability.
It is evident that EC-based approaches have demonstrated great potential when applied to a wide range
of image-related tasks.

36. REFERENCES
• A Survey on Evolutionary Computation for Computer Vision and Image Analysis:
Past, Present, and Future Trends; Ying Bi, Bing Xue, Pablo Mesejo, Stefano
Cagnoni, Mengjie Zhang.https://arxiv.org/abs/2209.06399v1
• Learning Adaptive Evolutionary Computation for Solving Multi-Objective
Optimization Problems. Remco Coppens, Robbert Reijnen, Yingqian
Zhang, Laurens Bliek, Berend Steenhuisen, https://arxiv.org/abs/2211.09719

37. THANK YOU