Real-world Cyber-Physical Systems (CPSs) are usually configurable. Through parameters, it is possible to configure, select or unselect different system functionalities. While this provides high flexibility, it also becomes a source for failures due to misconfigurations. The large number of parameters these systems have and the long test execution time in this context due to the use of simulation-based testing make the manual repair process a cumbersome activity. Subsequently, in this context, automated repairing methods are paramount. In this paper, we propose an approach to automatically repair CPSs’ misconfigurations. Our approach is evaluated with an industrial CPS case study from the elevation domain. Experiments with a real building and data obtained from operation suggests that our approach outperforms a baseline algorithm as well as the state of the practice (i.e., manual repair carried out by domain experts).

1. Automated Misconfiguration Repair of
Configurable Cyber-Physical Systems
with Search: an Industrial Case Study on
Elevator Dispatching Algorithms
Let’s
discuss at
ISSTA’23
Pablo Valle Aitor Arrieta Maite Arratibel

2. 2
Example of a Cyber-Physical System

3. 3
Configurability and Variability in
Elevators
55% of issues that appeared in
operation were due to
misconfigurations

4. 4
Challenges for Automated Repair of
Misconfigurations
Expensive Test
Execution
Large Configuration
Space
Multiple Requirements
Need to prioritize
severe scenarios

5. 5
Approach Overview
Misconfiguration
Repair Algorithm
Decision
Maker
Patch Validator
</>
Misc
FTC
Failing test
suite
PTC
Passing test
suite
Test
Oracle
</>
Plausible
Patch
</>
Patch
Partial
patch
archive
</>
Partial
Patch
FTC
</>
Misc’

6. 6
Misconfiguration Repair Algorithm

7. 7
Misconfiguration Repair Algorithm
Expensive Test
Execution
Archive-based search strategy
Parallelization of the simulation-
based test execution
Tried to accelerate test execution
through surrogate models, but…
didn’t work!

8. 8
Misconfiguration Repair Algorithm
Large Configuration
Space
Mechanism to measure
parameters’ suspiciousness
Parameters with high
suspiciousness get higher
probability to be mutated

9. 9
Misconfiguration Repair Algorithm
Multiple
Requirements
Each requirement modeled through
one test oracle that provides a
score [-1,0]
Solution stored in Archive based
on Pareto-optimality
Maximun of 2 x # of Reqs. solutions
in the Archive

10. 10
Misconfiguration Repair Algorithm
Need to prioritize
severe scenarios
Consider worst case scenario
per requirement for all test cases
executed

11. 11
• Select a partial patch based on domain-specific rules
• Priority 1: Select all partial patches with WT < 25 seconds
• Priority 2: Select all partial patches which had < 10% of
passengers waiting more than 55 seconds
• …
• After all priorities, if more than one patch exists, select
the patch with minimal changes to the original one
(based on Hamming Distance)
Decision Maker

12. 12
• Passing test cases are executed
• Encompassing full-day traffic profiles
• Regression test oracle employed (comparison with misconfigured
version)
• Other types of test cases executed using metamorphic
testing for CPSs
Patch Validator

13. 13
RQ1 – Sanity check: How does our approach compare to
the baseline?
RQ2 – Comparison with state of the practice: How does
our approach compare to manual repair carried out by
domain experts?
Evaluation – Research Questions

14. 14
• Case study
• Orona’s CGC dispatching algorithm
• Real misconfiguration that appeared in a real installation
• Operational data was available
• Data from manual repair was available
• Baseline algorithm: unguided version of our algorithm
(similar to random search)
• Evaluation metrics
• Hypervolume
• Individual objectives (after the patch is selected by the DM)
• Runs
• 10 executions for randomization
• 12 hours time budget
• Statistical tests
Evaluation – Experimental setup

15. 15
Results
Setup: No threshold provided  Optimize as much as possible
all objectives

16. 16
Results
Misconf Manual Baseline Repair
AWT (sec) 25.99 23.10 22.66 22.77
LWT (sec) 435.70 223.00 241.55 213.72
%WT>55 sec (%) 12.78 11.99 9.93 9.92
ATT (sec) 42.01 41.60 41.77 41.58
LTT (sec) 209.8 220.60 206.24 195.56
%WT>70 sec (%) 10.24 10.02 9.64 9.45
Setup: No threshold provided  Optimize as much as possible
all objectives

17. 17
For out of 14 executions, for our algorithm, half of the
executions repaired the misconfiguration in around 3.5
hours
Results (not in the paper)
Setup: Threshold specified
For out of 14 executions, for the baseline, only one of the
executions repaired the misconfiguration in 11 hours

18. 18
Answer to RQs
Our approach outperforms both
the baseline and the manual repair
provided by the domain experts.

19. 19
• Lesson 1 – Reduction of personnel cost
• Lesson 2 – Scalable technique
• Lesson 3 – Surrogate models did not help
• Lesson 4 – Challenging conflicting installation, with
many unforeseen situations
Conclusion and Lessons Learned
We provide a scalable and automated
approach for automated misconfiguration
repair in the context of configurable CPSs

20. Thank you!
Aitor Arrieta
aarrieta@mondragon.edu
Pablo Valle Aitor Arrieta Maite Arratibel