Practical trustworthiness for AI

1. AI Sensors and Dashboards:
Gauging and Monitoring the
Inference Capabilities of AI
Huber Flores
Associate Professor, University of Tartu
E-mail: huber.flores@ut.ee
Tartu, Estonia, Data Science Seminar – March 11, 2024
This research is part of SPATIAL project that has received funding from the European Union's
Horizon 2020 research and innovation programme under grant agreement No.101021808.

2. ● Not only refinement of algorithms
● Better tools to construct machine learning models
(TensorFlow, FedN, Flower)
● Increased processing capabilities
(Distributed systems) – Geoffrey Hinton
Machine and deep learning (aka AI)

3. ● Not only refinement of algorithms
● Better tools to construct machine learning models
(TensorFlow, FedN, Flower)
● Increased processing capabilities
(Distributed systems) – Geoffrey Hinton
● Classic verification methods are not
applicable for AI models
Machine and deep learning (aka AI)
[source] https://futureoflife.org/open-letter/pause-giant-ai-experiments/,
Accessed March 2024
One year ago!

4. ● Trustworthy computing defines a set of trustworthy
properties required for computing programs to be
considered trustful.
○ Secure
○ Reliability
○ Available
○ Private
○ Resilience
○ Transparency…
Trustworty computing
[source] https://www.heise.de/hintergrund/Microsofts-
Kampf-gegen-Bugs-1429333.html , Accessed March 2024
Bill gates internal memo

5. ● Trustworthy computing defines a set of trustworthy
properties required for computing programs to be
considered trustful.
○ Secure
○ Reliability
○ Available
○ Private
○ Resilience
○ Transparency…
● Trustworthy AI extended properties
○ Free of biases
○ Fair
○ Explainable
○ Interpretable…
Trustworty computing
EU AI ACT
US AI ACT
Chine generative
AI regulations
[source] https://www.heise.de/hintergrund/Microsofts-
Kampf-gegen-Bugs-1429333.html , Accessed March 2024
Bill gates internal memo

6. ● Fundamental mechanisms for data collection and
measurements
● Gauging = quantifying/characterizing
● Monitoring = Tuning
Sensors

7. (A few) Techonologies that have
changed the world
Fire
[source] https://science4fun.info/fire/,
Accessed March 2024

8. (A few) Techonologies that have
changed the world
Fire Electricity
[source] https://science4fun.info/fire/,
Accessed March 2024
[source]
https://www.pinterest.fr/pin/428827195765
555273/ Accessed March 2024

9. (A few) Techonologies that have
changed the world
Fire Electricity Trains
[source] https://science4fun.info/fire/,
Accessed March 2024
[source]
https://www.pinterest.fr/pin/428827195765
555273/ Accessed March 2024
[source] https://www.pinterest.com/pin/ancient-
antique-historic-vehicle-iron-locomotive-metal-
monument-railroad-railway-steam-locomotive-
steel--603060206326098813/
Accessed March 2024
UbiComp 2023 -
From Victorian trains to chatbots:
exploring AI interface design

10. Gauging through measurements
[source]
https://victorianweb.org/technology/rail
ways/44.html

11. Gauging through measurements
[source]
https://www.telegraph.co.uk/news/202
2/04/11/modern-trains-road-signs-
private-level-crossings-safety-move/
[source]
https://victorianweb.org/technology/rail
ways/44.html

12. Gauging through measurements
[source]
https://collection.sciencemuseumgroup
.org.uk/objects/co206028/north-
eastern-railway-dynamometer-car-
railway-carriage
[source]
https://victorianweb.org/technology/rail
ways/44.html

13. Gauging through measurements
[source]
https://collection.sciencemuseumgroup
.org.uk/objects/co206028/north-
eastern-railway-dynamometer-car-
railway-carriage
[source]
https://victorianweb.org/technology/rail
ways/44.html
[source]
https://en.wikipedia.org/wiki/Dynamom
eter_car#/media/File:X32_dynamomet
er_car.jpg

14. Gauging through measurements
[source] https://collection.sciencemuseumgroup.org.uk/objects/co206028/north-eastern-railway-
dynamometer-car-railway-carriage
[source]
https://en.wikipedia.org/wiki/Dynamometer_car#/media/File:X3
2_dynamometer_car.jpg
[source] https://americanprime.com.br/

15. ● How AI is built?
● Where is located in applications?
Sensors for AI?

16. AI model construction in a nutshell
[source] Ottun, Abdul-Rasheed., &.. Flores, Huber. (2023). Detection mechanisms to identify data biases and exploratory studies about
different data quality trade-offs. Deliverable 3.1, H2020 EU SPATIAL, 2023
Standard Machine
Learning Pipeline
[source] https://courses.minnalearn.com/en/courses/trustworthy-ai/overview/

17. AI models in applications
[source] Ottun, Abdul-Rasheed., &.. Flores, Huber. (2023). Detection mechanisms to identify data biases and exploratory
studies about different data quality trade-offs. Deliverable 3.1, H2020 EU SPATIAL, 2023

18. AI models in applications
[source] Ottun, Abdul-Rasheed., &.. Flores, Huber. (2023). Detection mechanisms to identify data biases and exploratory
studies about different data quality trade-offs. Deliverable 3.1, H2020 EU SPATIAL, 2023

19. AI models in applications
[source] Muccini, H., & Vaidhyanathan, K.
(2021, May). Software architecture for ml-
based systems: what exists and what lies
ahead. In Proceedings of IEEE/ACM
WAIN@ICSE 2021 (pp. 121-128). IEEE.
Classical architecture
+ Machine learning

20. AI models in applications
[source] Muccini, H., & Vaidhyanathan, K.
(2021, May). Software architecture for ml-
based systems: what exists and what lies
ahead. In Proceedings of IEEE/ACM
WAIN@ICSE 2021 (pp. 121-128). IEEE.
Classical architecture
+ Machine learning

21. AI models in applications
Classical architecture
+ Federated learning
[source] Ottun, Abdul-Rasheed., &.. Flores, Huber.
(2023). Detection mechanisms to identify data
biases and exploratory studies about different
data quality trade-offs. Deliverable 3.1, H2020 EU
SPATIAL, 2023

22. AI models in applications
[source] Muccini, H., & Vaidhyanathan, K.
(2021, May). Software architecture for ml-
based systems: what exists and what lies
ahead. In Proceedings of IEEE/ACM
WAIN@ICSE 2021 (pp. 121-128). IEEE.
Classical architecture
+ Machine learning

23. AI models in applications
[source] https://www.thefountaininstitute.com/blog/chat-gpt-ux-design
[source]
https://media2.giphy.com/media/W1fFHj6LvyTgfB
Ndiz/200.gif
[source]
https://media1.tenor.com/m/khe_nqmAFJMAAAAC
/driverless-car-veritasium.gif

24. AI models in (existing) applications
Movie matching score
= 75%
AI recommender and
personal guidance

25. AI models in (existing) applications
[source]
https://www.pinterest.com/pin/511440
101439075063/

26. AI sensors and dashboards
AI Sensor: Fairness
Education
Basic
High school
College
Socioeconomic
status Low
Middle
High
Age
Children
Adolescent
Adult
More…
AI movie recommender
AI
Sensor
AI sensors:
• Fairness
• Explainability
• Performance
• Robustness
• Transparency
• …..
Each AI sensor is
linked to a
trustworthy
property
[source] H. Flores: “AI Sensors and
Dashboards”, Computer Magazine

27. AI sensors: Instrumentation
AI trustworthiness
Fairness 85%
Accuracy 89%
Resilience 65%

28. AI sensors and dashboards
Monitoring and
adjusting
Trust
score?
Interest
Power
Users
System
Operators Testers
Developers
Auditors
low medium high
low
medium
high
[source] Boerger, Michell., &.. Nikolay Tcholtchev. (2022).
Requirements analysis for AI towards addressing security
risks and threats to system and network architectures
Deliverable 1.1, H2020 EU SPATIAL, 2022

29. AI sensors and dashboards

30. SPATIAL: AI sensors and dashboards

31. Q&A
Thank you!
huber.flores@ut.ee
http://spatial-h2020.eu
This project has received funding from the European Union’s Horizon 2020 research and innovation
programme under grant agreement N° 101021808.
AI
Sensor