This is my presentation on the Workshop on Digital Governance Science Base at the ICEGOV 2020 Conference. It includes the proposals for the structure, the neighboring domains, the assessment frameworks and problem space systemetisation,

1. Digital Governance Science Base:
On Assessment Methods, Problem and Solution Space
Yannis Charalabidis
Professor of Digital Governance
University of the Aegean
yannisx@aegean.gr
Zoi Lachana
Researcher, Ph.D. Candidate
University of the Aegean
zoi@aegean.gr
Panos Keramidis
MSc Student
University of the Aegean
icsd16067@icsd.aegean.gr

2. PART A
INTRODUCTION TO DGSB INITIATIVE
(Workshop number #10 on the subject)

3. Why do we need a “science base” for Digital Governance?
▪ There are hints, since Plato times, that governance has to be treated as a
science
– Plato, Politikos: “… the science of governance, this most difficult but also the most
important of all …”
▪ There are a lot to be gained, by systematically organizing knowledge and
practice in this important domain, for administration and societies: minimization
of solution time and cost, maximization of administration productivity, avoidance
of failure, better quality of life for citizens
▪ The neighboring of the Digital Governance domain with other, well – formulated
scientific domains like computer science, management science or law, brings
more value to the experiment

4. When will Digital Governance be considered a scientific domain ?
An example ?
How do we know that we have a method ?
The “Yannis Test” on Digital Governance Science Base
Digital Governance is a well-structured and manages scientific domain, if:
Whenever 2 independent, randomly selected Digital Governance experts (practitioners
or DG science)
are exposed to the same administration situation – problem, and working
separately, they:
‒ Come to the same diagnosis for the situation – problem of the administration
(from a technical, semantic, organizational, legal and policy or other aspects)
‒ Propose the same set of actions so that the administration will reach the
desired state.
Input from epistemology and various DGSB workshops says:
“Science means METHOD, defined, applied and evolving in a systematic way”

5. What are the elements of the Science Base ?
Rationale
(justifications on the need for a science
base supported by evidence)
WHY

6. What are the elements of the Science Base ?

Research Roadmap
(new research
directions, evolution
scenarios)
Domain Structure
(ontology of scientific
areas, sub-areas,
terms, sectors, etc)
Neighboring Domains
(domains, “points of
connections”, structure,
tools, methods, ideas,
laws, etc)
Training Curricula
(training programmes,
training courses,
material)
WHAT
Solution Space
(solution paths, cases,
knowledge base of
successes and failures)
Problem Space
(problem classifications,
multidimensional vector
definitions)
Assessment Tools
(assessment
frameworks,
assessment tools,
KPI’s)
Solution Methods &
Tools
(frameworks,
methodologies, toolsets,
applications, algorithms)
HOW
Rationale
(justifications on the need for a science
base supported by evidence)
WHY

7. What are the elements of the Science Base ?

Research Roadmap
(new research
directions, evolution
scenarios)
Domain Structure
(ontology of scientific
areas, sub-areas,
terms, sectors, etc)
Neighboring Domains
(domains, “points of
connections”, structure,
tools, methods, ideas,
laws, etc)
Training Curricula
(training programmes,
training courses,
material)
WHAT
Solution Space
(solution paths, cases,
knowledge base of
successes and failures)
Problem Space
(problem classifications,
multidimensional vector
definitions)
Assessment Tools
(assessment
frameworks,
assessment tools,
KPI’s)
Solution Methods &
Tools
(frameworks,
methodologies, toolsets,
applications, algorithms)
HOW
Rationale
(justifications on the need for a science
base supported by evidence)
WHY

8. What are the elements of the Science Base ?

Research Roadmap
(new research
directions, evolution
scenarios)
Domain Structure
(ontology of scientific
areas, sub-areas,
terms, sectors, etc)
Neighboring Domains
(domains, “points of
connections”, structure,
tools, methods, ideas,
laws, etc)
Training Curricula
(training programmes,
training courses,
material)
WHAT
Solution Space
(solution paths, cases,
knowledge base of
successes and failures)
Problem Space
(problem classifications,
multidimensional vector
definitions)
Assessment Tools
(assessment
frameworks,
assessment tools,
KPI’s)
Solution Methods &
Tools
(frameworks,
methodologies, toolsets,
applications, algorithms)
HOW
The big picture
(Rules, Theories, Laws)
Rationale
(justifications on the need for a science
base supported by evidence)
WHY
The correct application
(Code of Ethics)

9. Comparing Digital Governance and Medical Science

10. PART E
ON ASSESSMENT METHODS AND
PROBLEM SPACE

11. Let’s talk about the green side of things …

Research Roadmap
(new research
directions, evolution
scenarios)
Domain Structure
(ontology of scientific
areas, sub-areas,
terms, sectors, etc)
Neighboring Domains
(domains, “points of
connections”, structure,
tools, methods, ideas,
laws, etc)
Training Curricula
(training programmes,
training courses,
material)
WHAT
Solution Space
(solution paths, cases,
knowledge base of
successes and failures)
Problem Space
(problem classifications,
multidimensional vector
definitions)
Assessment Tools
(assessment
frameworks,
assessment tools,
KPI’s)
Solution Methods &
Tools
(frameworks,
methodologies, toolsets,
applications, algorithms)
HOW
The big picture
(Rules, Theories, Laws)
Rationale
(justifications on the need for a science
base supported by evidence)
WHY
The correct application
(Code of Ethics)

12. Assessment Tools : The big picture
Medicine Governance I Governance II
Assessment Tool
Blood Test
- Red Hemospheres
- White Hemospheres
- Cholesterol
- …
DESI index
- Connectivity
- Human Capital
- Use of Services
- …
IMAPS survey
- Legal
- Technical
- Semantic
- …
What is being
measured (subject)
Blood (of Humans) Countries (of Europe) Digital Services
How is it measured
(method)
Laboratory analysis
(automated)
Assessment by
trained staff (manual)
Assessment by
trained staff (manual)
What is the outcome
(results)
35 indicators in 7
categories
48 indicators in 6
categories
60 indicators in 4
categories
Is there a history ?
(the time element)
Yes Yes Yes
Do we know what is
good or bad ?
Yes, there are ranges
for ”normal”
Yes, but mainly in
relation to others
Yes, but mainly in
relation to others

13. Assessment Tools in Digital Governance : The Subject
What can be the Subject of a DG assessment ?
– A Service
– A System (composed of services, but also including other stuff)
– An organisation (composed of systems, but also including other stuff)
– People (their skills, their behavior, their opinions, and more)
– A City of a country
– A Region of a country
– A Country
– A set of Countries (e.g. a Continent)
– Other ??

14. Assessment Tools in Digital Governance : The Method
What can be the Method of a DG assessment ?
– A manual process (*), to produce the assessment results for the subject, utilizing the
structure of the assessment method, carried out by non-experts
– An automated process, where the assessment results are produced by systems
(sensors, information systems, web services, etc)
– A hybrid process, combining the above.
(*) the manual process may include
– digital inspection of services or systems
– sampling of opinions, through questionnaires or interviews
– gathering of data existing elsewhere
Postulation: the manual “raw” assessment, by means of putting values to
indicators (and not explaining what they mean) is not a process for experts
(e.g. doctors). Trained personnel (e.g. laboratory staff) can do the job.

15. Assessment Tools in Digital Governance : The Results
How do assessment results look like, in general ?
They contain:
– Categories (and sub-categories, and sub-sub categories …) of indicators
– Indicators
– Values of indicators DESI
1.Connectivity
1.a1 Fixed
broadband
coverage (%)
VALUE : 87%
1.a2 Fixed
broadband take-
up (%)
VALUE : 77%
2. Human
Capital
2.a1 At least basic
digital skills (%)
VALUE : 35%
2.a2 Above basic
digital skills (%)
VALUE : 13%
5 categories
48 indicators
Indicator = (Code*, Label, Measurement Unit, Measurement Range, Value)
* Code can contain the category, if properly designed

16. So how do an assessment look like ?
So, what ?
Assessment
Code I1 I2 I3 I4 I5 I6 … In
Label blabla blabla blabla blabla blabla blabla blabla
M. Unit % Integer List % Real List … %
Range 0-100 0-4 A-B-C 0-100 0-1 X-Y-Z … 0-100
Value 75 1 A 14 0.256 Υ … 32
Definition
Value
Or, if we just care for the value part:
Assessment = { V1, V2, V3, … , Vn }, where Vi is the value of indicator i

17. From assessment tools to the “book of diagnoses”
Hm, if assessment results are seen as vectors, with n-dimensions, … then a
specific vector denotes the current situation of what is being measured:
- DESIGREECE-2020 = { 17%, 31%, 46, 67%, 75%, 99, 0, 32%, …, 93% }
- DESIITALY-2019 = { 14%, 81%, 46, 93%, 70%, 91, 4, 22%, …, 88% }
- IMAPS PASSPORT ISSUING GREECE 2019 = {2, 4, 3, 0, 1, 1, 0, 0, 2, 2, 3, 4, 4, 3, 2, …, 1}
And then, we can have standards patterns of “problems”, like
IMAPS good service = {>2, >3, >1, >0, >3, >3, >3, =4, <3, 2-3, >3, 4, 4, 3, 2, …, 1} or
DESIEU average = { 25%, 35%, 50, 67%, 78%, 99, 0, 32%, …, 90% }
DESITYPICAL EU SOUTH= {15-17%, 20-30%, <45%, 0, >87%, …,80-90%}
IMAPS lack of technical interoperability = {-,-,-,-,-,-,-,<1,<3, <2, -,-,-,-,0,0,1,1}

18. Problem Space
We call Digital Governance Problem Space the n-dimensional space of
assessment / situation patterns that denote problematic situations that (usually)
need improvement, for a specific subject (e.g. a service, a city, a country)
An assessment model
(e.g.IMAPS)
Situation change : a (solution) path

19. Going from Situation-A to Situation-B
And then, we can have current situations (as-is) and target situations (to-be), eg:
- DESIITALY-NOW = { 17%, 31%, 46, 67%, 75%,…, 93% }
- DESIITALY-TOBE = { 24%, 81%, 46, 93%, 75%,…, 98% }
Diagnosis by the expert, based
on knowledge of problem space
patterns: “I see low connectivity,
very poor service automation,
good human skills, biggest
problem is interoperability”
Prescription by the expert,
based on knowledge of
solution paths : “hm, you have
to redesign several key
services, subsidise broadband
for SME’s, go mobile. See you
again in 3 months.”

20. As a conclusion on Assessment, Problem Space and Solution Paths
We need to systematically collect and analyse all assessment methods (e.g. DESI,
IMAPS, UN eGOV, Layne & Lee, etc, etc) and categorise them as to what they measure,
how, what they produce, etc. This taxonomy of methods will be our (DG Science Base)
set of “examination tools”
Assessment tools create multidimensional vectors as results. For each assessment tool,
we need to define typical patterns (ranges of values, for each dimension). These patterns
will our “problem space” (book of diagnoses)
For a subject (e.g. service, system, city, country, etc) to in a better situation, so from a
typical situation A to a typical situation B, we need to define some partways (set of
actions), based on our knowledge and experience. This will be our set of “drastic
substances” or “therapy protocols”
Then, we will have conquered Assessment tools, Problem Space and Solution Paths,
for Digital Governance Science Base …

21. Question Group 1
How do you rate the importance and possible impact of Assessment Tools /
Problem and Solution Space proposed approach within the DG science base ?
(1-5)
How do you rate the scientific rigor of the approach ? (1-5)
Do you have any other ideas for organizing or systematizing Assessment Tools /
Problem and Solution Space ?

22. A list of Assessment Frameworks
Country / National Level
– UN E-Government Development Index (EGDI)
– UN E-Participation Index (EPI)
– World Bank Digital Government Readiness Assessment Toolkit
– Digital Government Toolkit - Organisation for Economic Cooperation And Development
– EU DESI index
– EU NIFO - National Interoperability Framework Observatory
City Level
– UN Local Online Service Index (LOSI)
– ISO 37101 - Sustainable development in communities
– World Council on City Data https://www.dataforcities.org
– ITU 4903-2016 Key Performance Indicators for smart sustainable cities
Organisation, Systems & Services Level
– EU European Interoperability Framework
– EU National Interoperability Frameworks (27)
– EU IMAPS service assessment https://joinup.ec.europa.eu/collection/imaps-interoperability-
maturity-assessment-public-service/about

23. Question Group 2 : Assessment methods
How do you rate the various assessment frameworks concerning their
usefuleness for analysing countries, cities and organisations or systems ? (1-5 / I
do not know the framework)
Can you propose other assessment methods or frameworks for Digital
Governance issues ?