This presentation is a supplementary material for the following article -> Nikiforova, A. (2020). Assessment of the usability of Latvia’s open data portal or how close are we to gaining benefits from open data. In In IADIS 14th International Conference on Interfaces and Human Computer Interaction (pp. 51-28). Nowadays, more and more countries are launching their own open data portals, seeking to provide their citizens with open data in a form that is useful and suitable for the original purpose of the open data, and Latvia is not an exception. Despite the fact that the Latvian open data portal was launched only in 2017, it is considered to be a fast-tracker. However, despite the overall high evaluations, critical voices,and many discussions about whether the Latvia’s open data portal is of sufficient quality to be appeared. Therefore, while previous studies deal with quality of open data, this study focuses on the analysis of the Latvian open data portal and aims to find the key challenges that may have a negative impact on user experience.The paper assesses the current situation and recommends corrective actions,highlighting the aspects to be considered when developing and improving open data portals.

1. ASSESSMENT OF THE USABILITY OF LATVIA’S OPEN
DATA PORTAL OR HOW CLOSE ARE WE TO GAINING
BENEFITS FROM OPEN DATA
14th International Conference on Interfaces and Human Computer Interaction (part of 14th Multi Conference on Computer Science and Information Systems)
21 – 23 July 2020
Anastasija Nikiforova
Faculty of Computing, University of Latvia
Anastasija.Nikiforova@lu.lv

2.  in order to benefit from the open data, a sufficiently high-quality and user-friendly open data portal should be provided,
allowing access to the data for all stakeholders, which would result in the transformation of the data into value and
knowledge for the society;
 despite many countries develop and launch their own OGD portals, they received a great deal of criticism from both, society
and technical experts - Latvia is not an exception;
 since the Latvian open data portal is relatively new, it is not included in other studies, while other portals of our neighbours
(Lithuania, Estonia, Finland) have been investigated;
 in order to bring added value at the international level, this study provides brief comparison with other portals,
highlighting the aspects to be taken into account when developing and improving open data portals.
RATIONALE FOR THE STUDY
Aim: to explore Latvia’s Open Data portal, by evaluating it from different perspectives.
Research questions: whether our Open Data Portal is of adequate quality?
what are the main challenges to be addressed?

3.  the usability of open data portals is being studied
even less frequently - over the last 5 years, only
6% of studies covering open data portal topic at
least a little, mention the usability;
 compared to the leading countries, the topic of
Latvia’s open data portal is covered nearly 12
times rarely (1.33% of all studies, while for the
leading countries this ratio exceeds 15%).;
 the popularity of this topic in scientific literature
may sometimes (but not necessarily) be linked to
the state of the open data portals, as the number
of studies demonstrates the public’s (at least the
scientific part of society) interest in the subject.
STATE OF THE ART
despite the increase in popularity of the topics of the
OGD and open data portals, the ratio of these
researches to the researches dealing with open data
does not exceed 7%

4.  Existing studies:
✘ cover their own national OGD initiatives, which in most cases turns to the an assessment of national open data
portal(s), assessing them from different perspectives such as:
 data, functionality, features,
 stakeholder participation, stakeholder feedback;
 the relevance of the data sets of a portal to the “5-stars” classification etc..
✘ focus mainly on data delivery and the data environment, considering what data providers have done to facilitate
users, but have not actually consulted users;
✘ lack a user perspective;
✘ lack a common methodology that would allow comparisons between studies and portals.
STATE OF THE ART

5. STATE OF THE ART. BENCHMARKS AND INDEXES
 Global Open Data Index follows the state of the OGD of 94 countries analysing 15 key datasets per country,
 only 11% of the data set entries were open according to their open definition,
✘ data are currently available only for 2016 (the evaluation process is quite complex), their finding may be considered
outdated;
 Open Data Barometer provides a snapshot of OGD practices focusing on open data readiness, implementation, and emerging
impacts.
✘ the most urgent, the 4th edition assessed these aspects for a sample of 30 countries in 2017 (113 countries in 2016);
 EU Open Data portal assesses 4 key aspects, namely policy, portal, impact and quality.
✔ one of the most up-to-date assessments (is used in the study)
 etc.

6. BRIEFLY ABOUT THE PORTAL
Date of the launch: 2017
Owner: Latvian Ministry of Environmental Protection and Regional Development
Rank (according to European Data Portal Maturity Report): 11
Number of datasets: 413*
Number of publishers: 80*
Number of categories: 14
*July 24

7. Latvia is one of 70 countries participating in
the Open Government Partnership - an
international platform for domestic reformers that
committed to making their governments more open,
accountable, and responsive to citizens;
BRIEFLY ABOUT THE PORTAL
at the moment of its launch
33 data sets
from 13 data publishers
in July of 2018
139 data sets
from 41 publishers
in July of 2020
413 data sets
from 80 publishers.
Latvian open data portal has the highest rate of open data maturity in comparison
with neighbourhoods from Baltic States and Scandinavian countries.
Languages: Latvian and English
✔ attempt to ensure multilingualism
✘ content uniformity/ homogenity in different languages is not ensured
The most popular data formats: .CSV, .XLSX, .WMS
✔ data are machine-readable (is still a problem for many countries)
✔ the number of geospatial data increases significantly – the nature of data is up-to-date

8.  The impact and portal for Latvia are the
worst aspects among impact, policy,
portal, and quality (65 and 78%);
 Maturity of Latvian open data portal:
• in 2016 – 30th,
• in 2017 – 21st,
• in 2018 - 19th,
• in 2018 - 11th,
For more detail see Nikiforova, A. (2020) - Comparative analysis of national
open data portals or whether your portal is ready to bring benefits from open data.
In IADIS International Conference on International Conference on ICT, Society
and Human Beings
Rank Country 2019 2018 2017 2016
1 France 1 ↑+4 5 ↑+3 8 ↓-5 3
2 Spain 2 ↑+6 8 ↓-3 5 ↓-1 4
3 Ireland 3 3 3 ↑+7 10
4 Cyprus 4 ↓-2 2↑+22 24↓-6 18
5 Finland 5 ↓-4 1 ↑+5 6 ↑+3 9
6 Slovenia 6 ↑+1 7↑+13 20↓-7 13
7 Austria 7 ↑+3 10↓-1 9 ↓-4 5
8 Romania 8 ↓-4 4 4↑+11 15
9 Luxembourg 9 9 ↓-8 1 1
10 Netherlands 10↑+4 14↓-4 10↓-3 7
11 Latvia 11↑+8 19↑+2 21↑+9 30
12 Poland 12↓-1 11↑+12 23↑+1 24
13 Italy 13 13 13↑+6 19
14 Germany 14↑+9 25↓-23 2 ↑+4 6
15 Greece 15↓-9 6↑+20 26 26
16 Croatia 16↑+4 20↓-13 7 ↑+5 12
THE MATURITY OF OPEN DATA PORTALS
ACCORDING TO EUROPEAN DATA PORTAL
↓-n – decrease in rank by n positions compared to the previous year
↑+n – improve in rank by n positions compared to the previous year
ranking – below EU28+ average
*The analysis is carried out by author
✓ Latvia demonstrates the
most impressive result in
terms of continuous
development;
✓ only 4 countries improve
their positions from year to
year – Ireland, Latvia, Italy,
and Malta

9. Why? (Máchová et al., 2018)*** usability evaluation framework
✔ reflects all the functionality of the portal and typical tasks normally performed by users;
✔ considers a user perspective;
✔ complies with the majority principles used or mentioned by other well-known researchers ((Charalabidis et al., 2018),
(Attard et al., 2015), (Zuiderwijk et al., 2015));
✔ well-cited by other researchers.
What does it suppose?
 3 categories (data specification, data set feedback, data set request)  14 aspects (see the next slide
for detail)
 3-point Likert scale (1 – not fullfilled, 2 – partly, 3 - fulfilled)
***Máchová, R. et al. (2018). Usability evaluation of open data portals. Aslib Journal of Information Management
STAGE I:
a questionnaire of
Latvian citizens
(105 participants)
STAGE II:
(Máchová et al., 2018)*** usability evaluation
framework applied to the portal by 35 participants
with IT background*
*well-designed experiment requires at least 30 participants - ✔
STAGE III:
additional aspects not covered in the
applied methodology (or not detailed
enough) analysed by addressing
every data set
(395 data sets)
KEY POINTS

10.  the number of non-IT experts who are aware of the
existence of Latvia’s open data portal is higher than
the number of users with IT background by 13%,
 this trend is not valid in the case of its use -
only 7.7% of non-IT users use this portal regularly,
while for IT-users this ratio is 12%.
 the assessment received from IT users was lower by 1
point in the case of usability and 0.5 point in the case
of quality.
 105 voluntary participants representing three groups: (a) users with IT background,
(b) without IT background, divided into (b1) undergraduated and (b2) graduated.
 A list of simple question, including:
(1) whether the participant knows that Latvia has its own national open data portal?
(2) how he/ she assesses it (if «yes» for the 1st question) (2.1.) from a “usability”
perspective, (2.2.) from the quality of the data.
 only 53% of respondents are aware of the existence of Latvia’s open data portal,
13.5% suppose that this exists, however, are not sure,
 74.3% of those, who know about the existence of the portal, have not used it,
18% used it only once and only 7.7% use it in a daily manner.
 the overall [subjective] usability of the portal was assessed by 6.3 points out of 10, while the
quality of published data – 6.2 out of 10.
Despite the high results according to international assessment systems,
Latvian users are not satisfied with this portal in the highest way.
STAGE I: SURVEY

11. Category Aspect Description
Open dataset
specification
a) Description of dataset Portal provides datasets together with their description and how and for what purpose they were collected
b) Publisher of dataset Portal provides information about organization that published datasets
c) Thematic categories and tags
Portal provides thematic categories of datasets to address the main topics covered. It distinguishes categories (themes)
from tags (keywords)
d) Release date and up to date Datasets are associated with a time or period tag, that is, date published, date updated and its frequency
e) Machine-readable formats Portal provides datasets formats that are machine-readable and allow easy re-use
f) Open data licence Portal provides license information related to the use of the published datasets
g) Visualization and statistics
Portal provides visualization and analytics capabilities to gain information about a dataset, e.g. in charts or
visualizations in maps.
Open dataset
feedback
a) Documentation and tutorials Portal provides high quality of documentation and tutorials to help users
b) Forum and contact form
Portal provides an opportunity to submit feedback on a dataset from the users to providers and forum to discuss and
exchange ideas among the users
c) User rating and comments Portal provides capabilities allowing the collection of user ratings and comments
d) Social media and sharing
Portal provides the integration with social media technologies to create a distribution channel for open data and sharing
feedback
Open dataset
request
a) Request form Portal provides a form to request or suggest new type or format type of open data
b) List of requests Portal provides a list of requests received from users, including the current state of request processing
c) Involvement in the process Portal provides capabilities allowing the involvement in the same dataset
STAGE II: ASPECTS COVERED IN THE
FRAMEWORK TO BE APPLIED

12. * Categories
• I category: open data specification
• II category: open data set feedback
• III category: open data set request
 Result: 31.47 out of 42
 Average result: 2.25 out of 3 points
 Rank #17 out of 42 countries
 Rank #11, if you look on the countries included
in EDP rating
 When recalculated as a percentage (where not
fullfilled is 0), the average level of usability is 62,4%
Not bad, BUT, we wish to be better
For more detail see Nikiforova, A. (2020) - Comparative analysis of national open data
portals or whether your portal is ready to bring benefits from open data. In IADIS
International Conference on International Conference on ICT, Society and Human Beings
# Country Languages I* II* III* TOTAL Rank
(EDP)
1 Cyprus EL, EN 20,2 10,63 7,34 38,17 4
2 Russia RU, EN 16,61 11,39 8,49 36,49 N/A
3 France EN, FR, ES 20,2 9,38 6,59 35,93 1
4 Spain ES, CA, GL, EU, EN 17,97 9,58 8,01 35,56 2
5 Taiwan EN, TW, CN 17,09 10,84 7,06 34,99 N/A
6 Canada EN, FR 17,57 8,97 8,22 34,76 N/A
7 Austria AU 18,84 11,59 4,31 34,74 7
8 Colombia ES, EN 17,59 8,36 8,24 34,19 N/A
9 New Zealand EN 17,67 7,26 8,78 33,71 N/A
10 Ireland EN, GA 17,7 8,79 7,09 33,58 3
11 Portugal EN, FR, ES, PT 19,01 10,69 3,65 33,35 25
12 Finland FI, SV, EN 18,7 10,85 3,35 32,9 5
13 Lithuania LT, EN 15,27 8,19 8,75 32,21 27
14 Slovenia SL +Google Translate 18,71 9,15 4,28 32,14 6
15 India EN 18,39 8,16 5,22 31,77 N/A
16 Netherlands NL 15,97 8,41 7,16 31,54 10
17 Latvia LV, EN 16,43 8,28 6,76 31,47 11 **
18 USA EN 17,54 8,97 4,63 31,14 N/A
19 Singapore EN 19,08 6,51 4,90 30,49 N/A
20 Estonia EE, EN 16,81 8,66 4,94 30,41 18
21 Slovakia SK, EN 16,08 9,48 4,58 30,14 29
n – competitive in a specific aspect, should be considered as an example
n – very weak result, improvement needs to be made, it is worth looking at leaders (in green)
RESULTS I

13. Category Aspect Points (1 to 3 points)
Open dataset specification
a) Description of dataset 2,07
b) Publisher of dataset 2,74
c) Thematic categories and tags
2,36
d) Release date and up to date 2,9
e) Machine-readable formats 2,1
f) Open data licence 3
g) Visualization and statistics 1,26
Open dataset feedback
a) Documentation and tutorials
1,98
b) Forum and contact form 2,02
c) User rating and comments 1,76
d) Social media and sharing 2,52
Open dataset request
a) Request form 2,83
b) List of requests 1,88
c) Involvement in the process 2,05
RESULTS I

14. 0 0,5 1 1,5 2 2,5 3
2,58
2,46
2,64
2,40
2,22
2,63
1,77
2,17
1,94
1,65
2,15
2,06
1,55
1,54
Average rating for 42 countries
the aspect is assessed worse than in average for 42 countries the aspect is assessed better than in average for 42 countries
good enough
should be improved in the future, but can wait
must be improved!!!
RESULTS II

15. RESULTS III
good enough
should be improved
in the future, but can
wait
must be improved!!!
Overall OK, but far away from the excellent result and leaders
Feedback must be improved
NB! Feedback is the worst aspect for open data of all countries

16. For more detail see Nikiforova, A. (2020) - Comparative analysis of
national open data portals or whether your portal is ready to bring benefits
from open data. In IADIS International Conference on International
Conference on ICT, Society and Human Beings
RESULTS IV. COMPARISON WITH 41 COUNTRY

17. One of the most crucial aspects for open data, which affects users’ intention to reuse data and increase their value, however, is rarely
ensured.
the frequency of updates promised by the data publisher VS. the actual frequency of updates 
3 parameters were analysed: “created”, “last updated” and “frequency”
12,0%
2,7%
14,7%
5,4%
0,5%
4,2%
6,1%
27,7%
0,2%
18,6%
1,0%
6,1%
0,7%
0%
20%
40%
60%
80%
100%
%
of
datasets
frequency of updates
Values of "frequency of updates"
STAGE III. FREQUENCY OF UPDATES
46%
25%
29%
Whether the frequency of updates of
the data is in line with real life? (%)
yes
no
hard to say
 the use of different names designating the same object, or its property was
observed (e.g. “constantly” and “constantly updated”)  it is recommended to
involve a list of allowable values or to develop and introduce a vocabulary,
 values such as “unknown” and “other” should not be allowed since they do not
provide supposed information.

18. STAGE III. WHETHER API IS
AVAILABLE?
data sharing in the form of machine-readable APIs is more useful for its further use
and is one of the most critical success factors for open data initiatives,
one of the most significant challenges for open data, and, in the context of Latvia, this
is also valid - only 43.04% of datasets (170 out of 395) are supplemented with API
feature.
43%
57%
Whether API is
available? (%)
yes
no
51%
49%
Whether preview option
is available? (%)
yes
no

19. STAGE III. WHETHER THE DESCRIPTION
OF THE PARAMETERS IS AVAILABLE?
 a description of parameters are aimed at clarifying, what the name of each parameter means;
 only 20% of datasets are supplied with a description of parameters, where the textual
description appears to be less popular, since 54.4% of the data sets are supplemented with
the description available as a separate file.
21%
79%
Whether the description of
the parameters is available?
(%)
YES
NO
45%
55%
What is their form?
text
file
93%
7%
Whether the description
of dataset is provided?
(%)
YES
NO

20.  Latvia’s open data is re-used;
 the most popular topics demonstrate the
potential of open data and opportunities to
improve citizens’ lives by addressing topics
that are crucial for Latvia:
 tourism,
 calculation queues to the hospitals or
kindergarten,
 finding parking,
 finding the best place of life.
!!!It is strongly recommended to publish this
information on the portal to demonstrate the
positive effects of the use of open data, thereby
increasing public participation.
STAGE III. USE-CASES OR SHOW-
CASES
 it is good practice to provide users with information on the number of applications based on
open data that have been re-used – use-cases, which are able to attract more people to the
portal and its data - these data are not provided.
 71.43% of 42 analysed open data portals provide this feature, including our neighbours –
Lithuania (9), and Poland (27).
 France, Luxembourg or Portugal has even a “use case upload” feature,
 18 portals offer a mapping between the use cases and the datasets they are based on, e.g.
Ireland.
✘

21. BUT WE ARE GOOD ENOUGH IN …
despite a list of the challenges mentioned above that has not yet been solved, Latvia’s open data portal holds sufficiently high
positions in various ratings. Among the positive aspects, there are:
✔ the number of datasets and data publishers are provided and frequently updated;
✔ two languages, namely Latvian and English, are supported;
✔ the majority of datasets are downloadable and have at least basic set of [meta]data on it;
✔ search and filter by category, data format, tags, data publisher;
✔ a social media facility that can help create a social distribution channel for open data;
✔ a request form (appears to be a problem to a list of countries);
✔ a contact facility with data publisher;
✔ guidelines and news,
etc.
However, some aspects have not been checked on whether they are working as intended, thus, it is difficult to
draw any conclusions on their quality - only the fact of their presence is highlighted.

22.  The study addresses the various researchers’ calls by dividing the analysis into three parts:
(a) a survey which studies the knowledge of the Latvian society (105 participants),
(b) the application of (Máchová et al., 2018) framework to the Latvia’s open data portal (40 participants),
(c) an analysis of additional aspects inspecting each data set (395 datasets).
 the current situation is being assessed, and corrective actions are being recommended;
 the implementation of the guidelines defined should also improve user participation by encouraging the awareness and reuse
of open data, which appears to be one of the most important but weakest aspects;
 although the results of this study are more important mainly for open data portal holders, they could also be useful for users,
as the results:
(a) point to the weakest points to be taken into account using open data portal,
(b) provide aspects to be considered when selecting the portal to be used,
(c) potential open data publishers can use the results provided as a checklist when preparing their data for publishing.
RESULTS

23.  This study will be continued to carry out an in-depth analysis of the identified challenges
 to propose guidelines to address them,
 to provide a detailed comparison with more successful open data portals,
 a detailed literature review will also follow,
In addition, it is planned to find foreign collaborators (preferably from the countries representing the
leading portals) to jointly address challenges through exchange of experience.
FUTURE WORK

24. I am grateful to the participants taking part in the experiment within my workshop “Open data and data quality”,
which allowed to collect data and led to such results,
and Latvian Open Technology Association for sharing information regarding open data hackathons.
ACKNOWLEDGMENT

25. THANK YOU!
For more information, see ResearchGate
See also anastasijanikiforova.com
For questions or any other queries, contact me via email - Anastasija.Nikiforova@lu.lv
Article: Nikiforova, A. (2020). Assessment of the usability of Latvia’s open data portal or how close are we to
gaining benefits from open data. In In IADIS 14th International Conference on Interfaces and Human
Computer Interaction (pp. 51-28).