HICSS-53 ATLT presentation "Exploring the Learning Outcomes with Various Technologies Proposing Design Principles for Virtual Reality Learning Environments"

1. Exploring the Learning Outcomes with
Various Technologies - Proposing Design
Principles for Virtual Reality Learning
Environments
Holopainen, J. Lähtevänoja, A. Mattila, O. Södervik, I. Pöyry, E. Parvinen, P.
jani.m.holopainen@helsinki.fi

2. Learning in virtual environments
Most considering online or web-based learning environments
consumed from computer or mobile screens e.g.:
• 3D-objects and models enhance spatial understanding, arousal,
motivation, experiential, contextual and social learning as well as
innovation and planning (Dalgarno and Lee 2010)
• Enabling more interactions between user and learning environment
(Wann and Mon-Williams 1996)
• AR learning environments showed improved learning in short-run,
but in the long-run the effect was moderated (Sommerauer and
Müller 2018)

3. Learning in virtual environments
Research on immersive VR consumed with HMDs e.g.:
• VR enabling activities and interactions that are not
necessarily available in the real-world or that are
impossible to carry out or even dangerous in physical
environments (Thackray et al. 2010)
• Immersion is central (Dede 2009, Dunleavy and Dede
2009, Thackray et al. 2010, Scoresby and Shelton 2011)
• VR nor necessarily substituting more conventional
assignments and methods, such as classroom lectures
and laboratory work (Sathe et al. 2017)

4. Learning in virtual environments
• Not improving the learning and efficiency, when
the contents resemble the real-world (Tatli and
Ayas 2011)
• Complex systems and elements presented in VR
help students to concentrate (Pirker et al. 2017)

5. Learning in virtual environments
• Most VR studies
concentrating only on
lower levels of Bloom’s
taxonomy (remembering
and understanding)
(Ebert et al. 2016, Snowdon and
Oikonomou 2018)
• VR has potential to
impact higher levels of
Bloom’s taxonomy
(applying, analyzing and
evaluating) (Parmar et al. 2016)
Bloom’s taxonomy (Bloom 1971; Andersson et al. 2001)

6. Research framework and questions
• Three different virtual learning environments VR, 3D and 2D –videos
• Perceived affordance as explanatory variables (Bailenson et al. 2008,
Dede 2009, Ke et al. 2016)
• Different learning measured as outcome variables i.e. understanding
(textile recognition), remembering (after two weeks), and ability to
apply (drawing test)
• Research question 1: Does the different technology (VR, 3D, 2D)
result in different perceived affordances i.e. can the affordances -
theory be applied in building and explaining different learning
environments?
• Research question 2: What is the effect of the technology (VR, 3D, 2D)
on different learning outcomes (understanding, remembering and
ability to apply)?

7. Data and Method
• 97 people participated in a between-subjects experiment (32 in after-two-weeks survey)
• Three different design artifacts representing three different virtual learning environments 28 (30%) were assigned to VR, 34 (35%) to
3D and 34 (35%) to 2D
• 37% males and 63% females
• 3% under the age of 18, 42% 19-24 years, 41% 25-34 years, 13% 35 years or older
• 72% students, 28% others, 21% employed, 2% unemployed, 5% something else
• 43% had previous experience with some VR devices
Recordings from 3D (left) and 2D -videos (right). The VR experience was exactly the
same as the 3D video but consumed with the HMD and so allowing free moving and
therefore changing the viewpoint.

8. Results
• No statistical significance between technologies when measuring understanding and
remembering
• Cross-tabulation results for the learning results “ability to apply” –level and different
technologies:
Correct FALSE Total
VR^* 21 (75%) 7 (25%) 28
2D* 17 (50%) 17 (50%) 34
3D^ 23 (68%) 11 (32%) 34
^Pearson Chi-Square 4.526 (p=0.104)
*Pearson Chi-Square 4.045 (p=0.044)

9. Results
• The following affordances
differentiated VR from other
environments: customized
learning, challenging learning
environments, multi-sensory
effects, immersion, interactivity,
3D-dimensionality, engagement
as well as motivation towards
the content and technology
• These are the proposed design
principles for the VR learning
environments
VR 3D 2D Total Kruskall-Wallis
Mean SD Mean SD Mean SD Mean SD Sig.
1. I think this application makes it possible to visualize the learning
content (fabric texture).
6.21 0.82 6.09 0.93 5.97 0.97 6.08 0.91 0.63
2. This application makes it possible to visualize complex systems (e.g.
fabric texture).
6.17 0.89 6.00 0.89 6.06 1.13 6.07 0.97 0.58
3. I think this application makes it possible to visualize abstract
phenomena.
5.76 1.02 5.32 1.22 5.38 1.26 5.47 1.18 0.38
4. I think this application makes it possible to visualize microscopic things.
5.69 1.51 5.76 1.10 5.88 1.32 5.78 1.30 0.73
5. I think this application makes it possible to duplicate different learning
tasks.
5.86 1.16 5.41 0.99 5.82 1.11 5.69 1.09 0.10
6. I think this application makes it possible to modify different learning
tasks.
6.07 0.88 5.38 0.89 5.97 0.94 5.79 0.95 0.01*
7. With this application you can make learning tasks which are e.g.
expensive, dangerous or impractical to implement in traditional
classroom.
6.31 1.00 5.91 0.79 6.21 1.15 6.13 1.00 0.03*
8. I think that I can apply knowledge acquired from the application into
practice.
5.32 1.54 5.15 1.52 5.24 1.63 5.23 1.55 0.88
9. Application made collaborative learning possible.
3.93 1.62 3.68 1.68 4.18 1.82 3.93 1.71 0.59
10. I experience this application as multisensory.
5.97 1.02 4.29 1.64 4.74 1.44 4.95 1.56 0.00*
11. I got immersed to the application.
5.83 1.31 4.18 1.53 3.76 1.50 4.53 1.68 0.00*
12. I felt indisposition and dizziness in the application.
1.62 1.21 1.50 1.02 1.41 0.86 1.51 1.02 0.68
13. I felt that I was interacting with the teacher in the application.
4.03 1.82 3.00 1.94 2.24 1.44 3.04 1.87 0.00*
14. Application helps me to visualize in 3D.
6.14 0.95 5.65 0.98 4.79 1.47 5.49 1.28 0.00*
15. I felt that I was interacting with the 3D model in the application.
5.41 1.84 4.56 1.46 3.85 1.78 4.57 1.79 0.00*
16. This application helped me to visualize the texture of the fabric.
6.03 0.87 6.06 1.13 5.97 1.00 6.02 1.00 0.80
17. Learning content (fabric texture) was easy to study with the
application.
6.07 0.92 5.65 1.52 5.29 1.53 5.65 1.39 0.12
18. This application grew my engagement and motivation towards the
learning content (fabric texture). 5.24 1.41 4.24 1.46 3.82 1.60 4.39 1.59 0.00*
19. This application rose my interest to study the topic (fabric texture)
more deeply.
4.14 1.83 3.88 1.49 3.47 1.86 3.81 1.73 0.34
20. I think learning with this application were more motivating than in a
real classroom.
5.66 1.45 4.35 1.79 3.94 1.77 4.60 1.82 0.00*

10. Discussion
• VR has its advances on the apply -level or higher
• VR can easily provide new teaching methods
• Choosing the technology must be aligned with the learning objectives (also lack of
research on alignment of these two)
• Virtual learning environments have been found to be motivating in general (Dede 2009,
Olmos-Raya et al. 2018). Our results suggest that the VR learning environments can show
higher motivation compared to the two other technology environments (3D and 2D).
• Our results comply with following previous studies e.g.
• Multi-sensorial effects, interactions, individual experiences and immersion (Dunleavy and Dede
2009, Wann and Mon-Williams 1996)
• Customized learning (Naik and Kamat 2015)
• Understanding 3D-dimensionality (Zhang 2017)
• 3D models should be unique and not accessible in the real world (Thackray et al. 2010, Wann and
Mon-Williams 1996)
• Improved interactivity and presence (Renaud et al. 2003)

11. Interactivity
and presence
?
VR good in
educating
customers
Customized
learning and
motivation
VR has its
advances on the
apply -level (or
higher)
Multi-sensorial
effects, interactions,
individual
experiences and
immersion
VR can easily
provide new
teaching
methods
Better
understanding 3D-
dimensionality
Technology
must be
aligned with
the learning
objectives
Summary

12. Conclusions
• We found using the affordance framework very useful in evaluating the
different properties of technologies
Implications also to other research fields e.g. marketing and sales
• Future research could consider also other learning levels e.g. analyzing,
evaluation and creation or some other cognitive levels (Krathwohl et al.
2009)
• 3D asset libraries and do-it-yourself engines - research on learning with
these platforms and methods is completely absent
• Learning levels can be also overlapping which should be considered when
planning VR tasks and evaluations (Cannon and Feinstein 2005)
• as well as it’s a limitation for this study
• Also validation between affordances and learning outcomes is needed

13. jani.m.holopainen@helsinki.fi