Web browsing on Extended Reality enables users to navigate and interact with traditional Web-based content, as well as immersive WebXR experiences. In this presentation, we will draw from our experience creating Wolvic, an open-source Web browser for XR devices. We will begin with an overview of current Web-based
solutions for different use cases, from video and gaming to social
collaboration and productivity. We will discuss the technical challenges and design opportunities of creating a Web browser for XR devices. Finally, we will look at the complexities of developing a multiplatform and open-source XR application.
(c) GOSIM Workshop 2023
Sept 23-24
Grand Hyatt, Pudong, Shanghai
https://workshop2023.gosim.org/
https://www.bilibili.com/video/BV18w411r7Nm/
3. Presentation
• Felipe Erias
Software engineer and interaction designer
felipeerias@igalia.com
• Igalia
Igalia is a Free Software consultancy specialised in
open source technologies on a wide set of platforms
https://igalia.com
3
4. Introduction
• Challenges and opportunities of Web browsing on Extended Reality
(
XR
)
,
drawing from Igalia’s experience creating the Wolvic browser
• Web browsing on XR enables users to navigate and interact with
traditional web-based content as well as immersive experiences
• Wolvic is an Open Source Web browser for XR devices which supports
immersive experiences
• Wolvic is the only multi-platform Web browser for XR
4
6. Traditional Web browsing
• On desktop PCs and smartphones
• Windows and tabs, back/forward navigation
• Bookmarks, downloads, history
• Password management
• Support DRM media playback
• Extensions, private mode, etc.
• Some Web APIs need to be specifically implemented for XR
6
7.
8.
9.
10.
11.
12.
13.
14. Input methods
• Typically, XR devices use handheld controllers and/or hand gestures
Other methods are supported by some devices
• But most of the Web is made for desktops and touchscreens
• This mismatch can be noticeable for content that is not designed for XR
• Websites may use input modes which can not be replicated easily with
VR controllers, like pinching and dragging, or keyboard shortcuts
• In Wolvic, we usually “pretend” to be a touchscreen or a desktop;
we are also working to integrate external keyboards and voice input
14
17. Readability
• Reading documents is the most basic way to use the traditional Web
• On XR, the reading experience is limited by the resolution of the headset
• In Wolvic, we use a large window size with large text
• But there are several trade-offs with this solution
The screen can not be too large: it would cover the user’s field of vision
The text can not be too large either: the page would be harder to use
• Hopefully this limitation will get better as the hardware improves
17
18.
19. Productivity
• The large virtual space provides great flexibility to the UI
• We can arrange content to leverage spatial thinking and memory
• We can experiment with richer ways to present information to the user
• UI elements can be positioned on the edge of the user's field of vision,
leveraging peripheral awareness
• XR has the potential to become a productive and flexible way to work
19
26. Video and entertainment
• Usually provided as regular 2D video; also 3D stereoscopic video
• Popular streaming video platforms
YouTube, etc.
Platform must provide playback functionality via the Web (not only app)
DRM media content must be supported by the browser engine
Examples: Disney+, HBO Max, Prime, Xigua Video, Apple TV, U
-
NEXT…
26
32. Immersive video
• Served over the Internet as a regular video media file
Can be distributed via YouTube, etc.
The video needs the right metadata so it can be displayed correctly
• The XR browser displays the video according to a specific geometry
Monoscopic: same image for both eyes
Stereoscopic: one image for each eye
Field of view: 180º, 360º
32
37. Immersive experiences: WebXR
• WebXR is a collection of Web APIs that allow the development of
immersive XR experiences on the Web
https://immersiveweb.dev
https://immersive-web.github.io
https://www.w3.org/immersive-web
https://github.com/immersive-web
• In Wolvic, WebXR is mostly implemented by the Web engine, but
we still need to connect it to the graphical layers, input, etc.
37
WebXR
41. Education
• Both immersive video and interactive content
• XR is the best way to experience immersive information
• The Web makes it very easy to distribute this content
No need to create a separate app, release it on each app store, etc.
• Even existing platforms like YouTube and WikiMedia can be used
41
50. Gaming
• One of the most popular use cases for XR
• Limitations to deliver large assets
• Performance may be lower than in native applications
• Creative indie game developers
50
56. Wolvic: origins
• Wolvic started out as Firefox Reality
Initial release September 2018
Goal: bring Firefox to standalone VR headsets
Support the traditional Web plus immersive experiences
Part of a wider “mixed reality” initiative at Mozilla
• Handed over to Igalia in 2022
• https://wolvic.com
56
57. Components overview
• Custom 3D library (based on GLES2
)
• Web engine
(
Gecko, Chromium)
• Android UI components
• Mozilla mobile Android components (browser functionality)
• OpenXR to access device capabilities
• Some platform-specific libraries
57
58. Web engines
• Web engine: the component that interprets and renders Web content
• Gecko
Used by Mozilla Firefox
Provides a standalone Android library
• Chromium
Used by Google Chrome
Wolvic support currently under development
58
59. Multiplatform
• Wolvic uses OpenXR as abstraction layer to access most XR functionality
• But we still need to cover for devices with different capabilities
• We need to test specifically with each device (and work around bugs)
• We need to take device format into account:
VR headset, AR goggles, phone-tethered glasses…
• Different platforms provide different APIs and capabilities
For example, there isn’t an unified way to create a “desktop” app in XR
59
60. Developed in the open
• Mozilla Public License Version 2.0
• GitHub (code and issues): https://github.com/Igalia/wolvic
• Distribution
App stores: Huawei, Meta Quest, Pico XR
Side-loading packages: https://wolvic.com/dl/
• Funded through OpenCollective: https://opencollective.com/wolvic
60
65. Conclusions
• Web browsing on XR has great potential for a wide range of use cases,
from education and productivity to entertainment and gaming
• The Web lowers the cost of experimentation and distribution
• The Web can be a venue for innovation and creativity in the XR field
• Still open questions: UI design, input, interoperability…
• Wolvic is key as the open source and multi-platform Web browser for XR
65