VR Everywhere March 2015

VR everywhere
consumer virtual
reality for desktop,
mobile and web
tony parisi
march, 2015

about me
http://www.tonyparisi.com 3/18/2015
CONTACT
tparisi@gmail.com
skype: auradeluxe
http://twitter.com/auradeluxe http://www.tonypa
risi.com/
http://www.learningwebgl.com/
GET THE BOOKS!
WebGL: Up and Running
http://www.amazon.com/dp/144932357X
Programming 3D Applications with HTML and WebGL
http://www.amazon.com/Programming-Applications-HTML5-WebGL-
Visualization/dp/1449362966
MEETUPS
http://www.meetup.com/WebGL-Developers-Meetup/
http://www.meetup.com/Web-VR/
BOOK CODE
https://github.com/tparisi/WebGLBook
https://github.com/tparisi/Programming3DApplications
GET GLAM
http://www.glamjs.org/
https://github.com/tparisi/glam/
WORK
http://www.thirdeye.gl/
CREDS
Co-creator, VRML and X3D

the promised land!
consumer VR
3/18/2015http://www.tonyparisi.com

hardware explosion
headsets
input
devices
high performance laptops
AFFORDABLE STEREOSCOPIC
DISPLAYS, CRAZY-FAST GPUS,
CHEAP MOTION TRACKING
IMUS… FTW!

PC HMDs
 Provide deepest, most immersive VR
experiences
 Desktop peripheral - acts as external monitor
 Position and orientation tracking
 Tethered – one or more cables to the
computer; camera for position tracking
 Oculus Rift – the Gold Standard of VR
headsets
 Developer Kit CHEAP $350
 In production late 2015/early 2016
 HTC/Valve VIVE
 OSVR – Razer open dev kit and SDK;
ecosystem initiative

mobile HMDs
 Deluxe Mobile – Samsung GearVR
 Optics and IMU licensed from Oculus
 Custom Android build w/ VR extensions
and enhanced UI
 The best VR experience out there!
(Look Ma, no wires)
 BUT $200 + requires new Phone (Note
4) NOT CHEAP $1,000
 Support for S6 coming, price coming
down
 Orientation tracking only
 So far… no competition… but EXPECT
IT!

drop-in phone viewers
 Google Cardboard
 Just Add SmartPhone
 Orientation tracking only
 Open specification for mobile VR
 Standard Android OS support, using
simple stereo rendering and
accelerometer tracking
 $2 in parts or $25 ready-to-assemble
Kits WAY CHEAP
 Not as nice… but great for the price!
 MergeVR, Wearality, Moggles…
 Some of these have different rendering
thus require custom SDK

operating systems
 Desktop – Oculus SDK supports
Windows/Mac/Linux; Vive SDK not
yet released…
 GearVR - custom Android build
w/Oculus mobile SDK
 Cardboard
 Google has SDK for native Android
with simple stereo render and head
tracking; also adding first-class VR
support into future OS
 Cardboard is just side-by side
rendering and accelerometer
tracking… so it works on iOS too
 Mobile browsers can run Cardboard
apps! Just stereo render with
WebGL and track
w/DeviceOrientation API
The Oculus Store for GearVR

rendering
Oculus Rift/GearVR
 110 diagonal FOV
 Side-by-side stereo
 Barrel distortion
Cardboard
 90 degree FOV
 Side-by-side stereo

head tracking
 Oculus Rift/GearVR
 1000hz refresh rate
 Cardboard
 200hz refresh rate
 Desktop IMUs built into HMDs track position and orientation.
Use camera or wall-mounted sensors for position tracking
 Mobile IMUs track orientation only
INERTIAL MEASURE UNITS (IMUS)

the challenge:
“presence”
 Camera must respond to head tracking at 75FPS+
 higher latency risks motion sickness and breaks the illusion
 Rendering, as always, must be at 60FPS+
 “Motions-to-photons” latency goal is <20ms
 Desktop and GearVR can hit this, currently mobile can’t
(though it’s in the works); most phone IMU motions-to-photons
are are 50ms+
 What’s wrong with this picture???
 Classic targeting of 60FPS makes rendering the bottleneck
 If targeting 75FPS, that’s minimum 150FPS simulation (75FPS
x 2 cameras), or <7ms frame time!!!

performance (1)
 Software techniques
 Of course push everything you can onto the GPU
 New engine optimizations
Example: render each state change to each camera vs. whole scene
to each camera
 Mobile: create experiences with lower res than typical games...
till the hardware gets faster and resolutions get higher; low
latency trumps hi poly count
 Time Warp
 Oculus SDK technique, transforms image in Z-buffer at next refresh
based on new IMU position/orientation… even if new rendered frame
isn’t ready
 Already supported in game engines and other runtimes

performance (2)
First attempts at a comprehensive GPU approach to VR -
NVIDIA VR Direct
 New features enabled in Maxwell processors (GeForce GTX
980 and GTX 970)
 Overall attack on latency – whole pipeline
 Asynchronous Warp (NVIDIA’s version of Time Warp)
 Dynamic Super Resolution (DSR) – up to 4 megapixels per
eye downsampled to HD
 Multi-frame sampled anti-aliasing (MFAA)
 Boasts motion-to-photons latency of 25ms
 … plus “GeForce Experience” control panel – one end-user
settings dialog instead of several
ACCELERATING VR USING THE POWER OF THE GPU

developing desktop and
mobile VR
 Game engines
 Unity3D – targets all platforms: desktop Oculus, Gear VR,
Cardboard. Free and open prefabs available on Unity store and
from HMD vendors. Expect Vive and other platforms quickly; they
move fast.
 Unreal - desktop now, mobile in development
 Native SDKs
 Oculus SDK C++
 Oculus Mobile SDK C++
 Android SDK for Cardboard

WebVR:
fast, cheap and democratized
 Experimental browser API
 Query browser for attached HMD
devices
 Render fullscreen stereo in WebGL
 Track HMD position/orientation by
polling HMD device
 Already in Firefox and Chrome nightly
builds, uses same API
 Build apps using open source
JavaScript engines, or WebGL
“exported” from Unity/Unreal via
Emscripten cross-compiler
 Browser refresh currently throttled at
60FPS but they are lifting this limitation
Quake 3 WebVR demo, developed by Brandon
Jones of Google
http://media.tojicode.com/q3bsp/
NO-DOWNLOAD VR APPLICATIONS.
INSTANT ACCESS IN THE BROWSER, SHARE IT WITH A URL!
WRITTEN IN JAVASCRIPT!?

input
 We still don’t have the mouse of VR
 State of the practice is blind use of
keyboard or gamepad controller
 Motion controllers like Leap Motion,
Sixense STEM system are gaining
popularity for PC-based VR
 GearVR has a really innovative
trackpad design, plus simple, elegant
navigation UI
 Drop-in phone viewers are
experimenting with side-mounted click
devices and Bluetooth controllers
Leap Motion
controller
Sixense
STEM
System
image:
https://www.attachmate.com/blogs/legacymodernization/legacy-data-the-elephant-in-the-room/

augmented reality
 Explosion of VR has renewed
interest in AR
 Promising new hardware
 Microsoft Hololens
 CastAR - table-top AR display for
gaming, close to coming to
market
 Magic Leap ????
 A little farther out… most projects
don’t have ship dates
 Another SDK babel looming…
THE NEXT NEXT BIG THING?
Hololens Sculpting Tool Demo
CastAR Multi-Player Game

coming in May 2015

keep in touch
http://www.tonyparisi.com 3/18/2015
CONTACT
tparisi@gmail.com
skype: auradeluxe
http://twitter.com/auradeluxe http://www.tonypa
risi.com/
http://www.learningwebgl.com/ MEETUPS
http://www.meetup.com/WebGL-Developers-Meetup/
http://www.meetup.com/Web-VR/
BOOK CODE
https://github.com/tparisi/WebGLBook
https://github.com/tparisi/Programming3DApplications
GET GLAM
http://www.glamjs.org/
https://github.com/tparisi/glam/
WORK
http://www.thirdeye.gl/
CREDS
Co-creator, VRML and X3D
GET THE BOOKS!
WebGL: Up and Running
http://www.amazon.com/dp/144932357X
Programming 3D Applications with HTML and WebGL
http://www.amazon.com/Programming-Applications-HTML5-WebGL-
Visualization/dp/1449362966

VR Everywhere March 2015

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (17)

More from Tony Parisi

More from Tony Parisi (20)

Recently uploaded

Recently uploaded (20)

VR Everywhere March 2015