- What is SLAM (Simultaneous Localization And Mapping)? - The use cases of SLAM? - How does it work? - ARVR and SLAM? - AI and data science - Hardware and sensors

1. 1
SLAM aware applications
by Kuldeep Singh, Raju Kandaswamy, ARVR Practice
© 2019 ThoughtWorks

2. Know your
surroundings
What do you see around you?
2© 2019 ThoughtWorks
Map the area!

3. Know your location
Where are you?
3© 2019 ThoughtWorks
Your location in the map

4. Table of Content
● SLAM Overview
● Approaches of SLAM
● Usage of SLAM
● ARVR & SLAM
4© 2019 ThoughtWorks

5. SLAM OVERVIEW
5© 2019 ThoughtWorks

6. Simultaneous
Localization And
Mapping(SLAM)
SLAM technology knows the agents
location in the surrounding and at same
time learn the map.
SLAM aware applications have access to
location and the map.
KEEP TRACK OF THE MAP AND LOCATION
6© 2019 ThoughtWorks

7. 7

8. SLAM aware apps -
Crawl and Learn
Crawl thru the sea objects, learn and
recognize them:
- Self driving car
- Indoor positioning
- Augmented Reality Spatial
computing
Search engine of the physical world
8© 2019 ThoughtWorks

9. SLAM is an estimation
technique
SLAM is the technique to simultaneously
do following estimations :
● Pose estimation - Determining the
position and orientation of the agent
(camera) relative to the object (or
vice-versa).
● Map estimation - Estimate the
locations of the objects in the
environment.
9© 2019 ThoughtWorks

10. Why SLAM is difficult?
10© 2019 ThoughtWorks
● Pose estimation - Needs object’s
locations in the environment i.e Map.
● Map estimation - Needs information
on agent pose (camera pose) to
estimate object’s position.

11. Approaches of SLAM
11© 2019 ThoughtWorks

12. SLAM problem definition
12
SLAM is a process by which a mobile
device can build a map of an
environment and at the same time
use this map to deduce it’s location.
In SLAM both the trajectory of the
platform and the location of all
landmarks are estimated real time
without the need for any a prior
knowledge of location.

13. SLAM approaches
● EKF (Extended Kalman Filter) SLAM
● FastSLAM
● L-SLAM (Matlab code)
● GraphSLAM
● Parallel Tracking and Mapping
(PTAM)
● MonoSLAM
● CoSLAM
● SeqSlam
● iSAM (Incremental Smoothing and
Mapping)
● CT-SLAM (Continuous Time)
From mid eighties to now, a variety of approaches are there, solving it to a certain approximation
13
Open Source
● LSD-SLAM
● S-PTAM
● ORB-SLAM
● Google Cartographer
www.openslam.org

14. A deep dive one approach
Basic Functional EKF-SLAM Pseudo Code
14
%INITIALIZATION
initialize_map()
While (execution() == true) do
control = acquire control signal()
move device(control)
% LOOP SENSORS IN DEVICE
for each sensor in device−>list of sensors
raw = sensor−>acquire raw data()
% LOOP OBSERVATIONS IN EACH SENSOR
for each observation in sensor−>feasible observations()
% MEASURE LANDMARK AND CORRECT MAP
measurement = find known feature(raw, observation)
update map(sensor, landmark, observation, measurement)
end
% DISCOVER NEW LANDMARKS WITH THE CURRENT SENSOR
measurement = detect new feature(raw)
% INITIALIZE LANDMARK
landmark = init new landmark(robot, sensor, measurement)
create new observation(sensor, landmark)
end
end

15. Usage of SLAM
15© 2019 ThoughtWorks

16. Usage of SLAM
16© 2019 ThoughtWorks
Autonomous Vehicle - Space, Air, Earth, under the earth, under water

17. Usage of SLAM
17© 2019 ThoughtWorks
Autonomous Vehicle - vacuum cleaner, lawn mower, automated warehouse, robots

18. Usage of SLAM
18© 2019 ThoughtWorks
Augmented Reality & Virtual Reality

19. ARVR & SLAM
19© 2019 ThoughtWorks

20. 20© 2019 ThoughtWorks
● Development Tools
○ Android (NDK)
○ ARCore SDK
○ Unity
○ Vuforia
○ Wikitude
○ Unreal
● SLAM
○ Concurrent odometry
and mapping
● Sensors - IMU
● Features
○ Motion tracking
○ Light estimation
○ Feature points
○ Cloud Anchors
Android Phone and Tablets iOS Phone and Tablets
● Development Tools
○ ARKit SDK
○ Unity
○ Vuforia
○ Wikitude
○ Unreal
● SLAM
○ Visual Inertial Odometry
● Sensors - IMU
● Features
○ Motion tracking
○ People Occlusion
○ Multi camera, face
tracking

21. Head Mounted Display - Mixed Reality
21© 2019 ThoughtWorks
Microsoft HoloLens
● Development Tools
○ Universal Windows Platform (UWP)
○ Mixed Reality ToolKit (MRTK)
○ DirectX custom engine
○ Unity
○ Unreal
● SLAM
○ Custom PTAM
● Sensors
○ IMU( Accelerometer, Gyroscope)
● Features
○ Motion tracking
○ Spatial mapping and sound
○ Scene detection and storage

22. Head Mounted Display - Mixed Reality
22© 2019 ThoughtWorks
Magic Leap One
● Development Tools
○ Lumin SDK
○ Unity
○ Unreal
● SLAM - Deep CNN
○ Magic Point
○ MagicWarp
● Sensors
○ Lightwear IMU( Accelerometer, Gyroscope)
○ EM tracking
○ Controller IMU
● Features
○ Motion tracking
○ Spatial mapping

23. Head Mounted Display - Virtual Reality
23© 2019 ThoughtWorks
Oculus Quest
● Development Tools - Android
○ Oculus Quest Native SDK
○ vrAPI (Native Engine)
○ Unity
○ Unreal
● SLAM - Oculus Insight
○ LSD-SLAM: Large-scale direct monocular SLAM
○ ORB-SLAM
● Sensors
○ Multiple IMU( Accelerometer, Gyroscope)
○ Infrared
○ Camera

24. Way to go...
24© 2019 ThoughtWorks
● Web based SLAM is still to be explored.
● Visual SLAM has a lot of environmental limitations
● AI advancements will take the journey further
● Small footprint algorithm are being built to run on small devices
Visit Us @ ARVR Stall

25. THANKYOU
25
RAJU KANDASWAMY
ARVR Practice
raju.kandaswamy@thoughtworks.com | thoughtworks.com
© 2019 ThoughtWorks
KULDEEP SINGH
ARVR Practice
kuldeeps@thoughtworks.com | thoughtworks.com