Automotive LIDAR - OSRAM Presentation at Electronica 2018

Automotive LIDAR 2018
Lasers and Detectors: Requirements, Considerations and Emerging Trends for Automotive LIDAR
R. Thakur | electronica 2018 | Munich
Light is OSRAM
www.osram-os.com

Automotive LIDAR 2018 – OSRAM Opto Semiconductors| | R. Thakur
OSRAM | electronica 2018 | Munich
2
Agenda
Page
1. OSRAM Introduction 03
2. Human vs. Autonomous Driver 04
3. Requirements for AV/LIDAR derived from Human drivers 06
4. Range, Resolution and FOV Revisited 07
5. Data Throughput challenges 10
6. LIDAR Architectures – Selection Flowchart 13
7. LIDAR – House of Quality 14
8. Sensor Fusion Challenges and Architecture Proposal 16
9. Laser packages and Technology 18
10. Notable Market News and Crystal Ball 20

3
Global Market Leader in LED and Laser
1) At the end of the fiscal year 2017. 2) Countries where OSRAM had operations at the end of the fiscal year; OLED – Organic Light Emitting Device; AEB – Automatic Emergency Braking
Automotive Lighting
Matrix LED light
OLED rear light
Laser front light
Consumer
LIDAR – Infrared Lasers – AEB
Industry
26,400
Employees1
>120 countries
Worldwide Presence2
€4.1bn
Revenue1
General LightingGeneral Lighting

4
The Human Driver – Near Term AI Goal
Autonomous vehicles with all their sensors and AI software have the goal to replace the human
driver. Let us examine the Driving HW and SW of the human driver …
Sensor Parameter Human AV Winner Comment
Vision – Daytime
370 – 730 nm
Resolution/
# of Pixels
0.005°
576 MP1
0.04°
1.3 MP
Human Human eye can also distinguish 10 million colors;
foveal vision concentrates on 2° (7 MP)
Field of View 120°V x 175°H 28°V x 50°H Human With surround vision/multiple
camera’s, AV beats human
Dynamic Range 280 dB 110 dB Human Human eye has logarithmic sensitivity to brightness
Depth Perception Stereo Vision Mono Vision Human Human brain constructs a 3D image
of surrounding with depth estimation
Processing speed per frame 13 ms2 >50 ms Human Human brain processing speed increases
with complexity of information presented
Frame Rate >60 Hz 30 Hz Human Video games run at >100 Hz refresh
to ensure we see smooth motion
Vision –
Night (low/no visible light)
Low/No
visible light <0.02 cd/m2
Mono vision up to 10-6 cd/m2 CMOS image sensors poor
sensitivity
Human Both humans and CMOS visible camera’s
have poor response in darkness/low light
Vision – Infrared 850 – 950 nm Low/No stimuli >30% QE at 850 AV Machine vision with active NIR illumination
extends sensing beyond human eye capability
Hearing 20 – 20 k Hz Best @ 1 – 5 k Hz As needed AV AV can have multiple microphones which can
enrich information of the surroundings
Gesture Interact with surrounding
using gesture
Hands, eyes, voice, head Still innovating – light
and sound
Human
 AV
Currently humans can use many gestures to interact with
surrounding pedestrians and traffic. Expect AV to develop
corresponding gestures
1) http://art-sheep.com/the-resolution-of-the-human-eye-is-576-megapixels; 2) The Effects of Low Latency on Pointing and Steering Tasks – Sebastian Friston, et all ; ieee visualization and computer graphics, vol. 22, no. 5, may 2016

5
The Human Driver – Other Factors
Human drivers are often impaired by many factors … this is where AV can be clear winners
Factor Description Human AV Winner Comment
Emotion Anger/Road rage Susceptible NA
AV
AV do not get emotional, drowsy, tired and drunk/under influence
Drowsy/
Fatigue
Tired and sleepy Susceptible NA AV
Substance Abuse
Alcohol
Marijuana, …
Susceptible NA AV
Distraction Smartphone, billboards Susceptible NA AV AV process billboards like any other object (unlike humans)
Speeding Driving well above speed limit Susceptible NA AV
AV can keep safe distances, speeds and change lanes better
Tailgating Unsafe driving Susceptible NA AV
Age
Perception
and reaction
Gaussian Square Wave AV
Young and old adults are typically poor drivers;
the reliability of machines are predictable
SW Update Improving driving skills Developed over time
Over the
cloud update
AV
AV can learn from each other and each accident;
Human’s sometimes learn from own mistakes
Moral Compass Deer, Geese, Moose, Ducks, ... Susceptible NA Human
Humans likely to wait for a squad of geese
to cross … AV – depends on programmer
Local Driving Rules Driving rules vary by region Susceptible NA AV
Humans cannot easily drive in a new country ...
Unlike autonomous vehicles (SW stack update)
Courteous Driving Yielding your right occasionally Done often w/ gestures In Training Human
Human driving behavior is not easy to predict
or program (too many variables)

6
AV Performance Targets – Based on Human Driver
Need for environmental object recognition targets – Human vs Machine under boundary conditions …
Object Lighting Weather Vehicle Speed Human AV Sensor now Suggested AV Target ( >3 yr.) Comments
Pothole
(>2 feet wide)
Lowlight with
Low Beam
Headlights
Clear 75 mph 100 m
TBD
150 m
The objective here is to develop a
consensus on performance
requirements for sensors/driving –
in relation to human driver
The market will take a longer time to
agree on this list – without debate
and proposal from the eco-system
Heavy Rain 25 mph 50 m 75 m
Heavy Fog 25 mph 50 m 75 m
Brick
(8'' x 3.6''x 2.3'')
Clear 75 mph 50 m 75 m
Heavy Rain 25 mph 30 m 50 m
Heavy Fog 25 mph 25 m 50 m
Compact Car Heavy Snow 50 mph 50 m 75 m
... many more scenarios can be imagined
The market will take a long time to agree on this list – needs debate and proposal from the eco-system

7
System Latency  Range
1) Regression Based Model for Autosteering of a Car with Delayed Steering Response ; Vsevolod Nikulin, et al 2017 International Conference on Data Science and Advanced Analytics
2) The min value for wet roads is 3.7 m/s2 – German directive for road construction
Steering is
best option
Cannot
avoid crash
Braking is
best option
No
Yes
Prepare for crash
Continue
Brake
Steer to safety
Is projected vehicle trajectory
safe for next XX meters?
Calculate Time to Crash
Latency Assumptions
Steering system Latency1 = 400 ms
Sensor latency (Camera/RADAR/LIDAR) = 30 Hz  33 ms
Computational Latency (Algorithm/CPU) – Assume 50 ms
 Total Response latency ~500 ms
Scenario 1
Straight dry road, clear weather | Two vehicles approach each other at max. speed
• Closing Speed = 100 mph x 2  ~45 m/s x 2 = 90 m/s
• Minimum Range needed = 90 m/s x 0.5 sec = 45 m
(One car moves over to its legal side)
• Question: Is 500 ms latency adequate?
How far can the best human driver see while driving ?
Scenario 2
Straight icy road2 | Vehicle at max speed approaches stationary object
• Min Range = (45 m/s)2 / 2 x 3.7(m/s2) ~274 m
• Question: Would a good human driver drive 100 mph on icy roads?
Scenario 1 and 2 are likely edge cases. Real need is somewhere in between (45 – 274 m).
The market will naturally move towards the maximum range that the consumer will pay for – over time.
In other words, anything over what is really needed is waste/will need to give away for free

8
Resolution
What is the smallest object that must be detected to drive safely?
1) Bosch Multi Purpose Camera (MPC2), 1,280 x 960 pixels, 50° HFOV, 28° VFOV; 2) Velodyne VLP16 (0.1° – 0.4°); 3) RADAR equation
1.5 m 0.25 m 0.4 m 0.2 m 0.1 m
• Standard object list for detection does not exist (ignore/standardize with risk)
• LIDAR is capable of <0.3° resolution at >100 m (with small form factor)
• RADAR size for 0.5° resolution not practical (~0.5 m for 76 GHz RADAR)
• Camera range needs to improve and image quality in lowlight (or infrared)
SVGA resolution (800 x 600) at <100 meters maybe enough for
forward LIDAR (Much better than human eye in poor lighting conditions)
Who decides?
Ignore objects at own risk ...
• 1° @ 100 m  1.7 m
• 1° @ 200 m  3.4 m
• 0.1° @ 200 m  0.4 m
Resolution Size (m)
• Camera1  25 pixels/°
• LIDAR2  0.3°
• RADAR3  2.6°
(76 GHz, 10 cm aperture)
Typical Angular Resolution
θ:
λ: Wavelength
w: Aperture width
65λ
w

9
Pictures: istockphoto
Azimuth and Elevation
Field of View – What is really needed?
140° Azimuth and 80° Elevation seem to be the upper limit for current LIDAR requirements
Traffic Lights and Overhead
Signs Need High FOV
Up and Down Ramps
Need High FOV
Winding Roads
Need Wide FOV

10
Data Throughput Challenges
Cloud points of Data for 1 Long Range LIDAR  ~288 GFLOPS
• Assumptions: Azimuth = 80°, Elevation = 60°, Range = 200 m,
Angular Resolution = 0.1°, Linear resolution = 1 cm (0.01 m), Frame rate = 30 Hz
• Number of points per frame = 80/0.1 x 60/0.1 = 480,000
• Number of frames in 1 data set = 200/0.01 = 20,000
• Number of cloud points in 1 data set = 480,000 x 20,000 = 9.6 x 109
• Number of cloud points per Sec = 9.6 x 109 x 30 = 288 x 109 (~288 GFLOPS)
Cloud points of Data for 1 Medium Range LIDAR  ~324 GFLOPS
Assumptions: Azimuth = 180°, Elevation = 60°, Range = 100 m,
Angular Resolution = 0.1°, Linear resolution = 1 cm (0.01 m), Frame rate = 30 Hz
Data Throughput Rate needed
• 1 long and 4 Medium range LIDAR  288 + 324 x 4 = 1584 GFLOPS
• For perfect overlap, assume RADAR and Camera needs same amount
• Data throughput = 3 x 1,584 = 4,752 GFLOPS  ~5 TOPS (Trillion Operations per Sec)
• Data processing  Points  Clustered into lines/curves/volumes  Objects with velocity
• At least 5 additional operations with each point before objects are identified and referenced into environment
• 5 TOPS x 5 = 25 TOPS is ~compute speed needed with above assumptions
• Nvidia Xavier (Volta) – 20 TOPS … getting close what is needed …

11
Data Throughput Challenges
• Fastest known rate for data transmission –
Fast Ethernet – 1000 GB/sec
• Protocol used – point to point,
802.3 full duplex Ethernet Technology
Best Case Time Budget –
0.32 sec (Sensor  CPU) + 1.2 sec CPU Processing
+ 1 sec Steering Latency = 2.5 sec
• Max LIDAR Range Needed = 90 m/s closing speed x 2.5 sec = 225 m
Needed/cost effective? … TBD
CPU for AV (1.2
Sec) (20 TOPS
Capacity Need
25 TOPS)
4 Medium Range
LIDAR
Long Range
LIDAR
4
RADAR 4 Cameras
Steering
and Braking
Actuators
1 – 2 Hz (Natural
Frequency for most
Cars; Faster control
affects comfort)
0.29 Sec @
1TFLOP/Sec
0.32 Sec@1TFLOP/Sec
0.32 Sec @
1TFLOP/Sec
0.03 Sec
@ 1TFLOP/Sec
0.5 to 1 sec delay
288 GFLOP 324 GFLOP/LIDAR
33 GFLOP/
RADAR
324 GFLOP/
Camera

12
Data Throughput Challenges – Take away
• Data transmission and processing delays are significant
for consideration in sensor fusion architecture
• Multiple fast Ethernet lines maybe needed from sensors to CPU
• Central processing with capabilities >25 TOPS maybe needed (Trillion Operations per sec)
• Minimum response latency for reliable sensing and processing is ~2.5 sec
• Vehicle natural frequency (for comfortable ride) contributes ~1 sec. latency
• A reduced vehicle speed assumption(operation design domain) is next big contributor to LIDAR range
• Vehicle Scenario corner cases (2 vehicles approaching at 100 mph)
should be filtered for probability and clipping (AV operation not allowed)
• The cost of computation is significant ($750 at $0.03/GFLOP1; 25 TFLOPS needed)
• The power consumption is significant ( ~1,563 W @ 16 GFLOPS/W2)
• …
Defining the operational domain for the AV (Max speed, lighting) pays high dividends!

13
LIDAR Architecture Options
Flowchart biased to 905 nm … For cost, availability, power consumption and detector options
Start
Reflectivity <5%
Laser Wavelength –
1550 nm OR Detector
with very high gain (SiPM)
Achievable with 905 nm Lasers
Is FOV >140° ?
Mechanical
Rotation Scanner
Matched pair of
laser and detector
Is Resolution <0.5°?
905 nm
Laser
Is range <100 m?
1. Laser Line Scan and APD Array
2. High resolution Flash LIDAR
Low Resolution
Flash LIDAR
GET range, FOV,
resolution,
target reflectivity
Range >200 m?
Laser dot Scan and
Detector (array)
Scanner Options
Optical Phase Array
Liquid Crystal
MEMS
Mechanical
Detector Options
APD
SiPM
Yes
YesYes
Yes
Yes
No
No
NoNo

14
LIDAR – House of Quality
The analysis above is subjective and continuously changing with sensor improvements!
Laser
wavelength
Detector
choice
Scanning
technic
Beamoptics
Laserdriver
LaserPeak
Power
Signal
Processing
SmallForm
factor
6 6 4 2 4
Maturity (10 High  1 Low)
Customer
Requirement
PriorityWt
(1–5)
Correlation (-3 Low  +3 High)
Solid-State
Mechanical
MEMS
OPA
ToFCamera
Comments
Eye Safe –
Class 1
5 3 3 -3 Given; Who certifies?
Long
Range
(>200 m,
10% refl.)
5 3 3 2 3 3 2 -3 + ++ + + -- Regulation/
Sensor architecture/
Autonomy level
High
resolution
(~0.1°)
4 3 3 3 ++ + ++ ++ ++ Road conditions/
weather/what is
limiting object
(size/reflectivity)
Large FOV
(±40° V,
±20° H)
3 3 3 3 3 -2 + ++ + + + Bigger problem than
range/resolution; with
GPS and 3D Maps?
Low Cost
($250)
4 -2 -2 -3 -2 --- -- + + - Consider cost of com-
plete sensor suite (8
cameras vs 2 LIDAR?)
Solid-State
Array of Lasers and Detectors
Mechanical
Motor/Vibration Micro-Electro
Mechanical mirror for scanning
OPA
Optical Phased Array for
Scanning Time of Flight
Camera: Range gated

15
Sensor Fusion Challenges –
Need for overlapping sensor technologies
Object_list RADAR Camera LIDAR
Sensor
Fusion
Car @150 m
Don’t see it
(Noise)
Not_Classified
@ 100 m and
low light
Evaluate TTC
and brake if
unresolved ?
@ 50 m
Person on
bicycle
Not classified
Don’t see it
(Noise)
Brake or
ignore?
Potholes
and stuff
What can be
safely ignored?
• Object Identification and Classification in range
and FOV of interest must be comparable
• LIDAR + Camera fusion potentially better
(Due to angular resolution)
• Camera improvements – Range (~70 m);
speed (30 – 60 Hz) and low light sensitivity
Objective of Sensor Fusion – Determine environment around vehicle trajectory
with enough resolution, confidence and speed – to navigate efficiently

16
Sensor Fusion Strategy Example
Range Velocity GPS 3D Map Visibility Object
HFOV
Object
VFOV
Object
Size
Object
Re-
flectivity
Prime
Sensor
1
Prime
Sensor
2
Comments
>200 m <75mph No Yes Poor <30° <30° >1 M <10% RADAR LIDAR
Highway; Camera out of range; RADAR
better under poor visibility: large Object
<75 m <35mph Yes Yes Good >30° >30° <1 M >50% Camera LIDAR
City; Short Range; Small Object;
Good Visibility; Ideal for Camera
<75 m <35mph Yes No Poor >30° >30° >1 M >50% Camera LIDAR
City; Short Range; Big Object;
Poor Visibility
...
Typical Sensor Maps
not suitable for fusion!
A holistic environment sensing strategy is often missing (or not shared) …
• Important to rely on more than one sensor technology
• Prime sensors must be able to independently
classify object robustly under pole conditions
(Better than R95C95)
• When fusion results do not yield high confidence,
then warn/change Levels/evade/prepare for collision!
Sensor
Spec

17
AV Sensor Packages and Architecture Proposal
(1X) Long Range LIDAR
200 m, 10% Reflectivity
80° Azimuth x 0.1° Res.
Range and FOV
overlaps RADAR
<10 meter dark zone
(No LIDAR/RADAR/
Ultrasonic/Visible Camera
in low light conditions)
(4X) Medium Range LIDAR
100 m, 10% Reflectivity
180° Azimuth x 0.1° Res.
Range and FOV
overlaps Camera
(4X) Medium Range RADAR
(FOV not representative)
Camera works with Visible
light in Daytime and Near
infrared at Night

18
905 nm Laser Diodes for LIDAR –
OSRAM Product Portfolio and concepts
In Mass Production 2018 – Improved Chip … and Future concepts
SPL LL90_3 Hybrid pulse laser
with integrated driver circuit
Peak Power 75 W @ 30 A; Pulse length >20 ns)
SPL DS90_3 Chip
75W peak power @ 30 A
SPL DS90A_3 Bare Die
120 W peak power @ 40 A
SPL SxL90A_3 A01
Integrated SMD Solutions
1/4 channel laser
SMT Package
SPL PL90_3 Pulse laser
Peak Power 75 W @ 30 A
Short pulses (FWHM 4 ns demonstrated)
SPL UL 90_A T08 Pulse laser
Peak Power 120 W @ 40 A
Multi-channel Bars concepts
Scalable power:
Peak Power per channel: 120 W @ 40 A

19
VCSEL Technology
Advantages
• Surface mount, chip-on-board packaging
• 1D and 2D arrays
• Circular, low divergence beam
• High modulation rate
• Narrow spectrum
• Low Wavelength shift with temperature
Challenges
• Automotive qualification
• Higher power in small form factor (>100 W)
VCSEL – Vertical Cavity Surface Emitting Laser

20
LIDAR and Market News
• Lumotive – Lidar using solid-state beam steering based on a liquid crystal metasurface (LCM) chip
• Cepton – Micromotion scanning with multiple lasers
• Vergence Automation – “4D Camera” using CMOS technology –
Provides lighting invariant video where each pixel has X, Y, Z and intensity
Technology processes all pixels simultaneously (Throughput advantages)
• Many OEM’s have taken the path of designing their own LIDAR –
Since the market is often not meeting their requirements for performance and price
• Multiple channel lasers provide the flexibility of more power and wider field of view – Increased interest in market
• Techniques to reduce spectral width of high power multi-mode lasers are gaining traction. APD detectors still rule
• Development focus shifting to medium range LIDAR (<100 m)

21
Summary and Crystal Ball
Requirements
• Humans have superior visual processing, communication and response in daylight
• Emotion/Age/distractions/fatigue/poor night vision are areas where AV sensors and processing are more reliable
• Using a Human driver to derive basic requirements for sensing and processing helps to measure our progress in AV technology
• There is a need for consensus on sensing and performance targets to enable faster development of AV technology
Technology
• Data transmission/throughput has significant impact on AV sensing architecture
(Need Fast Ethernet with >1000 GB/sec; CPU with >25 TOPS)
• High power multi-channel lasers, VCSEL showing high interest in market
Architectures
• One long range LIDAR (200 m) and 3 – 4 Medium range (100 m) which together cover all sides seem to be market direction
• Many OEM’s resigned to developing own LIDAR for faster development and control
Price
Low cost automotive LIDAR (<100$) is not in sight yet. New target is ~300$ in 3 years
LIDAR continues to retain high interest and innovation. No clear winner in sight, market continues to scan!

22
Many Thanks.

Automotive LIDAR - OSRAM Presentation at Electronica 2018

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Automotive LIDAR - OSRAM Presentation at Electronica 2018

Similar to Automotive LIDAR - OSRAM Presentation at Electronica 2018 (20)

Recently uploaded

Recently uploaded (20)

Automotive LIDAR - OSRAM Presentation at Electronica 2018