Walter Greenleaf's presentation to the Virtual Medicine 2019 Conference at Cedars-Sinai Medical Center Immersive Environments, Machine Learning, Neuroimaging, And Wearable Sensing Technology - Treating Depression, Addictions, and Facilitating Behavior Change Using A Precision Medicine Model Based on the methods used in the ENGAGE Study Walter Greenleaf, PhD ​Virtual Human Interaction Lab | Stanford University Precision medicine models for treating depression, managing addictions and achieving sustained behavior change are largely outside of current clinical practice. Yet, changing self-regulatory behavior is fundamental to the self-management of complex lifestyle-related chronic conditions such as depression and substance use disorder - two top contributors to the global burden of disease and disability. To optimize treatments and address these burdens, methods to facilitate behavior change and self-regulation must be better understood in relation to their neurobiological underpinnings. Treatment strategies can then be developed that leverage the recent advances in immersive environments, machine learning, and wearable sensing technology and apply them to treat depression, manage addictions, and facilitate behavior change using a precision medicine model that is personalized to the individual. This presentation will review the conceptual framework and protocol for a large multi-subject longitudinal study named Project ENGAGE. The ENGAGE study integrates neuroscience with behavioral science to better understand the self-regulation related mechanisms of behavior change for improving mood and weight outcomes among adults with comorbid depression and obesity. We collect assays of three self-regulation targets (emotion, cognition, and self-reflection) in multiple settings: neuroimaging and behavioral lab-based measures, virtual reality, and passive smartphone sampling. By connecting human neuroscience and behavioral science in this manner within the ENGAGE study, we can develop a prototype for elucidating the underlying self-regulation mechanisms of behavior change outcomes and their application in optimizing intervention strategies for multiple chronic diseases. https://www.ncbi.nlm.nih.gov/pubmed/29074231 Behav Res Ther. 2018 Feb;101:58-70. doi: 10.1016/j.brat.2017.09.012. Epub 2017 Oct 7. The ENGAGE study: Integrating neuroimaging, virtual reality and smartphone sensing to understand self-regulation for managing depression and obesity in a precision medicine model. Leanne M. Williams, Adam Pines, Andrea N. Goldstein-Piekarski, Lisa G. Rosas, Monica Kullar, Matthew D. Sacchet, Olivier Gevaert, Jeremy Bailenson, Philip W. Lavori, Paul Dagum, Brian Wandell, Carlos Correa, Walter Greenleaf, Trisha Suppes, L. Michael Perry, Joshua M. Smyth, Megan A. Lewis, Elizabeth M. Venditti, Mark Snowden, Janine M. Simmons, Jun Ma

1. Walter Greenleaf PhD
Immersive Environments, Machine Learning,
Neuroimaging, & Wearable Sensing
Technology
Treating Depression, Addictions, and Facilitating
Behavior Change
Using A Precision Medicine Model
Presented at the 2019 Virtual Medicine Conference

2. Walter Greenleaf PhD
Treating Depression, Addictions, and Facilitating
Behavior Change
Using A Precision Medicine Model

3. The Stanford Laboratory for
Brain Health Innovation and
Entrepreneurship
VR-IT
Stanford Virtual Reality
Immersive Technology Clinic
My Academic Affiliations

4. My Non-Academic Affiliations

6. Healthcare Crisis:
Aging Populations
0
50
100
150
200
250
300
350
400
1900 1910 1920 1930 1940 1950 1960 1970 1980 1990 2000 2010 2015 2020 2025
Age 85+
Age 75-84
Age 65-74
Age 45-64
Age 5-44
Under Age 5

7. Transforming HealthCare with Technology

8. Digital Health
Revolution
• Mobile Health / eHealth
• Wearable Sensors
• Patient Centered
• Leverages Internet:
social, quantitative, collaborative

9. Every Medical Device Reinvented

10. Medical Applications
of Virtual Reality &
Augmented Reality
Technology
Digital Health Revolution

11. Now is the time for VR & AR
VR technology is now affordable,
scalable and accessible
Facebook - Oculus
Samsung - GearVR Sony – PlayStation VR
Microsoft - HoloLens
HTC - Vive Google - DayDream

13. • Prevention and Wellness
• Objective Assessments
• Functional Training
• Improved Interventions
• Facilitate Adherence
• Distributed Care Delivery
VR and AR technology will significantly impact Medical Care

14. My Journey….

15. VR Collaborators and Colleagues – Over the last 30 Years

16. Academic research has indicated that Virtual Reality can effectively treat a
wide variety of clinical problems – ranging from addictions, to stroke, to PTSD

17. Treating Post
Traumatic
Stress
With Virtual
Reality
2008 Sichuan
Earthquake

18. Treating Post
Traumatic
Stress
With Virtual
Reality
2017 Mexico City
Earthquake

19. Every Medical Device Reinvented

20. Entering the Era of Medical Wearables

21. Digital Health Platforms deliver interventions to patients, and
parse data for enhanced analysis and improved protocols

22. Digital systems are not just measuring the physical parameters of our health.
New methods are being developed that will allow us to collect and analyze biomarkers that
reflect our cognitive and emotional status.
Here are four examples…
Brain Health BioMarkers - Passive Data from Smart Phones
Voice Analytics as a BioMarker of Depression
Facial expressions in response to a VR Challenge
Cognitive function assessment using AR and smartphone sensors

23. Mindstrong tracks five digital biomarkers
associated with brain health: Executive
function, cognitive control, working memory,
processing speed, and emotional valence.
These biomarkers are generated from
patterns in smartphone use such as swipes,
taps, and other touchscreen activities, and
are scientifically validated to provide
measurements of cognition and mood.
Brain Health Biomarkers - Passive Data from Smart Phones

24. Ellipsis Health analyzes natural conversations using Deep Learning to
generate a real time emotional status vital sign.
The system provides a simple, cost effective way for providers, care
teams, and healthcare systems to continually screen their patient
population for depression, anxiety and other behavioral health issues.
Voice Analytics as a BioMarker of Depression

25. Facial expressions reflect emotional status
Collecting and analyzing emotional and physical
responses in VR

26. Cognitive function assessment at the primary
care level – using AR and smartphone
sensors
10 minute test
Diagnostic accuracy of 94%
FDA Class II Medical Device
Continuous data of everyday functions
250 features assessed at 300Hz
3D trajectory neuromotor parameters

28. Standardized Environments for Neurocognitive Evaluation
New Approaches for Cognitive Assessment
Migrates traditional paper
and subjective evaluations
to a more sophisticated
level.
Provides robust
assessments that can
challenge cognitive skills in
a more natural,
standardized, objective
and reproducible manner.

29. Standardized Environments for Neurocognitive Evaluation
New Approaches for Cognitive Assessment

30. Lea Williams - PanLab
Precision Psychiatry and Translational
Neuroscience

31. Using VR as an assay of brain circuits in action
Mindfulness-
based. TMS
New stratification: biotypes

32. Using VR as an assay of brain circuits in action
Mindfulness-
based. TMS
New stratification: biotypes
We focus on brain circuits within a new brain-based taxonomy
for depression. Specific dysfunctions in these circuits form
specific “biotypes” of depression and accompanying anxiety.
These biotypes may inform different types of interventions.

33. Using VR as an assay of brain circuits in action
Mindfulness-
based. TMS
New stratification: biotypesThe goal of the ENGAGE project is to identify assays that can assess
these biotypes in the natural world. We use lab-based imaging to assess
the biotypes in the lab setting, VR to create virtual environments that
reflect these lab-based circuits (a first step in moving to the natural world,
but in a still controlled way), and then use Mindstrong passive sampling to
assess the relations to natural world behaviors.

34. We use brain circuit biotypes to connect
brain imaging with VR
Neuroimaging measures
1. Scan brain at rest
2. Emotion regulation task
3. Cognitive control task
Virtual Reality:
1. Relaxing scene
2. Emotion regulation task
3. Cognitive control game

35. We envision a precision health model using VR to elicit
biotypes to help refine the intervention choices
Cognitive training TMS
Eliciting biotypes using
normed environmentsHeterogeneous
Disorder
Quantify and refine
biotypes
Relaxation Negative Scene
Positive Scene Cognitive Game
1st line Antidepressants
Mindfulness-based. TMS
Non-drug and
new drug options
Matching biotypes to the
intervention “menu”
Default Mode
Negative Affect
Positive Affect
Cognitive Control

36. Brain during cognitive control

37. NIH “ENGAGE” project
n=100 people scanned 5 times over 2 years
During a behavioral intervention for depression and weight management study
Phase
Baseline Repeat sampling and outcomes
Week 1 Week 8 Week 24 Week 52 Week 104
IMAGING of brain circuits X X X X X
VR-elicited regulation of brain circuits X X X X X
PASSIVE SAMPLING X ® ® ® ®

38. Jeremy
Bailenson, PhD,
Stanford
Jun Ma, MD PhD
Univ Illinois Chicago
Lisa Goldman
Rosas, PhD MPH,
Stanford
Mark Snowden, MD
, Univ Washington
Elizabeth Venditti,
PhD, Univ Pitts
William Haskell, PhD
Stanford
Megan Lewis,
PhD
RTI
Philip Lavori, PhD
Stanford
Lan Xiao, PhD
PAMFRI
Olivier Gevaert,
PhD, Stanford
Leanne Williams, PhD
Stanford
Joshua Smyth, PhD
Penn State Univ
Paul Dagum, MD
PhD, Mindstrong
Trisha Suppes,
MD PhD, Stanford
Brian Wandell,
PhD, Stanford
Walter
Greenleaf,
PhD, Stanford
The “ENGAGE” team
Williams PanLab for Precision Psychiatry and Translational Neuroscience
Co-PIs. Jun Ma and Leanne Williams
NIH Science of Behavior Change
“ENGAGE” Project: UH2HL132368.

39. Standardized platforms and norms
Williams et al. invited revisions
We are developing the first norms for VR to be used as
assays correlated with brain circuits
Lab-Based Virtual Reality Naturalistic
fMRI Head Movement 288 Continuous Smart-Phone
Sampling Variables
Behavioral task Performance Screen Capture
Self-Report Self-Report
Self-Regulation
Cognitive
Emotional
Self-Reflective

40. Standardized Environments for Neurocognitive Evaluation

41. Standardized Environments for Neurocognitive Evaluation
New Approaches for Cognitive Assessment

42. Digital Health Platforms deliver interventions to patients, and
parse data for enhanced analysis and improved protocols

43. Stroke and Traumatic Brain Injury
Physical / Occupational Therapy
New Approaches to Physical Medicine & Rehabilitation

44. VR & AR - A Key Part of a Combination Therapy
Digital Health Platform
Patient-facing software
designed to enhance
medication efficacy
Medication with
clinical benefit
eFormulation-
pharmaceutical
product with
enhanced efficacy
=+

45. A Fully Integrated, Closed Loop Solution for Mental Health and Wellness -
Enabled on a Mobile, Digital Health Platform
Miniaturized, programmable,
low power electronic
architecture w/extensive therapy
delivery options

46. Portable Telemedicine Platform

47. The Neuroscience of How VR Promotes Behavior Change

48. The Neuroscience of How VR Promotes
Behavior Change
VR can promote behavior change by
taking advantage of the way our brain’s
learning and reward systems function
Activate neuroplastic change via reward
systems
Shorten the reward feedback loop –
show progress
Leverage mirror neuron systems
VR systems can:

49. Neuroscience Rationale
Repetition is required
It is critical to engage the
brain's reward systems
It is necessary to activate the associated brain system
to enable neuroplasticity

50. Ability to change attitudes and behavior after “being” one’s future self.
Leveraging Mirror Neurons

51. Your Future Self
Students interacted with 3-D avatars of future self.
Participants who interacted with future self put more than twice
as much money into retirement account.

52. August 5th & 6th 2019
Denver
VR for Mental & Behavior Health Conference

53. Walter Greenleaf PhD
Immersive Environments, Machine Learning,
Neuroimaging, And Wearable Sensing Technology