Price, S.L. (2017, April). Refining a self-report instrument for measuring subjective emptiness. Oral presentation at the 24th Annual Psychology Research Fair, California State University San Marcos, San Marcos, CA.

1. Refining A Self-Report Instrument
for Measuring Subjective Emptiness
Stephanie L. Price1, Christopher J. Hopwood2, Heike I.M. Mahler1, Kristyana Kern1, Tyan Carrico1
1California State University San Marcos
2Michigan State University

2. Subjective Emptiness
Construct definition:
oUnwillingly drifting through life,
detached and hollow inside.
Experienced by:
oPsychiatric patients
oGeneral population
Complications:
oSubstance use
oSelf-injury
oSuicide

3. Problem
Difficult to define
and measure
Lack of research
and accessible
treatment

4. Content Validity
Testing
Refinement
Pilot Testing
Qualitative Phase
Validation
Quantitative Phase
Item
Generation

5. Content Validity
Testing
Refinement
Pilot Testing
Qualitative Phase
Validation
Quantitative Phase
Item
Generation
• Literature
review
• Exemplar
quotes
• In-depth
interviews
(n = 18)

6. Content Validity
Testing
Refinement
Pilot Testing
Qualitative Phase
Validation
Quantitative Phase
Item
Generation
• Expert panel
(n = 15)
• Interviewee
panel (n = 10)
• Content
Validity Ratio

7. Content Validity
Testing
Refinement
Pilot Testing
Qualitative Phase
Validation
Quantitative Phase
Item
Generation

8. Aim 1. Develop a unidimensional cohesive scale
Aim 2. Assess if items measure the focal construct
Aim 3. Evaluate how well items measure construct
Aims of the Refinement Study

9. Participants

10. • Subjective Emptiness Scale (SES)
Measures

11. • Center for Epidemiologic Studies Short Depression Scale (α = .83)
• Zanarini Rating Scale for Borderline Personality Disorder (α = .82)
• Purpose in Life test short form (α = .86)
• Barratt Impulsiveness Scale–Brief (α = .84)
Measures (Con.)

12. • Recruitment
oSampling
▪ Convenience
▪ Referral
oChannels
▪ Blogs
▪ Social media
▪ Websites
• Survey website
oConsent
oQuestionnaires
oDebrief
oNo compensation
Procedure

13. • Analyses
• Principle components extraction
• Constrained to 1 factor
• Item selection criteria
• Strong factor loadings: over .40
Aim 1: Dimensionality and Internal Consistency

14. • Analyses
• Confirmatory factor analysis
• Maximum likelihood estimation
• Reliability analysis
• Cronbach’s alpha
• Item selection criteria
• Goodness of Fit
• GFI > .80
• RMSEA < .09
• NFI > .90
• CFI > .90
• TLI > .90
• AGFI > .80
• Local independence
• Standardized covariances among residuals below ꟾ2.58ꟾ
• α = .70-.95
Aim 1: Dimensionality and Internal Consistency (Con.)
Figure 1. Path model of best fitting solution

15. • Analyses
• Item-criterion correlations
• Pearson's Product-Moment Correlation
• Item selection criteria
• Depression (+)
• Borderline personality disorder (+)
• Emptiness criterion (+)
• Meaning in life (-)
• Impulsivity (±)
Aim 2: Validity
Figure 2. Hypothesized nomological network
Emptiness
Meaning
Impulsivity
Depression
BPD

16. • Analyses
• Item level descriptive statistics
• Kurtosis
• Skew
• Mean
• Standard deviation - lower
• Flesch–Kincaid grade level
• Item-total correlation
• Item selection criteria
• 4th grade reading level
• Item-total correlation coefficients: .40-.90
Aim 3: Item Performance

17. Aim 3: Item Distribution Criteria
Student Sample
1 2 3 4
Clinical Sample
1 2 3 4
Figure 3. Desired item residual distributions

18. Aim 1: Results – Dimensionality and Internal Consistency
Table 2. Confirmatory factor analysis fit indices and reliability
Clinical Sample Student Sample
GFI .94 .93
RMSEA .09 .09
NFI .95 .94
CFI .95 .96
TLI .94 .94
AGFI .90 .90
Cronbach’s α .93 .93

19. Aim 2: Results – Validity
Table 3. Item-criterion correlation coefficients
Clinical Sample Student Sample
r r
Depression .43-.63 .46-.66
Borderline personality disorder .41-.58 .48-.62
Emptiness item .50-.76 .38-.66
Meaning in Life -.48-.58 -.34-.51
Impulsivity .13-.22 .20-.29
Note: All correlations were significant, p < .05.

20. Aim 3: Results - Item Performance
Table 4. Item descriptive statistics and reading level
Clinical Sample Student Sample
Skew -0.75-0.54 1.32-2.76
Mean 2.12-3.10 1.27-1.58
SD 1.02-1.23 0.61-0.85
Item-total correlations 0.62-0.83 0.67-0.81
Flesch–Kincaid grade level range 0.1 – 3.7

21. Aim 1. Develop a unidimensional cohesive scale
Aim 2. Assess if items measure the focal construct
Aim 3. Evaluate how well items measure construct
• Potential applications
oEmpirical
oClinical
▪ Assessment
▪ Treatment efficacy
Discussion

22. • Limitations
• Low completion rate (~48%)
oOptimize survey length and design
Next steps
oDifferential item functioning (IRT)
oValidation study
Limitations and Next Steps

23. Acknowledgements
• RISE grant (GM-64783)
• Research advisors
o Dr. Heike Mahler
o Dr. Marie Thomas
o Dr. Christopher Hopwood
• Research assistants
o Kristyana Kern
o Tyan Carrico
• Expert panel
• Recruitment collaborators

24. • Emptiness is associated with high-risk complications.
• Difficulty defining and measuring emptiness has resulted in a
lack of research and treatment.
• The Subjective Emptiness Scale is a promising instrument for
potentially measuring emptiness in research and clinical
settings.
Key Points

26. Final 10-Item Scale
• I feel empty inside.
• I feel hollow.
• I am living a life that is drained of emotion.
• I feel as though I am disconnected from the world.
• I am in the world but not part of it.
• I feel all alone in the world.
• I feel absent in my own life.
• I feel like I am forced to exist.
• No matter what I do, I still feel unfulfilled.
• My life does not feel as meaningful as I would like it to feel.

27. 1. I have a void inside of me.
2. I feel hollow.
3. I feel empty inside.
4. I feel like I'm lacking whatever it is that makes everyone else human.
5. I am missing something that everyone else has that makes them whole.
6. I am lacking whatever it is that makes people feel alive.
7. I observe life happening around me yet I do not feel like I am part of it.
8. I am in the world but not part of it.
9. I feel absent in my own life.
10. I feel all alone in the world.
11. I am drifting through life without any sense of direction.
12. I go through the motions of living.
13. I feel like I am forced to exist.
14. I don't see the point in doing anything.
Refined Item Pool

28. 1.John Peteet MD – Harvard Medical School
2.Mark Zimmerman MD - Brown University, Rhode Island Hospital
3.Clive Hazell PhD – Private Practice Counseling Psychologist
4.Marilyn Charles PhD – Austen Riggs Center
5.Nancy Richman PhD - Volunteers in Medicine Clinic of Monroe County
6.Kristian Markon PhD – University of Iowa
7.Lee Anna Clark PhD – Notre Dame University
8.Bill Ellison PhD – Trinity University
9.Nina Brown PhD – Old Dominion University
10.Jennifer Cheavens PhD – Ohio State University
11.Katherine Dixon-Gordon PhD - University of Massachusetts
12.Stephanie Mullins-Sweatt PhD – Oklahoma State University
13. Charles Wang MD - Private Practice Psychiatrist, Cascade Behavioral Hospital
14. Melvin Singer MD - Private Practice Psychiatrist
15. Yolanda Rosillo PhD - Psychologist and Psychotherapist based in Italy
Subject Matter Experts

30. • 22% of the items were rated as Essential
by a majority of the expert panelists
• 11% of the items were rated as Essential
by half of the expert panelists
22%
11%
67%
Majority
50/50
Minority
Results – Content Validity Study