Our paper presented at AMIA 2014 Annual Symposium. Paper is available at: http://www.knoesis.org/library/resource.php?id=2002 Citation: Ashutosh Jadhav, Amit Sheth, Jyotishman Pathak 'Analysis of Online Information Searching for Cardiovascular Diseases on a Consumer Health Information Portal', AMIA Annual Symposium 2014, Washington DC, November 15-19, 2014

1. Semantic Analysis of Online Health Information
Seeking for Cardiovascular Diseases
1
Ashutosh Jadhav
ashutosh@knoesis.org
AMIA 2014 Annual Symposium
Washington, DC

2. • Speaker discloses that he has no
relationships with commercial interests.
Disclosure

3. Collaborators
Prof. Amit Sheth (PhD Advisor)
Kno.e.sis Center, Wright State
University, OH, USA
Dr. Jyotishman Pathak (Mentor)
Mayo Clinic, Rochester, MN, USA

4. http://www.internetlivestats.com/internet-users/
Around 3 Billions (40%) of the
world population
Around 300 Million (87 %) of
the US population
4
Internet Users in the World

5. Online Health Information Seeking
5

6. Online Health Resources
6

7. Online Health Information Seeking
7
According to the Pew Survey, approximately 8 in 10 online
health inquiries initiate from a search engine.
Fox S, Duggan M. Pew Internet & American Life Project. 2013. Health online 2013

8. • According to Center for Disease Control and
Prevention, in the United States
– CVD is one of the most common chronic
diseases
– the leading cause of death (1 in every 4 deaths)
• CVD is common across all socioeconomic
groups and demographics
• Online health resources are “significant
information supplement” for the patients with
chronic conditions
8
Cardiovascular Diseases (CVD)
Use-case

9. Motivation
• Although cardiovascular diseases (CVD) affect a large
percentage of the population, few studies have
investigated what and how users search for CVD related
information online
• Such knowledge can be applied to improve the online
health search experience as well as to develop more
advanced next-generation knowledge and content
delivery systems

10. 10
Methods Overview

11. • Data:
– CVD related search queries
– Limited to United States
• Data timeframe:
– September 2011 to August 2013
• Data collection tool:
– IBM NetInsight On Demand
(Web Analytics tool)
• Dataset size:
– 10 million CVD related SQ
– Significantly large dataset for a
single class of diseases. 11
Dataset Creation

12. 12
Top CVD Search Queries
Top 1-5 Queries Top 6-10 Queries
heart attack symptom congestive heart failure
blood pressure chart low blood pressure
how to lower blood pressure stroke symptoms
heart rate normal blood pressure
broken heart syndrome high blood pressure symptoms

13. Health Categories
• Selected “14 consumer oriented” health categories,
representing health information needs
• Methods
– Focus group study (Published in JMIR)
– Online health information seeking literature
– Empirical data analysis
– Health categories on popular health websites
• The health categories and the classification scheme is reviewed and
validated by the Mayo Clinic clinicians and domain experts.
13

14. Health Categories
Health Categories Health Categories
1 Symptoms 8 Living with
2 Causes 9 Prevention
3 Risks & Complications 10 Side effects
4 Drugs and Medications 11 Medical devices
5 Treatments 12 Diseases and conditions
6 Tests and Diagnosis 13 Age-group References
7 Food and Diet 14 Vital signs
14
Drugs and Medications: tylenol raise blood pressure, ibuprofen heart rate,
dextromethorphan blood pressure, medications pulmonary hypertension,

15. Health Categories Example
15
Search Query Health Categories
Heart palpitations with headache Symptoms
Tylenol and blood pressure Medication, Vital sign
Pump for pulmonary
hypertension
Medical device,
Disease
Red wine heart disease Food, Disease
Bypass surgery Treatment

16. Classification: Possible Approaches
• Statistical Machine Learning algorithms
– Require training data
– For multiclass classification problem with 14 classes, we
need lot of training data
– Training data
• expensive to create as it should be created manually by
domain expert
• Coverage will be limited
– Does not consider semantics of queries
16

17. Domain Constraint
Classifier trained for one disease may
not work for other diseases as the
symptom, treatment, drugs and
medications varies by the diseases
17

18. Background Knowledge
• UMLS (Unified Medical Language System)
– Comprises over 1 million biomedical concepts and 5
million concept names
– Incorporates variety of medical vocabularies and concepts,
and maps each concept to semantic types
– Contains Consumer Health Vocabulary (CHV)
• Hair loss => Alopecia
– Quarterly updated with new concepts
18

19. Semantic
Analysis
• UMLS Semantic Type
– Example: symptom or sign, disease or syndrome
• UMLS Concepts
– Example: blood pressure, heart rate
• UMLS MetaMap
– Tool for recognizing UMLS concepts in the text
19

20. MetaMap Usage Challenge and Solution
20
Hadoop-MapReduce framework with 16 Nodes
Functional overview of a mapper

21. Gold Standard Dataset Creation
• Randomly selected 2000 search queries from the analysis
dataset.
• Two domain experts manually annotated 2000 search queries
by labeling one search query with zero, one, or more than
health category
• The annotators first discussed and agreed upon the annotation
scheme.
• To reduce the probability of human errors and subjectivity, the
two annotators discussed together and annotated each query
and created a gold standard dataset with 2000 search queries.
• The gold standard dataset is further divided into training and
testing dataset with 1000 search queries each.
21

22. 22
Health
Category
Categorization
Rule
Example
Drugs and
Medications
• ST:
ORCH|PHSU,
CLND, PHSU
• CC: medication,
medicine,
drugs, dose,
dosage, tablet,
pill
• KW: meds
• without CC:
alcohol,
caffeine, fruit,
prevent
• Tylenol raise
blood pressure
• Medications
pulmonary
hypertension
• ibuprofen heart
rate
• Dextromethorph
an blood
pressure

23. 23
Intent classes UMLS Semantic Types (ST), UMLS Concepts (CC) and Keywords (KW)
Symptoms ST: SOSY CC: symptoms, signs
Causes CC: cause, reason
Risks & Complications
CC: risk, complications
Drugs and Medications
ST: ORCH|PHSU, CLND, PHSU CC: medication, medicine, drugs, dose,
dosage, tablet, pill KW: meds (without CC: alcohol, caffeine, fruit, prevent)
Treatments
ST: TOPP, FTCN (treatment, surgery), CNCE (treatment), CC: remedy,
remediate (without CC: prevention and ‘Drugs and Medication’ queries)
Tests and Diagnosis
ST: DIAP, LBPR, LBTR CC: Test, diagnosis (without ST: DIAP| TOPP, CC:
alcohol, blood caffeine)
Food and Diet
ST: FOOD CC: caffeine, recipe, meal, menu, diet, eat, breakfast, lunch, dinner,
alcohol, drink
Living with
CC: control, manage, reduce, lower, coping, cure, recover KW: living with,
bring down, low down
Prevention CC: prevent, avoidance, low risk
Side effects CC: side effect KW: side effect
Medical devices ST: MEDD
Diseases and conditions ST: DSYN
Age-group References ST: AGGP
Vital signs
CC: blood pressure, heart rate, pulse rate, temperature, Heart beat, blood
glucose (without high/low blood pressure as we considered them under
‘Diseases and Conditions’)

24. Evaluation: Micro average
Precision Recall
• Classify 1000 search queries from the testing dataset
using the rule-based classifier
• Based on the evaluation, our classification approach has
very good Micro Average
– Precision: 0.8842,
– Recall: 0.8642
– and F-Score: 0.8723
24

25. Evaluations: Precision and Recall
Analysis for each Health Category
25

26. 26
Results
No Intent Classes Total Queries
Percentage
Distribution
1 Diseases 4,232,398 40.66
2 Vital signs 3,455,809 33.20
3 Symptoms 1,422,826 13.67
4 Living with 1,178,756 11.32
5 Treatments 955,701 9.18
6 Food and Diet 779,949 7.49
7 Med Devices 665,484 6.39
8 Drugs and Medications 603,905 5.80
9 Causes 599,895 5.76
10 Tests & Diagnosis 344,747 3.31
11 Risks and Complication 277,294 2.66
12 Prevention 136,428 1.31
13 Age-group References 87,929 0.84
14 Side effects 25,655 0.25
Total 10,408,921 100

27. 27
Results
8%
48%
40%
4%
0%
Distribution of search queries by number of
intent classes in which they are categorized
0
1
2
3
4 and 5

28. 28
Data Analysis Results

29. 29
• Average search query length for CVD is 3.88 words and 22.22 characters
• Around 80% of the CVD search queries have 3 or more words.
• CVD search queries are longer than previously reported non-medical as well
as medical queries
Data Analysis Results

30. Discussion and Conclusion
• We found that use of MetaMap and UMLS concepts/semantic type
to be a very good approach for customized health categorization
• The top searched health categories for CVD are ‘Diseases and
Conditions’, ‘Vital Sings’, ‘Symptoms’, and ‘Living with’.
• Most of the queries (around 88%) are categorized into either one
or two health categories.
• To the best of our knowledge, there is not much research on
understanding online health information searching for chronic
diseases and especially for CVD.
• This study addresses this knowledge gap and extends our
knowledge about online health information search behavior.

31. Thanks!
Ashutosh Jadhav
ashutosh@knoesis.org