Summary of EOSDIS User Survey

1. 2011 ACSI Survey Summary
HDF/HDF-EOS Workshop
Riverdale, MD
April 18, 2012

2. Project Background
Measurement timetable
Finalized questionnaire
August 1, 2011
Data collection via web
September 12, 2011 –
October 18, 2011
Sending invitations spanned the first two weeks.
Sending reminders spanned the last two weeks. The
survey was in the field for a longer time this year for
resending invitations.
Topline results
October 26, 2011
Results briefing
November 29, 2011

3. Project Background
Those who
answered for
more than one
data center:
Two: 103
Three: 14
Data collection
Respondents
• 3,996 responses were received
• 3,996 responses were used for modeling
Four: 2
Data Center
Description
Original
ASDC–LaRC
ASF SAR DAAC
CDDIS
GES DISC
GHRC
LP DAAC
MODAPS LAADS
NSIDC DAAC
OBPG/Ocean Color
ORNL DAAC/FLUXNET
PO.DAAC-JPL
SEDAC
Total
2350
1371
1302
1551
678
25503
6939
5487
4893
3988
1361
2728
58151
NASA Survey Responses
Emailed a
Cleaned
Survey
Invitation
2350
2349
1370
1364
1275
1271
1544
1533
674
670
25490
25475
6839
6805
5487
5468
4893
4891
3976
3966
1352
1348
2728
2724
57978
57864
Bounce Backs
Responded
Response Rate
135
108
468
357
81
1477
482
619
721
197
103
148
4896
194
172
95
97
69
1849
484
398
200
229
85
124
3996
9%
14%
12%
8%
12%
8%
8%
8%
5%
6%
7%
5%
8%
E-mail addresses from lists associated with some of the data centers were included to reach the large number of users who may
have accessed data via anonymous ftp.

4. NASA EOSDIS Benchmarks
Strong performance continues …
ACSI (Overall) Q2 2011
76
65
Federal Government (Overall) 2010
NASA EOSDIS - Aggregate 2011
77
News & Information Sites
(Public Sector) 2011
75
30
40
50
60
70
ACSI (Overall) is updated on a quarterly basis, with specific industries/sectors measured annually.
Federal Government (Overall) is updated on an annual basis and data collection is done in Q3.
Quarterly scores are based on a calendar timeframe: Q1- Jan through March; Q2 – April through June;
Q3 – July through Sept.; Q4 – Oct. through Dec.
80

5. NASA EOSDIS
Customer satisfaction remains steady
N=1016
N=2857
N=2291
2004
2005
2006
2007
75
78
74
(+/-) 0.9
ACSI
N=1263
(+/-) 0.7
79
N=2601
N=3842
N=4390
N=3996
2008
2009
2010
2011
75
77
77
77
77
(+/-) 0.5
(+/-) 0.6
(+/-) 0.5
(+/-) 0.4
(+/-) 0.4
(+/-) 0.4
82
78
80
81
81
81
81
73
73
71
73
74
73
74
74
71
76
72
73
75
75
75
75
Overall satisfaction
How satisfied are you with the
data products and services
provided by [DAAC]?
Expectations
To what extent have data products
and services provided by [DAAC]
fallen short of or exceeded
expectations?
Ideal
How close does [DAAC] come
to the ideal organization?

6. NASA EOSDIS Model
Product Search/Selection/Documentation most critical
Customer
Support
Product
Documentation
Product
Selection and
Order
86
1.7
76
0.9
87
77
1.1
75
Product Search
0.9
77
Recommend
3.8
Customer
Satisfaction
Index
89
Future Use
3.2
78
Product Quality
0.4
81
Sample Size: 3996
Delivery
0.4
Scores
The performance of each component on a 0 to 100 scale. Component scores are made
up of the weighted average of the corresponding survey questions.
Impacts
The change in target variable that results from a five point change in a component score.
For example, a 5-point gain in Product Search would yield a 0.9-point improvement in Satisfaction.

7. User background and interests
User background and interests
questions
questions
Have you
Have you
searched,
searched,
ordered,
ordered,
downloaded
downloaded
data?
data?
Search questions
Search questions
2011 EOSDIS Survey Overview
no
Did you look for
Did you look for
or get
or get
documentation
documentation
??
Delivery
Delivery
questions
questions
no
Documentation
Documentation
questions
questions
Did not search
Rate
Rate
search
search
Did not order
Rate
Rate
delivery
delivery
Format
Format
questions
questions
Order questions
Order questions
Rate
Rate
format
format
Rate
Rate
order
order
Usage
Usage
questions
questions
• Blue boxes designate general survey areas
• White boxes indicate rating questions
• Embedded skips are shown with arrows
Have you
Have you
reported
reported
aa
problem?
problem?
Rate problem
Rate problem
resolution
resolution
Rate
Rate
documentation
documentation
Have you
Have you
requested
requested
assistance
assistance
from
from
customer
customer
services?
services?
Did you get
Did you get
help 11sttime?
help st time?
no
ACSI standard
ACSI standard
33questions
questions
Customer
Customer
Service
Service
questions
questions
ACSI outcomes
ACSI outcomes
22questions
questions
Rate customer
Rate customer
service
service
Thank you!
Thank you!
no

8. User background and interests
User background and interests
questions
questions
2011 EOSDIS Survey Overview
3996
3996
Have you
Have you
searched,
searched,
ordered,
ordered,
downloaded
downloaded
data?
data?
Search questions
Search questions
no
3673
3673
Delivery
Delivery
questions
questions
Did you look for
Did you look for
or get
or get
documentation
documentation
??
Documentation
Documentation
questions
questions
no
2954
2954
Did not search
Rate
Rate
search
search
Did not order
Rate
Rate
delivery
delivery
Format
Format
questions
questions
Order questions
Order questions
Rate
Rate
format
format
Rate
Rate
order
order
Usage
Usage
questions
questions
• Blue boxes designate general survey areas
• White boxes indicate rating questions
• Embedded skips are shown with arrows
Rate problem
Rate problem
resolution
resolution
Rate
Rate
documentation
documentation
Have you
Have you
requested
requested
assistance
assistance
from
from
customer
customer
services?
services?
Have you
Have you
reported
reported
aa
problem?
problem?
Did you get
Did you get
help 11sttime?
help st time?
no
ACSI standard
ACSI standard
33questions
questions
Customer
Customer
Service
Service
questions
questions
ACSI outcomes
ACSI outcomes
22questions
questions
Rate customer
Rate customer
service
service
Thank you!
Thank you!
no

9. NASA EOSDIS 2008 – 2011
Scores hold steady; no change more than one point
77
77
77
77
Customer Satisfaction
Index
86
86
85
84
81
80
81
81
Customer Support
Delivery
78
77
77
74
77
77
76
77
Product Quality
Product Selection
and Order
76
76
77
75
75
76
75
75
Product Documentation
Product Search
2011
=Significant Difference vs. 2010
2010
2009
2008
(+/-) 0.4
(+/-) 0.9
(+/-)
0.5
(+/-) 0.6
(+/-) 0.5
(+/-) 0.5
(+/-) 0.5

10. Product Quality
One-point gain from last year
78
77
Product Quality
77
74
78
77
Ease of using the data product
in the delivered format
77
74
2011
=Significant Difference vs. 2010
2010
2009
2008
Impact=0.4

11. Product Quality
Preferences somewhat in line with what provided
GeoTIFF is most preferred format, while HDF-EOS/HDF is format in which products were provided the most.
Only 8% of products provided in GIS although nearly one-quarter prefer that format.
In 2010, 57%
said products
were provided
in HDF-EOS
and HDF and
42% said they
were their
preferred
method.
Format data products were provided
HDF-EOS/HDF
NetCDF
Binary
ASCII
GeoTIFF
JPEG, GIF, PNG, TIFF
OGC Web services
GIS
KML, KMZ
CEOS
Don´t know
Other format
Number of Respondents
~Multiple responses allowed
53%
13%
9%
17%
41%
15%
1%
8%
5%
2%
4%
2%
3,673
Format preferred~
HDF-EOS/HDF
NetCDF
Binary
ASCII
GeoTIFF
JPEG, GIF, PNG, TIFF
OGC Web services
GIS
KML, KMZ
CEOS
OPeNDAP
Other preferred format
Number of Respondents
40%
20%
12%
24%
53%
18%
4%
23%
13%
2%
2%
3%
3,673

12. HDF-EOS/HDF Format
Tools used when data was provided in HDF format
Many of the respondents (687) selected ‘Other’ and listed alternate tool names or described custom
approaches. Of these respondents 69 selected 'other‘ exclusively.
Tools used with HDF
Number
%
867
818
493
509
512
506
163
73
123
144
438
109
42
96
303
1961
44%
42%
25%
26%
26%
26%
8%
4%
6%
22%
22%
6%
2%
5%
15%
ENVI
ArcGIS
ERDAS
IDL
MATLAB
MODIS Reprojection Tool
SeaDAS
Geomatica®
Global Mapper
IDRISI
HDFView
HEG
NCL
GrADS
Other (Please specify)
Number of HDF-EOS/HDF respondents
~Multiple responses allowed
2011 EOSDIS Survey Flow Overview CLB

13. Experience with HDF
Mostly high ratings but some “Ease of Use” problems
HDF Users Experience Ratings
700
Ease of Use
Quality of Product
Usability of Data
600
500
Over 60% of the
respondents rated
all three areas as
8, 9 or 10..
400
t
d
n
p
s
R
f
o
r
e
b
m
u
N
300
200
100
0
1
2
3
4
5
6
7
Ratings (10 = Excellent)
2011 EOSDIS Survey Flow Overview CLB
8
9
10

14. HDF User Comments
Comments are both positive and negative
• Survey respondents provided ~ 90 comments about their
experience with HDF format, for example pertaining to
– Search method
“I found all of the HDF-4 files I needed easily, and in small sizes too which was a
plus.”
– Order processing
“A mosaicking option for all data sets would be nice”
– Preferences
“Please no more HDF4 with irritating custom extensions”
– What they are not finding
“I need data in ASCII format . . . data from HDF is complicated”
– Looking for documentation
“Format Conversion (HDF to netcdf).”
– Over half were voluntary comments or suggestions
“ . . . size and complexity (HDF-format) of the data files . . . can be ameliorated
with web services . . . “
• Verbatim comments are available for analysis
2011 EOSDIS Survey Flow Overview CLB

15. Summary
 Satisfaction with NASA EOSDIS has held at
77 for four years. NASA continues to meet
data users needs.
 HDF-EOS/HDF is a well supported format
• Not all users are comfortable or satisfied with
HDF
• Comments received provide insight into users
effective use and/or problems
• Verbatim comments are supplied in separate
word documents.

16. Comments
Verbatim comments are supplied in separate word documents.
In what format(s) were your data products provided to you? (select any that apply)
•Other (please specify and/or comment)
Did you use software tool(s) to work with the data (e.g., format conversion, analysis,
visualization, etc.?)
•Yes (Please specify which tool or tools you used to work with the data.)
•No, I couldn’t find what I needed (please specify what you were looking for)
•No, I couldn’t understand how to use it (please specify what you were trying to use)
Do you have any additional comments or suggestion about possible improvements to data
products, services, tools, documentation, or the websites that you would like to share? Are
you finding what you need on our websites? (please comment)